MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

UNIT I DISTRIBUTED SYSTEMS 9

Introduction to Distributed Systems – Characterization of Distributed Systems – Distributed
Architectural Models –Remote Invocation – Request-Reply Protocols – Remote Procedure Call –
Remote Method Invocation – Group Communication – Coordination in Group Communication –
Ordered Multicast – Time Ordering – Physical Clock Synchronization – Logical Time and Logical
Clocks

Part A
1) What is a distributed system?
A distributed system is one in which components located at networked computers
communicate and coordinate their actions only by passing messages. The components
interact with each other in order to achieve a common goal.

2) Mention few examples of distributed systems.

Some examples of distributed systems are web search, Massively multiplayer online
games(MMOGs), Financial trading markets, SOA based systems etc.

3) Mention the trends in distributed systems.

Following are the trends in distributed systems:
Σ Emergence of pervasive networking technology
Σ Emergence of ubiquitous computing coupled with desire to support user
mobility in distributed systems
Σ Increasing demand for multimedia services
Σ The view of distributed systems as a utility.

4) What does the term remote invocation mean?

Remote invocation mechanism facilitates to create a distributed application. It provides a
remote communication using two objects stub and skeleton. In this client-server
approach remote object plays main role and it is an object whose method can be invoked
from another JVM. In the client side, stub acts as a gateway. In the server side, skeleton
acts as the gateway.

5) What is the role of middleware?

The term middleware applies to a software layer that provides a programming
abstraction as well as masking the heterogeneity of the underlying networks, hardware,
operating systems and programming languages. In addition to solving the problems of
heterogeneity, middleware provides a uniform computational model for use by the
programmers of servers and distributed applications.

6) What are the challenges of distributed systems?

The main challenges of distributed system are:

Mrs.J.Srivandhana.,AP/MCA Page 1
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

† Heterogeneity
† Openness
† Security
† Scalability
† Failure handling
† Concurrency
† Transparency
† Quality of service

7) What are the 2 security challenges that are not fully met by distributed systems?
 Denial of service attacks: One of the security problems is that a user may wish to
disrupt a service for some reason. This can be achieved by bombarding the service
with such a large number of pointless requests that the serious users are unable to use
it. This is called a denial serviceof attack. There have been several denial of service
attacks on well-known web services.
 Security of mobile code: Mobile code needs to be handled with care. Consider
someone who receives an executable program as an electronic mail attachment. The
possible effects of running the program are unpredictable.

8) When a system can be described as scalable in nature?

A system is described as scalable if it will remain effective when there is a significant
increase in the number of resources and the number of users. Distributed systems operate
effectively and efficiently at many different scales, ranging from a small intranet to the
Internet.

9) What are the challenges faced by a scalable distributed system?

 Controlling the cost of physical resources: As the demand for a resource grows, it
 should be possible to extend the system, at reasonable cost, to meet it.
 Controlling the performance loss: Consider the management of a set of data whose
 size is proportional to the number of users or resources in the system.
 Preventing software resources running out: An example of lack of scalability is
 shown by the numbers used as Internet (IP) addresses
 Avoiding performance bottlenecks: In general, algorithms should be decentralized
 to avoid having performance bottlenecks.
10) What are the techniques used for dealing failures in a distributed system.
Σ Detecting failures: Some failures can be detected. For example, checksums can be
used to detect corrupted data in a message or a file.
Σ Masking failures: Some failures that have been detected can be hidden or made less
severe.

Mrs.J.Srivandhana.,AP/MCA Page 2
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Σ Tolerating failures: Most of the services in the Internet do exhibit failures – it would
not be practical for them to attempt to detect and hide all of the failures that might
occur in such a large network with so many components.
Σ Recovery from failures: Recovery involves the design of software so that the state of
permanent data can be recovered or ‘rolled back’ after a server has crashed.
Σ Redundancy: Services can be made to tolerate failures by the use of redundant
components.

11) How the availability of a system can be measured?

The availability of a system is a measure of the proportion of time that it is available
for use. When one of the components in a distributed system fails, only the work that was
using the failed component is affected. A user may move to another computer if the one
that they were using fails; a server process can be started on another computer.

12) Define Transparency. What are its types?

Transparency is defined as the concealment from the user and the application
programmer of the separation of components in a distributed system, so that the system
is perceived as a whole rather than as a collection of independent components.
Its types are:
Σ Access transparency
Σ Location transparency
Σ Concurrency transparency
Σ Replication transparency
Σ Failure transparency
Σ Mobility transparency
Σ Performance transparency
Σ Scaling transparency
13) What are the non-functional properties of a system that affects its quality of
service?
The main nonfunctional properties of systems that affect the quality of the service
experienced by clients and users are reliability, security and performance. Adaptability to
meet changing system configurations and resource availability has been recognized as a
further important aspect of service quality.

14) Why HTTP called as request-reply protocol?

In case of HTTP protocol, the client sends a request message to the server containing the
URL of the required resource. The server looks up the path name and, if it exists, sends back
the resource’s content in a reply message to the client. Otherwise, it sends back an error
response such as the familiar ‘404 Not Found’. Hence it is called as request-reply protocol.

15) What are the issues in distributed system?

∑ There is no global time in a distributed system, so the clocks on different computers

do not necessarily give the same time as one another.

Mrs.J.Srivandhana.,AP/MCA Page 3
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

∑ All communication between processes is achieved by means of messages. Message

communication over a computer network can be affected by delays, can suffer from a variety
of failures and is vulnerable to security attacks.

16) What is a multicast operation?

A multicast operation is the mechanism where an operation sends a single message

from one process to each of the members of a group of processes, usually in such a way that
the membership of the group is transparent to the sender.

17) What are the design issues for RPC.

 The style of programming promoted by RPC programming with interfaces;

 The call semantics associated with RPC;
 The key issue of transparency and how it relates to remote procedure calls.

18) What is meant by Group communication?

Group communication offers a service whereby a message is sent to a group and then
this message is delivered to all members of the group. In this action, the sender is not aware
of the identities of the receivers.

19) Define Closed and open groups.

A group is said to be closed if only members of the group may multicast to it. A
process in a closed group delivers to itself any message that it multicasts to the group. A
group is open if processes outside the group may send to it.

20) What is meant by FIFO ordering?

First-in-First-out (FIFO) ordering (also referred to as source ordering) is concerned

with preserving the order from the perspective of a sender process, in that if a process sends
one message before another, it will be delivered in this order at all processes in the group.

21) What is meant by Causal ordering?

Causal ordering takes into account causal relationships between messages, in that if a
message happens before another message in the distributed system this so-called causal
relationship will be preserved in the delivery of the associated messages at all processes.

22) What is the use of middleware?

Middleware is a layer of software whose purpose is to mask heterogeneity and to

provide a convenient programming model to application programmers. Middleware is

Mrs.J.Srivandhana.,AP/MCA Page 4
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

represented by processes or objects in a set of computers that interact with each other to
implement communication and resource sharing support for distributed applications.

PART B
1) INTRODUCTION TO DISTRIBUTED SYSTEMS.
Definition:

“A distributed system is a collection of independent computers that appear to the users of

the system as a single computer.”

Characterization of Distributed Systems:

Introduction

 A distributed system is one in which components located at networked computers

communicate and coordinate their actions only by passing messages.
 A distributed system as one in which hardware or software components located at
networked computers communicate and coordinate their actions only by passing
messages.
 This simple definition covers the entire range of systems in which networked
computers can usefully be deployed.
 Computers that are connected by a network may be spatially separated by any
distance. They may be on separate continents, in the same building or in the same
room.

Our definition of distributed systems has the following significant consequences:

Concurrency: In a network of computers, concurrent program execution is the norm. I can

do my work on my computer while you do your work on yours, sharing resources such as
web pages or files when necessary. The capacity of the system to handle shared resources can
be increased by adding more resources (for example. computers) to the network.

No global clock: When programs need to cooperate they coordinate their actions by
exchanging messages. Close coordination often depends on a shared idea of the time at
which the programs’ actions occur. But it turns out that there are limits to the accuracy with
which the computers in a network can synchronize their clocks – there is no single global
notion of the correct time. This is a direct consequence of the fact that the only
communication is by sending messages through a network.

Mrs.J.Srivandhana.,AP/MCA Page 5
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Independent failures: All computer systems can fail, and it is the responsibility of system
designers to plan for the consequences of possible failures. Distributed systems can fail in
new ways. Faults in the network result in the isolation of the computers that are connected
to it, but that doesn’t mean that they stop running.

In fact, the programs on them may not be able to detect whether the network has failed or
has become unusually slow. Similarly, the failure of a computer, or the unexpected
termination of a program somewhere in the system (a crash), is not immediately made
known to the other components with which it communicates. Each component of the system
can fail independently, leaving the others still running.

Examples of Distributed systems

To place distributed systems in a realistic context through examples: the Internet, an intranet
and mobile computing.

1. The Internet (Figure 1) :

 A vast interconnected collection of computer networks of many different types.
 Passing message by employing a common means of communication (Internet
Protocol).
 The web is not equal to the Internet.

2. Intranets (Figure 2):

 An intranet is a private network that is contained within an enterprise.

Mrs.J.Srivandhana.,AP/MCA Page 6
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

 It may consist of many interlinked local area networks and also use leased lines
in the Wide Area Network.
 It separately administrated and enforces local security policies.

Intranet by preventing unauthorized messages leaving or entering
 Some are isolated from the Internet
 Users in an intranet share data by means of file services.

3. Mobile and Ubiquitous Computing (Figure 1.3)

a. Distributed systems techniques are equally applicable to mobile computing involving
laptops, PDAs and wearable computing devices.

b. Mobile computing (nomadic computing) - perform of computing tasks while moving

(nomadic computing)

c. Ubiquitous computing - small computers embedded in appliances

i. harness of many small, cheap computation devices

ii. It benefits users while they remain in a single environment such as home.
Distributed In Figure 3 user has access to three forms of wireless connection:

d. A laptop is connected to host's wireless LAN.

e. A mobile (cellular) phone is connected to Internet using Wireless Application Protocol

(WAP) via a gateway.

Mrs.J.Srivandhana.,AP/MCA Page 7
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

f. A digital camera is connected to a printer over an infra-red link.

2) CHARACTERISTICS OF DISTRIBUTED SYSTEMS.

Characterization of Distributed Systems:
Introduction
A distributed system is one in which components located at networked
computers communicate and coordinate their actions only by passing
messages.

A distributed system as one in which hardware or software components

located at networked computers communicate and coordinate their
actions only by passing messages.

This simple definition covers the entire range of systems in which

networked computers can usefully be deployed.

Computers that are connected by a network may be spatially separated by

any distance. They may be on separate continents, in the same building or
in the same room. Our definition of distributed systems has the following
significant consequences:
Concurrency:

Mrs.J.Srivandhana.,AP/MCA Page 8
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

In a network of computers, concurrent program execution is the norm. I

can do my work on my computer while you do your work on yours,
sharing resources such as web pages or files when necessary. The capacity
of the system to handle shared resources can be increased by adding more
resources (for example. computers) to the network.
No global clock:
When programs need to cooperate they coordinate their actions by
exchanging messages.
Independent failures:
All computer systems can fail, and it is the responsibility of system
designers to plan for the consequences of possible failures. Distributed
systems can fail in new ways.

Faults in the network result in the isolation of the computers that are
connected to it, but that doesn’t mean that they stop running. In fact, the
programs on them may not be able to detect whether the network has
failed or has become unusually slow.
Similarly, the failure of a computer, or the unexpected termination of a
program somewhere in the system (a crash), is not immediately made
known to the other components with which it communicates. Each
component of the system can fail independently, leaving the others still
running.

3) Distributed Architectural Models

Mrs.J.Srivandhana.,AP/MCA Page 9
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Mrs.J.Srivandhana.,AP/MCA Page 10
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Mrs.J.Srivandhana.,AP/MCA Page 11
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Mrs.J.Srivandhana.,AP/MCA Page 12
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Mrs.J.Srivandhana.,AP/MCA Page 13
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Mrs.J.Srivandhana.,AP/MCA Page 14
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Mrs.J.Srivandhana.,AP/MCA Page 15
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Mrs.J.Srivandhana.,AP/MCA Page 16
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

4) Remote Invocation – Request-Reply Protocols – Remote Procedure

Call – Remote Method Invocation

Remote Invocation
1. Introduction
2. Remote Method Invocation (RMI)
3. RMI Invocation Semantics

Remote Invocation
• RPC (Remote Procedure Call)
‣ allows client programs to call procedures in server programs running in
separate processes (and generally in different computers from the client)
• RMI (Remote Method Invocation)

‣ Extension of local method invocation of object-oriented programming

allows an object living in one process to invoke the methods of an object living in
another process. Most famous example: Java RMI

Remote Method Invocation (RMI)

Let Us Start from Scratch: the Object Model
An object-oriented program (Java, C++, ...) consists of a collection of interacting objects, each
of which consists of a set of data and a set of methods
• An object can communicate with other objects by invoking their methods, generally passing
arguments and receiving results (request/reply protocol)

Mrs.J.Srivandhana.,AP/MCA Page 17
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

• Objects can encapsulate their data and the code of their methods
• Some languages (JAVA, C++) allow programmers to define objects whose instance variables
can be accessed directly
• BUT in a distributed object system, an object’s data should be accessible only via its
methods (or interface)

Actions in the Object Model

• An action in an object-oriented program is initiated by an object invoking a method in
another object
• The receiving object executes the appropriate method and then returns control to the
invoking object, sometimes supplying a result
• An invocation of a method can have 3 possible effects:
‣ the state of the receiver may be changed
‣ a new object may be instantiated (i.e., by using a constructor in Java)
‣ Further invocations on methods in other objects may take place
How to extend the “traditional” object model to make it applicable to distributed
systems?

Mrs.J.Srivandhana.,AP/MCA Page 18
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Mrs.J.Srivandhana.,AP/MCA Page 19
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Mrs.J.Srivandhana.,AP/MCA Page 20
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Mrs.J.Srivandhana.,AP/MCA Page 21
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Mrs.J.Srivandhana.,AP/MCA Page 22
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Mrs.J.Srivandhana.,AP/MCA Page 23
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

† Remote Procedure Call (RPC) Distributed operating systems are typically structured
around the client-server paradigm. In this model, a user process, called the client,
requests another user process, called the server, to perform some work for it by
sending the server a message and then blocking until the server sends back a reply.
† The communication mechanism used to implement the client-server model is called
RPC. The RPC abstraction lets the programmer think in terms of normal procedure
calls, which are well understood and have been around for a long time.
† This is in sharp contrast with, for example, the ISO OSI model. In this model,
communication is treated as an input/output device, with user primitives for sending
messages and getting indications of message arrivals. Many people think that
input/output should not be the central abstraction of a modern programming
language. Therefore, most distributed system builders, language designers, and
programmers prefer RPC.

Mrs.J.Srivandhana.,AP/MCA Page 24
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

5) Group Communication in distributed

Systems
Communication between two processes in a distributed system is required to
exchange various data, such as code or a file, between the processes. When one
source process tries to communicate with multiple processes at once, it is
called Group Communication.

A group is a collection of interconnected processes with abstraction. This

abstraction is to hide the message passing so that the communication looks like a
normal procedure call. Group communication also helps the processes from
different hosts to work together and perform operations in a synchronized manner,
therefore increases the overall performance of the system.

Mrs.J.Srivandhana.,AP/MCA Page 25
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Types of Group Communication in a Distributed System :

 Broadcast Communication :

When the host process tries to communicate with every process in a distributed
system at same time. Broadcast communication comes in handy when a common
stream of information is to be delivered to each and every process in most
efficient manner possible.
Since it does not require any processing whatsoever, communication is very fast
in comparison to other modes of communication. However, it does not support a
large number of processes and cannot treat a specific process individually.

A broadcast Communication: P1 process communicating with every process in the system

 Multicast Communication :

Mrs.J.Srivandhana.,AP/MCA Page 26
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

When the host process tries to communicate with a designated group of

processes in a distributed system at the same time. This technique is mainly used
to find a way to address problem of a high workload on host system and
redundant information from process in system. Multitasking can significantly
decrease time taken for message handling.

A multicast Communication: P1 process communicating with only a group of the process in the system

 Unicast Communication :

When the host process tries to communicate with a single process in a distributed
system at the same time. Although, same information may be passed to multiple
processes. This works best for two processes communicating as only it has to treat
a specific process only. However, it leads to overheads as it has to find exact
process and then exchange information/data.

A broadcast Communication: P1 process communicating with only P3 process

Mrs.J.Srivandhana.,AP/MCA Page 27
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

6) CO-ORDINATION IN GROUP COMMUNICATION AND ORDERED

MULTICAST:
 A group is an operating system abstraction for a collective of related processes.
 A set of cooperative processes may, for example, form a group to provide an extendable,
efficient, available and reliable service.
 The group abstraction allows member processes to perform computation on different hosts
while providing support for communication and synchronisation between them.

The term multicast means the use of a single communication primitive to send a message to a
specific set of processes rather than using a collection of individual point to point message
primitives. This is in contrast with the term broadcast which means the message is addressed to
every host or process.
Multicast
• 1 process sends a message to a group of other processes
• More efficient than looping in the application sending a message to each
• May be able to guarantee that either all recipients get the message or none does.
A consensus protocol allows a group of participating processes to reach a common decision, based
on their initial inputs, despite failures.

Mrs.J.Srivandhana.,AP/MCA Page 28
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

A reliable multicast protocol allows a group of processes to agree on a set of messages

received by the group. Each message should be received by all members of the group or by
none. The order of these messages may be important for some applications. A reliable
multicast protocol is not concerned with message ordering, only message delivery
guarantees.

Ordered delivery protocols can be implemented on top of a reliable multicast service.

Multicast algorithms can be built on top of lower-level communication primitives such as

point-to-point sends and receives or perhaps by availing of specific network mechanisms
designed for this purpose.

The management of a group needs an efficient and reliable multicast communication

mechanism to allow clients obtain services from the group and ensure consistency among
servers in the presence of failures. Consider the following two scenarios:-

 A client wishes to obtain a service which can be performed by any member of the
group without affecting the state of the service.

Mrs.J.Srivandhana.,AP/MCA Page 29
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

 A client wishes to obtain a service which must be performed by each member of

the group.
In the first case, the client can accept a response to its multicast from any member of the
group as long as at least one responds. The communication system need only guarantee
delivery of the multicast to a nonfaulty process of the group on a best-effort basis.

In the second case, the all-or-none atomic delivery requirements requires that the multicast
needs to be buffered until it is committed and subsequently delivered to the application
process, and so incurs additional latency.

Failure may occur during a multicast at the recipient processes, the communication links or
the originating process.

Failures at the recipient processes and on the communication links can be detected by the
originating process using standard time-out mechanisms or message acknowledgements.
The multicast can be aborted by the originator, or the service group membership may be
dynamically adjusted to exclude the failed processes and the multicast can be continued.

If the originator fails during the multicast, there are two possible outcomes. Either the
message has not have arrived at any destination or it has arrived at some. In the first case,
no process can be aware of the originator's intention and so the multicast must be aborted.
In the second case it may be possible to complete the multicast by selecting one of the
recipients as the new originator. The recipients would have to buffer messages until safe
for delivery in case they were called on for this role.

A reliable multicast protocol imposes no restriction on the order in which messages are
delivered to group processes. Given that multicasts may be in progress by a number of
originators simultaneously, the messages may arrive at different processes in a group in
different orders. Also, a single originator may have a number of simultaneous multicasts in
progress or may have issued a sequence of multicast messages whose ordering we might
like preserved at the recipients. Ideally, multicast messages should be delivered
instantaneously in the real-time order they were sent, but this is unrealistic as there is no
global time and message transmission has a possibly significant and variable latency.

A number of possible scenarios are given below which may require different levels of
ordering semantics. G and s represent groups and message sources. s may be inside or
outside a group. Note that group membership may overlap with other groups, that is,
processes may be members of more than one group.

Mrs.J.Srivandhana.,AP/MCA Page 30
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

(a) (b)

(d)

Ordered Reliable Multicasts

A FIFO ordered protocol guarantees that messages by the same sender are delivered in the
order that they were sent. That is, if a process multicasts a message m before it multicasts a
message m', then no correct process receives m' unless it has previously received m. To implement
this, messages can be assigned sequence numbers which define an ordering on messages
from a single source. Some applications may require the context of previously multicast
messages from an originator before interpreting the originator's latest message correctly.

However, the content of message m may also depend on messages that the sender of m
received from other sources before sending m. The application may require that the context
which could have caused or affected the content of m be delivered at all destinations of m,

Mrs.J.Srivandhana.,AP/MCA Page 31
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

before m. For example, in a network news application, user A broadcasts an article. User B
at a different site receives the article and broadcasts a response. User C can only interpret
the response if the original article is delivered first at their site. Two messages are said to
be causally related if one message is generated after receipt of the other. Causal order is a
strengthening of FIFO ordering which ensures that a message is not delivered until all
messages it depends on have been delivered.

This causal dependence relation is more formally specified as follows:- An execution of

a multicast or receive primitive by a process is called an event.
Event e causally precedes event f (i.e. happened before), (e®f), if an only if:
1. A process executes both e and f in that order, or
2. e is the multicast of message m and f is the receipt of m, or
3. there is an event h, such that e®h and h®f.
A causal protocol then guarantees that if the broadcast of message m causally precedes the
broadcast of m', then no correct process receives m' unless it has previously received m.

The definition of causal ordering does not determine the delivery order of messages which
are not causally related. Consider a replicated database application with two copies of a
bank account x residing at different sites. A client side process at one site sends a multicast
to the database to lodge £100 to account x. At another site simultaneously, a client side
process initiates a multicast to add 10% interest to the current balance of x. For consistency,
all database replicas should apply the two updates in the same sequence. As these two
messages are not causally related, a causal broadcast would allow the update messages to x
to be delivered in different sequences at the replicas.

Total Ordering guarantees that all correct processes receive all messages in the same order.
That is, if correct processes p and q both receive messages m and m', then p receives m before m' if
and only if q receives m before m'. The multicast is atomic across all members of the group.

Note that this definition of a totally ordered broadcast does not require that messages be
delivered in Causal Order or even FIFO Order, so it is not stronger than these orderings.
For example, if a process suffers a transient failure during the broadcast of message m, and
subsequently broadcasts m', a totally ordered protocol will guarantee only that processes
receive m'.
FIFO or Causal Ordering can be combined with Total Ordering if required.

All reliable multicasts have the following three properties.

Validity: If a correct process multicasts a message m, then all correct processes
eventually receive m.
Agreement: If a correct process receives a message m, then all correct processes
eventually receive m.
Integrity: For any message m, every correct process receives m at most once and
only if m was multicast by the sender of m.

Mrs.J.Srivandhana.,AP/MCA Page 32
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

The protocols only differ in the strength of their message delivery order requirements.

Multicast Algorithms

In the algorithms to follow, R stands for Reliable Multicast, F for FIFO, C for Causal and A
for Atomic.
In an asynchronous system where a reliable link exists between every pair of processes, the
algorithm below demonstrates how a Reliable multicast can be achieved.

Every process p executes the following:-

multicast(R, m):
tag m with sender(m) and seq#(m)
send(m) to all group including p

The receive(R, m) occurs as follows:

upon arrival(m) do
if p has not previously executed receive(R, m) then
if sender(m) <> p then
send(m) to all group
receive(R, m)

It is easy to use Reliable Multicast to build a FIFO Multicast algorithm. To F-multicast a

message m, a process q simply R-multicasts m. The difference is at the receiver which
orders the delivery.

multicast(F, m):
multicast(R, m)

If m is the ith message from q, then m is tagged sender(m)=q and seq#(m)=i.

For each q, every process p maintains a counter next[q] that indicates the sequence number
of the next F-multicast from q that p is willing to F-deliver. Incoming messages are placed
in a message bag from which messages that can be FIFO delivered (according to the value
of next[q]) are removed.

Every process p initialises:-

msgbag = Æ
next[q] = 1 for all q

The receive(R, m) occurs as follows:

upon arrival(R, m) do
q := sender(m);
msgbag := msgbag È {m};

Mrs.J.Srivandhana.,AP/MCA Page 33
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

while ($ m'Î msgbag: sender(m')=q and seq#(m')=next[q]) do

receive(F, m')
next[q] = next[q]+1;
msgbag := msgbag - {m'}

A Causal multicast algorithm can be constructed on top of a FIFO multicast.

To C-multicast a message m, a process p uses the FIFO multicast algorithm to F-multicast
the sequence of messages that p has causally received since its previous C-broadcast
followed by the message m. A process q receives all of the messages in the sequence only
which it previously has not received.

Initialisation:
prevReceives = ^

multicast(C, m):
multicast(F, <prevReceives || m>) /* || is list concatenation operator */
prevReceives = ^

The receive(C, m) occurs as follows:

upon arrival(F, <m1,m2,…mn>) do
for i := 1 to n do
if p has not received mi then
receive(C, mi)
prevReceives := prevReceives || mi;

One of the fundamental results about fault-tolerant distributed computing is the

impossibility of achieving consensus in asynchronous systems which suffer crash failures.
This is primarily due to the fact that it is impossible to distinguish between a process that
has crashed and one that is extremely slow. The consensus problem can be easily reduced
to implementing atomic multicast. For example, to propose a value, a process A-multicasts
it. To decide on a value, each process picks the value of the first message that is A-received.
By total order of atomic broadcasts, all processes agree on the same value. Consensus and
Atomic Multicast are therefore equivalent in asynchronous systems with crash failures and
so there are no deterministic Atomic Multicast protocols (total ordering protocols) for such
systems.
Atomic multicast can be deterministically implemented in synchronous systems where
upper bounds on message latency exist.

If we assume no failures, the algorithm given below can be used to implement a totally
ordered multicast.

Each site maintains a 'local clock'. A clock doesn't necessarily have to supply the exact
time, but could be implemented simply by a counter which is incremented after

Mrs.J.Srivandhana.,AP/MCA Page 34
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

each send or receive event that occurs at the site, so that successive events have different
'times'.

The algorithm executes in two phases. During the first phase the originator multicasts the
message to all destinations and awaits a reply from each. Each receiver queues the message
and assigns it a proposed timestamp based on the value of its local clock. This timestamp is
returned to the originator. The originator collects all the proposed timestamps for the
multicast and selects the highest. During the second phase of the algorithm, the originator
commits the multicast by sending the final chosen timestamp to all destinations. Each
receiver then marks the message as deliverable in its queue. The message queue is ordered
on the value of the timestamps associated with each message each time a timestamp is
updated. When a message gets to the top of the queue and is deliverable it may be
delivered immediately to the application.

As an example, consider three simultaneous multicasts, m1, m2 and m3 received in arbitrary

order by three sites 1, 2 and 3. The local time at Site1 is initially = 15, the local time at Site2
is initially=16 and the local time at Site3 is initially = 17. To make timestamps unique they
can be generated by concatenating the local time at a site with the unique name of the site.

Initially the messages arrive in the following order at the sites:

Site 1 Site 2 Site 3

Each site returns the timestamps it assigned the respective messages to their originators.
Let's say the originator of m1 is the first to get all replies and issues a commit with
timestamp = 17.3.
Site 1 Site 2 Site 3

Message m1 is now
deliverable at Site3.
If a commit for message m2 should arrive at the sites with timestamp = 19.3 we get:-
Site 1 Site 2 Site 3

Mrs.J.Srivandhana.,AP/MCA Page 35
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Message m1 is now deliverable at Site2.

If a commit for message m3 should arrive at the sites with timestamp = 18.3 we get:-

Site 1 Site 2 Site 3

Consider the costs of achieving totally ordered message delivery:-

Latency Cost: Each message which arrives at a site must wait in a queue for a commit and
until all messages with earlier timestamp in the queue are delivered.

Bandwidth Cost: The protocol involves the exchange of several messages:- The original
multicast, all the individual timestamp messages returned by the recipients and the final
commit multicast.

Storage Cost: Messages are stored in a queue from time of arrival until time of delivery. This
time may be longer if reliable delivery is required to all destinations as a site may have to
take the role of initiator.

The causal multicast generally incurs less latency cost as messages can be delivered without having
to wait for a commit. The size of each message may be larger due to the forwarding of causal
dependent context with the message. The effect of this is not too severe however as with most
systems it is often the additional number of messages and not their size which is

Mrs.J.Srivandhana.,AP/MCA Page 36
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

7) Time Ordering

Time in Distributed Systems: Clocks and Ordering of Events

Clocks in Distributed Systems

ordering of updates at different replicas, ordering of multicast messages sent in a group)

some fixed duration of each other (Ex:

Replay of stolen messages in distributed authentication protocols like Kerberos)

networks, sleep/wakeup scheduling)

obally synchronized clock is available, but

bounds may be enough

s
synchronization of the order of minutes

ocal clock

ization is accurate within bound ∂

Mrs.J.Srivandhana.,AP/MCA Page 37
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

9) Physical Clock Synchronization

Mrs.J.Srivandhana.,AP/MCA Page 38
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Mrs.J.Srivandhana.,AP/MCA Page 39
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Mrs.J.Srivandhana.,AP/MCA Page 40
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Handling message delay: try to estimate the time the

message with the timer server’s time took to reach the client
 Measure round trip time and halve it
 Make multiple measurements of round trip time, discard too high values,
take average of rest
 Make multiple measurements and take minimum
 Use knowledge of processing time at server if known to eliminate it from
delay estimation (How to know?)

Handling fast clocks Do not set clock backwards; slow it down

over a period of time to bring in tune with server’s clock
Ex: increase the software clock every two interrupts instead of one

Mrs.J.Srivandhana.,AP/MCA Page 41
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Time server can handle faulty clocks by eliminating client clock values that
are too low or too high
What if the time server fails? Just elect another node as the time server (Leader
Election Problem)
Note that the actual time of the central server does not matter, enough for it
to tick at around the same rate as other clocks to compute average correctly
(why?)

Mrs.J.Srivandhana.,AP/MCA Page 42
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Cannot be used for external synchronization

Works well in small LANs only for the same reason as Cristian’s

External Synchronization with Real Time

 Clocks must be synchronized with real time

 But what is “real time” anyway?

Measurement of Time
Astronomical Time
Traditionally used
 Based on earth’s rotation around its axis and around the sun
 Solar day : interval between two consecutive transits of the sun
 Solar second : 1/86,400 of a solar day
 Period of earth’s rotation varies, so solar second is not stable
 Mean solar second : average length of large no of solar days, then
divide by 86,400
Atomic Time
 Based on the transitions of Cesium 133 atom
 1 sec. = time for 9,192,631,770 transitions
 Many labs worldwide maintain a network of atomic clocks
 International Atomic Time (TAI) : mean no. of ticks of the clocks
since Jan 1, 1958
 Highly stable
 But slightly off-sync with mean solar day
 A leap second inserted occasionally to bring it in sync.

Mrs.J.Srivandhana.,AP/MCA Page 43
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

 Resulting clock is called UTC – Universal Coordinated Time

UTC time is broadcast from different sources around the world, ex.
National Institute of Standards & Technology (NIST) – runs WWV radio
station, anyone with a proper receiver can tune in
United States Naval Observatory (USNO) – supplies time to all
defense sources
National Physical Laboratory in UK
Satellites
Many others
Accuracies can vary (< 1 milliseconds to a few milliseconds)

NTP : Network Time Protocol

Cristian’s method and Berkeley algorithm are primarily for
Intranets. The Network Time Protocol(NTP) defines a time service to
distribute time information over the Internet.
Clients across the Internet to be synchronized accurately to UTC.
Statistical techniques
Reliable service that can survive lengthy losses of connectivity.
Redundant servers and redundant paths between servers.
Clients resynchronized sufficiently frequently to offset the rates of
drift.
Protection against interference with time services. Authentication
technique from claimed trusted sources.

Mrs.J.Srivandhana.,AP/MCA Page 44
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Mrs.J.Srivandhana.,AP/MCA Page 45
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

10) Logical Time and Logical Clocks

Mrs.J.Srivandhana.,AP/MCA Page 46
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Mrs.J.Srivandhana.,AP/MCA Page 47
 MC5207 UNIT-1 CLOUD COMPUTING TECHNOLOGIES I MCA- II SEM

Mrs.J.Srivandhana.,AP/MCA Page 48