DISTRIBUTED COMPUTING

Iftikhar Hussain Babur

System Analyst
Quaid-e-Azam Academy for Educational Development
Govt. of the Punjab, Wahdat Road Lahore
[email protected]

Superior University, Gold Campus

 Types of Distributed Systems
•Distributed Computing Systems

Distributed systems used for high-performance computing task

1. Cluster computing - the underlying hardware consists of a
collection of similar workstations or PCs, closely connected by
means of a high-speed local-area network, each node runs the
same operating system.
2. Grid Computing - constructed as a federation of computer
systems, where each system may fall under a different
administrative domain, and may be very different when it
comes to hardware, software, and deployed network
technology.
Cluster Computing Systems
• price/performance ratio of personal computers and workstations
became financially and technically attractive to build a
supercomputer using off-the-shelf technology by simply hooking
up a collection of relatively simple computers in a high-speed
network.
• In virtually all cases, cluster computing is used for parallel
programming in which a single (compute intensive) program is run
in parallel on multiple machines.
• example of a cluster computer - Linux-based Beowulf clusters
• Each cluster consists of a collection of compute nodes that are
controlled and accessed by means of a single master node.
Cluster Computing Systems
• master node:
• handles the allocation of nodes to a particular parallel program
• maintains a batch queue of submitted jobs
• provides an interface for the users of the system
• the master runs the middleware needed for the execution of
programs and management of the cluster
• middleware is formed by the libraries for executing parallel
programs.
• many of these libraries effectively provide only advanced
message-based communication facilities, but are not capable of
handling faulty processes, security, etc.
• compute nodes - often need nothing else but a standard operating
system.
Cluster Computing Systems
An example of a cluster computing system.
Cluster Computing Systems
• Alternative to hierarchical organization - symmetric approach is
followed in the MOSIX system
• MOSIX attempts to provide a single-system image of a cluster - to
a process, a cluster computer appears to be a single computer.
• providing such an image under all circumstances is impossible.
• In MOSIX, the high degree of transparency is provided by allowing
processes to dynamically and preemptively migrate between the
nodes that make up the cluster.
• Process migration allows a user to start an application on any
node (referred to as the home node), after which it can
transparently move to other nodes, for example, to make efficient
use of resources.
Grid Computing Systems
• Grid computing systems have a high degree of
heterogeneity: no assumptions are made concerning
hardware, operating systems, networks, administrative
domains, security policies, etc.
• Key issue in a grid computing system - resources from
different organizations are brought together to allow the
collaboration of a group of people or institutions - forms
a virtual organization.
• Members of the same virtual organization have access
rights to the resources that are provided to that
organization.
Grid Computing Systems
• Resources consist of compute servers (including
supercomputers, possibly implemented as cluster
computers), storage facilities, and databases. In
addition, special networked devices such as telescopes,
sensors, etc., can be provided as well.
• Software required for grid computing evolves around
providing access to resources from different
administrative domains, and to only those users and
applications that belong to a specific virtual
organization.
• A layered architecture for grid computing systems.
Grid Computing Systems-Four layer architecture
Grid Computing Systems-Four layer architecture
1. fabric layer - provides interfaces to local resources at a specific
site.
• interfaces are tailored to allow sharing of resources within a virtual
organization.
• provide functions for querying the state and capabilities of a
resource, along with functions for actual resource management
(e.g., locking resources).
2a. connectivity layer - consists of communication protocols for
supporting grid transactions that span the usage of multiple
resources.
• contains security protocols to authenticate users and resources.
• in many cases human users are not authenticated - programs
acting on behalf of the users are authenticated.
Grid Computing Systems-Four layer architecture
2b. resource layer - responsible for managing a single resource.
• uses functions provided by the connectivity layer and calls directly
the interfaces made available by the fabric layer.
• responsible for access control, and hence will rely on the
authentication performed as part of the connectivity layer.
3. collective layer - handles access to multiple resources
• consists of services for resource discovery, allocation and
scheduling of tasks onto multiple resources, data replication, etc.
• may consist of many different protocols for many different
purposes, reflecting the broad spectrum of services it may offer to
a virtual organization.
4. application layer - consists of the applications that operate within
a virtual organization and which make use of the grid computing
environment.
Grid Computing Systems
Grid Middleware layer - collective, connectivity, and
resource layers.
• provide access to and management of resources that are
potentially dispersed across multiple sites.
• shift toward a service-oriented architecture in which
sites offer access to the various layers through a
collection of Web services
• led to the definition of an alternative architecture known
as the Open Grid Services Architecture (OGSA).
• consists of various layers and many components, making
it rather complex.
Distributed Information Systems
• Evolved in organizations that were confronted with a wealth of
networked applications, but for which interoperability turned out
to be problematic.
• Many of the existing middleware solutions are the result of
working with an infrastructure in which it was easier to integrate
applications into an enterprise-wide information system.
Several levels at which integration took place:
1. a networked application simply consisted of a server running that
application (often including a database) and making it available to
remote programs, called clients.
• such clients could send a request to the server for executing a
specific operation, after which a response would be sent back.
Integration at the lowest level would allow clients to wrap a
number of requests, possibly for different servers, into a single
larger request and have it executed as a distributed transaction.
Distributed Information Systems
• The key idea was that all, or none of the requests would be
executed.
2. As applications became more sophisticated and were gradually
separated into independent components (notably distinguishing
database components from processing components), it became
clear that integration should also take place by letting applications
communicate directly with each other.
Results: two forms of distributed systems
• transaction processing systems
• enterprise application integration (EAI)
 Transaction Processing System
• Focus on database applications - operations on a database are
usually carried out in the form of transactions.
• Programming using transactions requires special primitives that
must either be supplied by the underlying distributed system or by
the language runtime system.
• Example primitives for transactions

Primitive Description
BEGIN_TRANSACTION Mark the start of a transaction
END_TRANSACTION Terminate the transaction and try to commit
ABORT_TRANSACTION Kill the transaction and restore the old values
READ Read data from a file, a table, or otherwise
WRITE Write data to a file, a table, or otherwise
Transaction Processing System
• Exact list of primitives depends on what kinds of objects are being
used in the transaction.
e.g. mail system - may be primitives to send, receive, and forward
mail.
e.g. accounting system - may be different.
• Ordinary statements, procedure calls, etc., are allowed inside a
transaction.
• Remote procedure calls (RPCs) procedure calls to remote servers,
are often also encapsulated in a transaction, leading to what is
known as a transactional RPC.

Properties of transactions (ACID):

Atomic
To the outside world, the transaction happens indivisibly.

• ensures that each transaction either happens completely, or not at

all
• if it happens, it happens in a single indivisible, instantaneous
action.
• while a transaction is in progress, other processes (whether or not
they are themselves involved in transactions) cannot see any of
the intermediate states.
Consistent
The transaction does not violate system invariants.

• if the system has certain invariants that must always hold, if they
held before the transaction, they will hold afterward too

e.g. a banking system - a key invariant is the law of conservation of

money. After every internal transfer, the amount of money in the
bank must be the same as it was before the transfer
Isolated
Concurrent transactions do not interfere with each other.

• transactions are isolated or serializable

• if two or more transactions are running at the same time, to each

of them and to other processes, the final result looks as though all
transactions ran sequentially in some (system dependent) order.
Durable
Once a transaction commits, the changes are permanent.

• once a transaction commits, no matter what happens, the

transaction goes forward and the results become permanent.

• no failure after the commit can undo the results or cause them to
be lost

• A nested transaction is constructed from a number of sub

transactions.
Sub transactions
• The top-level transaction may
fork off children that run in
parallel with one another, on
different machines, to gain
performance or simplify
programming.

• Each of these children may also

execute one or more sub
transactions, or fork off its own
children.
Problem of Sub Transactions
Permanence applies only to top-level transactions.

e.g.
• a transaction starts several sub transactions in parallel, and one of
these commits, making its results visible to the parent transaction.
• after further computation, the parent aborts, restoring the entire
system to the state it had before the top-level transaction started
• the results of the sub transaction that committed must
nevertheless be undone.
Problem of Sub Transactions
Solution:

When any transaction or sub transaction starts, it is conceptually

given a private copy of all data in the entire system for it to
manipulate as it wishes.
• If it aborts, its private universe just vanishes, as if it had never
existed.
• If it commits, its private universe replaces the parent's universe.
• Thus if a sub transaction commits and then later a new sub
transaction is started, the second one sees the results produced by
the first one. Likewise, if an enclosing (higher-level) transaction
aborts, all its underlying sub transactions have to be aborted as
well.
Sub Transaction
 Sub Transaction
• Nested transactions provide a natural way of distributing a
transaction across multiple machines.
• They follow a logical division of the work of the original transaction.
• Each sub transaction can be managed separately and independently.
• Early enterprise middleware systems handled distributed (or nested)
transactions using a transaction processing monitor or TP monitor for
integrating applications at the server or database level.
• Its main task was to allow an application to access multiple
server/databases by offering it a transactional programming model
• The role of a TP monitor in distributed systems.
Enterprise Application Integration
• The more applications became decoupled from the databases they
were built upon, the more evident it became that facilities were
needed to integrate applications independent from their
databases.
• Application components should be able to communicate directly
with each other and not merely by means of the request/reply
behavior that was supported by transaction processing systems.
• Result: This need for inter-application communication led to many
different communication models
Enterprise Application Integration
Middleware as a communication facilitator in enterprise application
integration.
Several types of communication middleware
1. Remote procedure calls (RPC) -
• an application component can send a request to another
application component by doing a local procedure call, which
results in the request being packaged as a message and sent to the
callee.
• the result will be sent back and returned to the application as the
result of the procedure call.
2. Remote method invocations (RMI) –
• An RMI is the same as an RPC, except that it operates on objects
instead of applications.
Several types of communication middleware
Problems with RPC and RMI:
• The caller and callee both need to be up and running at the time
of communication.
• They need to know exactly how to refer to each other.
Results:
• Message-oriented middleware (MOM) - applications send
messages to logical contact points, often described by means of a
subject.
• Publish/subscribe systems - applications can indicate their interest
for a specific type of message, after which the communication
middleware will take care that those messages are delivered to
those applications.
Distributed Pervasive Systems
Above distributed systems characterized by their stability:
• nodes are fixed and have a more or less permanent and high-
quality connection to a network.
Mobile and embedded computing devices:
• instability is the default behavior
Distributed pervasive system - is part of the surroundings ->
inherently distributed.
Features:
• general lack of human administrative control
• devices can be configured by their owners
• they need to automatically discover their environment and fit in as
best as possible. This nestling in has been made more precise by
Grimm et al.(2004) by formulating the following
Distributed Pervasive Systems
Three requirements for pervasive applications (Grimm et al. 2004) :
Embrace contextual changes.
• a device must be continuously be aware of the fact that its
environment may change all the time
Encourage ad hoc composition.
• many devices in pervasive systems will be used in very different
ways by different users – make it easy to configure the suite of
applications running on a device
Recognize sharing as the default.
• devices generally join the system in order to access (and possibly
provide) information
Distributed Pervasive Systems
Examples:
Home Systems
• built around home networks
• consist of one or more personal computers
• integrate consumer electronics such as TVs, audio and video
equipment, gaming devices, (smart) phones, PDAs, and other
personal wearables into a single system.
• Now/Soon: all kinds of devices such as kitchen appliances,
surveillance cameras, clocks, controllers for lighting, and so on, will
all be hooked up into a single distributed system.
Distributed Pervasive Systems
Several challenges:
• System should be completely self-configuring and self-managing
e.g. Universal Plug and Play (UPnP) standards by which devices
automatically obtain IP addresses, can discover each other, etc.
(UPnP Forum).
• Unclear how software and firmware in devices can be easily
updated without manual intervention, or when updates do take
place, that compatibility with other devices is not violated.
Managing "personal space."
• A home system consists of many shared as well as personal
devices
• Data in a home system is also subject to sharing restrictions.
Self Reading / Research Task

Examples of distributed pervasive systems: home

systems, electronic health-care systems and sensor
networks

• Study Home systems, Electronic health-care systems

and Sensor networks in detail from the book
(page 26-30)

• Search more examples of distributed pervasive systems