Starter

In recent years, the availability of databases and of computer networks has given rise to a new
field, which is a distributed database. A distributed database is integrated database, which is
built on top of a computer network rather than on a single computer. The data, which constitute
the database, are stored at the different sites of the computer network, and the application
programs, which are run by the computers, access data at different sites. Databases may involve
different database management systems, running on different architectures, that distributes the
execution of transactions.

A Distributed Database Management System (DDBMS) is defined as the software that handles
the management of the DDB (Distributed Database) and makes the operation of such a system
appear to the user as if it were a centralized database.

Distributed database is a collection of data, which belongs logically to the same system but are
spread over the sites of a computer network.

Thus, we say that:

(1) Data is stored at several sites, each managed by a DBMS that can run independently.

(2) The data from different sites are tied together using computer network.

(3) On each site there are two types of data

(a) Data required for local site.
(b) Data required for other sites.

Following two properties are satisfied by distributed databases:

§ Distributed Data Independence: Users should not have to know where data is located
(extends Physical and Logical Data Independence principles).

§ Distributed Transaction Atomicity: Users should be able to write Xacts accessing multiple
sites just like local Xacts.

From above properties, Users have to be aware of where data is located, i.e., Distributed Data
Independence and Distributed Transaction Atomicity are not supported. These properties are hard
to support efficiently. For globally distributed sites, these properties may not even be desirable
due to administrative overheads of making location of data transparent.

Advantages and Disadvantages

Following are few advantages of DDBMS:

Data is located near the greatest demand site, access is faster, processing is faster due to several
sites spreading out the work load, new sites can be added quickly and easily, communication is
Sushil Kulkarni 2

improved, operating costs are reduced, it is user friendly, there is less danger of a single-point
failure, and it has process independence.

Several reasons why businesses and organizations move to distributed databases include
organizational and economic reasons, reliable and flexible interconnection of existing database,
and the future incremental growth. Companies believe that a decentralized, distributed data
database approach will adapt more naturally with the structure of the organizations. Distributed
database is more suitable solution when several databases already exist in an organization. In
addition, the necessity of performing global application can be easily performed with distributed
database. If an organization grows by adding new relatively independent organizational units,
then the distributed database approach support a smooth incremental growth.

Data can physically reside nearest to where it is most often accessed, thus providing users with
local control of data that they interact with. This result in local autonomy of the data allowing
users to enforce locally the policies regarding access to their data.

One might want to consider a parallel architecture is to improve reliability and availability of the
data in a scalable system. In a distributed system, with some careful tact, it is possible to access
some or possibly all of the data in a failure mode if there is sufficient data replication.

DDBMS have following few disadvantages:

Managing and controlling is complex, there is less security because data is at so many different
sites.

A distributed database provides more flexible accesses that increase the chance of security
violations since the database can be accessed throughout every site within the network. For many
applications, it is important to provide secure. Present distributed database systems do not provide
adequate mechanisms to meet these objectives. Hence the solution requires the operation of
DDBMS capable of handling multilevel data. Such a system is also called a multi level security
distributed database management systems (MLS-DDBMS). MLS-DDBMS provides a
verification service for users who wish to share data in the database at different level security. In
MLS- DDBMS, every data item in the database has correlated with one of several classifications
or sensitivities.

The ability to ensure the integrity of the database in the presence of unpredictable failures of both
hardware and software components is also an important features of any distributed database
management systems. The integrity of a database is concerned with its consistency, correctness,
validity, and accuracy. The integrity controls must be built into the structure of software,
databases, and involved personnel.

If there are multiple copies of the same data, then this duplicated data introduces additional
complexity in ensuring that all copies are updated for each update. The notion of concurrency
control and recoverability consume much of the research efforts in the area of distributed
database theory. Increasing in reliability and performance is the goal and not the status quo.

Example

Following example illustrates DDB

Sushil Kulkarni 3

Consider the bank having three branches connected by computer network containing teller
terminals of the branch and the account database of the branch All the branches can process the
database. Each branch that has local database constitutes one site of the distributed database.

The applications that are required from the terminals of a branch need only to access the database
of that branch. Local applications are applications processed by the computer of the branch from
where they are issued.

A global application involves the generation of a number of sub-transactions, each of which may
be executed at different sites.

For instance, suppose the customer wants to transfer the money from one account of one branch
to an account of another branch. This application requires updating the database at two different
branches. One has to perform two local updates at two different branches. Thus if A is the data
item with two copies A 1 and A 2 exists at sites S 1 and S 2 , then the operation of updating the
value of A involves generating two sub-transactions T S 1 and T S 2 , each of which will perform the
required operation from sites S 1 and S 2.

The branches are located at different geographical areas, then the banking system is called
distributed database on a geographical dispersed network. [See the figure]

The branches are situated in one building and are connected with a high bandwidth local network.
The teller terminals of the branches are connected to their respective computer by telephone lines.
Each processor and its database constitute a site of the local computer network. Here the
distributed database is implemented on a local network instead of a geographical network and is
called distributed database on a local network. [See the figure by replacing communication
network to local network]

Let us now consider the same example as above. The data of the different branches are distributed
on three backend computers, which perform the database management functions. The application

T T T T T T
Site 1 Site 2

COM COM
1 1

Communicatio
n Network

T T T
Site 3

COM
1
Sushil Kulkarni 4

programs are executed by different computer, which request database access services from the
backends when necessary. This type of system is not distributed database because the data is
physically distributed over different processors but their distribution is not relevant from the
application viewpoint. Here the existence of local application on computers can’t be executed.

From above examples, one can summarized and obtain the following working definition:

A distributed database is a collection of data, which are distributed over different computers of
a computer network. Each site of the network has processing capacity and can perform local
applications. Each site also participates in the execution of at least one global application.
Thus we have in short,

§ Can have many sites, each with their own DBMS.

§ Local data stored at each site.
§ Sites connected by communication network.
§ To a user (application program) all sites together appear as one big DB.

Functions of DDBMS

Functions of a centralized DBMS plus:

(1) extended communication to allow the transfer of queries and data among sites
(2) extended system catalog to store data distribution details
(3) distributed query processing , including query optimization
(4) extended concurrency control to maintain consistency of replicated data.
(5) extended recovery services to take account of failures of individual sites and common links

Two main issues

In DDBMS, there are two main issues:

§ Making query from one site to the same or remote site.

§ Logical database is partitioned in to different data streams and located at different sites.

User from one site makes a request from a site. This request can be made as if it is made to local
database. Users can see only one logical database and does not need to know the names of
different data streams. In fact, user need not know that the database is partitioned and the location
of data streams.

Component Architecture

The components, which are necessary for building distributed databases, are as follows:

User Request Interface is typically a client program that acts as an interface to the Distributed
Transaction Manager and accepts user's requests for transactions.

The distributed transaction manager or transaction processor (TP) is a program that

translates requests from the user and converts them into actionable requests for the database
managers, which are typically distributed.
Sushil Kulkarni 5

A Database Manager or database processor (DP) is software, which is responsible for

processing a segment of the distributed database.

Together, the distributed transaction managers and the database managers make up the DDBMS

A node is a piece of hardware that can implement a transaction manager, a database manager, or
both.

Distributed database systems can be comprised of infinitely many architectures. This

characteristic is key for maintaining a system that is to be scalable. For instance, it is possible for
DBMSs to be a collection of mainframes, minicomputers, or even PC's, or any combination
contained therein. This can allow for scalable systems, which integrate all three types of
computers into one system.

Types of distributed data bases

There are two types of distributed database systems: Homogeneous and Heterogeneous
distributed database system.

Homogeneous: If data is distributed but every site runs same type of DBMS, then the distributed
database is called homogeneous database.

Heterogeneous: If data is distributed but different sites run different DBMSs (different DBMSs
or even non-relational DBMSs), then the distributed database is called heterogeneous database.
Following figure illustrates this. This system is build using gateway protocols, which is API that
exposes DBMS functionality to external applications. Example is ODBC and JDBC.

DBMS1 DBMS2 DBMS3

Gateway
Distributed database architecture

There are two alternative approaches used to separate functionality across different distributed
databases related processes: Client-server, Collaborating server.

Client-server architecture:

This architecture is similar to the system of restaurant where many customers give order to a
waiter.

In this architecture there are three components clients, servers, and communication middleware

§ The client ships the request to a single site, which process the query.
§ The server process all queries send by the client
Sushil Kulkarni 6

§ The communication middleware is communication network by which client and server

communicate.

client client client

Query

server

The following figure illustrates this architecture:

Collaborating server architecture

The client-server architecture does not allow a single query to span multiple servers because the
client process would have to be capable of breaking such a query into appropriate subqueries to
be executed at different sites and then piecing together the answer to the subqueries. His difficulty
can be eliminated using collaborating server architecture

In this architecture, we have a collection of database servers, each capable of running transactions
against local data, which co-operatively execute transactions spanning multiple servers.

server
Client
server server Other
servers
Query server

A ‘client’ server makes a request for a data to a server using a query. The server generates
appropriate subqueries to be executed by ‘other’ servers and puts the result together to compute
the answer of the original query.

Data allocation

How should a database be distributed among the sites of a network? We discussed this important
issue using following strategies.
Sushil Kulkarni 7

§ Centralized
§ Replication
§ Fragmentation

We will explain and illustrate each of these approaches using relational databases.
Suppose that a bank has numerous branches located throughout a state. One of the base relations
in the bank’s database is the customer relation given below. For simplicity the sample data in the
relation apply to only two of the branches (Mumbai and Pune). The primary key in the relation is
account number (Ac_no). B_name is the name of the branch where customers have opened their
accounts.

Ac_no Customer_name B_name Balance

200 Smita Mumbai 1000
324 Hemant Pune 250
153 Smith Pune 38
426 Aishwarya Mumbai 1500
500 Tina Pune 168
683 Samira Mumbai 1398
252 Ajay Mumbai 303

Centralized

All data can be stored at one site. The processing can be done only from this site. Other sites
have to approach this site for data. If problem occurs at this site then the system fails. This
method of storing data has high communication costs.

Fragmentation

Fragmentation consists of breaking a relation into smaller relations or fragments and storing the
fragments at different sites. Thus partitioning of global relation R into fragments R1, R2, . . . , Rn
that contain sufficient information to reconstruct the original relation..

Following are the advantages of fragmentation

• Allows parallel processing of data

• tuples are located where they are most frequently accessed

To fragment the relation the prerequisite is the access patterns (typical queries) and application
behavior must be known; that is, which applications access which (portions of the) data.
There are two types of fragmentation: horizontal, vertical,

Horizontal fragmentation

With this fragmentation, some of the rows of a table are put into a base relation at one site, and
other rows are put into a base relation at another site. More generally, the rows of a relation are
distributed to many sites. Fragments are required to be disjoint.

Each fragment assigned to one site locality of reference can bee high, storage costs are low - no
replication, reliability and availability are low
Sushil Kulkarni 8

For a given relation R(A1, . . . , An) a set PR = {p1, . . . , pn} of simple predicates is determined.
Each pi has the pattern Ai q <value>, with q Î. {=, ¹,< ,>}.

Assume the two relations:

EMP(EmpNo, EName, Job, Sal, Deptno ®. DEPT)

DEPT(Deptno, DName, Location)

Furthermore, assume that there are two applications. One application typically retrieves
information about employees who earn more than 5000 Rs., the other application typically
manages information about clerks.

The simple predicates will be PEMP = {Job = ’Clerk’, Sal > 5000}and set of minterm predicates
MR = {mi | [not]p1 . . . . . [not] p n }

m1 = Job = ’Clerk’ Ù Sal > 5000 m2 = Job = ’Clerk’ Ù Sal £ 5000

m3 Job ¹ ’Clerk’ Ù Sal > 5000 m4 ¹ Job = ’Clerk’ Ù Sal £ 5000

Thus the fragments EMP i = s mi (EMP), i = 1, . . . , 4

Vertical fragmentation

Here the idea is to split schema R into smaller schemas R i. Splitting is based on which attributes
are typically accessed together by (different) applications. Each fragment R i must contain
primary key in order to reconstruct original relation R.

With this fragmentation, some of the columns of a relation are projected into a base relation at
one of the sites, and other columns are projected into a base relation at another site. More
generally, the columns may be projected to several sites. For example two fragments of EMP can
be written as

EMP1 (EmpNo, EName, Sal), EMP2 (EmpNo, Job, Deptno)

This fragmentation is more difficult than horizontal fragmentation, because more alternatives
exist. The fragmentation can be achieved in this case by two approaches: (1) grouping attributes
to fragments, (2) splitting relation into fragments.

The collection of vertical fragmentation should be losses-join decomposition

Following figure illustrates these fragments.

Sushil Kulkarni 9

For the given relation we can fragment as follows:

[A] Following are the horizontal fragmentation according to branches

Ac_no Customer_name B_name Balance

200 Smita Mumbai 1000
426 Aishwarya Mumbai 1500
683 Samira Mumbai 1398
252 Ajay Mumbai 303
Mumbai branch

Ac_no Customer_name B_name Balance

324 Hemant Pune 250
153 Smith Pune 38
500 Tina Pune 168
Pune branch

[B] Following are the vertical fragmentation.

Ac_no Customer_name B_name

200 Smita Mumbai
324 Hemant Pune
153 Smith Pune
426 Aishwarya Mumbai
500 Tina Pune
683 Samira Mumbai
252 Ajay Mumbai

Ac_no Balance
200 1000
324 250
153 38
426 1500
500 168
683 1398
252 303

When a relation is fragmented, we must be able to recover the original relation from the
fragments.

Correctness of Fragmentation

§ Completeness - Decomposition of relation R into fragments R1, R2, . . . , Rn is complete if

and only if each tuple in R can also be found in some Ri.

§ Reconstruction - If relation R is decomposed into fragments R1, R2, . . . , Rn, then there
should be some relational operator Ä such that R = Ä 1£ i £ n ( R i ).
• Disjointness: If a relation R is partitioned, and a tuple t is in fragment R i, then t should be in
no other fragment Rj, j ¹ i
Sushil Kulkarni 10

Replication

System maintains multiple copies of data (fragments) stored at different sites, for faster retrieval
and fault tolerance.

• A relation or fragment of a relation is replicated if it is stored redundantly at two or more sites.

• Full replication of a relation is the case where the relation is stored at all sites.

• Fully redundant databases are those in which every site contains a copy of the entire database.

• Rule:

Advantages

• Availability: Failure of a site containing fragment/relation R does not result in unavailability of

R if replicas of R exist.

• Parallelism: Queries on R may be processed by several nodes in parallel.

• Reduced data transfer: Relation R is available locally at each site containing a replica of R

Disadvantages

• Increased cost of updates: Each replica of R must be updated.

• Increased complexity of concurrency control: Concurrent updates two distinct replicas may
lead to inconsistent data unless special concurrency control mechanisms are implemented.

Strategies

• Synchronous replication: Before an update transaction commits, it synchronizes all copies of

the modified data (_ Read-One-Write-All (ROWA) technique)

• Asynchronous replication: Changes to primary copy are propagated (periodically) to

secondary copies (_ snapshots).

The allocation schema describes the mapping of relations or fragments to the servers that store
them. This mapping can be:

– Non-redundant, when each fragment or relation is allocated to a single server

– Redundant, when at least one fragment or relation is allocated to more than one
server
Sushil Kulkarni 11

Transparencies in DDBMS

A distributed database system has certain functional characteristics. These characteristics are
grouped together and called set of transparency. The DDBMS transparency features are:

Distributed Transparency

This allows the user to see the database as a single, logical entity. If this transparency is exhibited
then the user does not need to know that

§ The data are partitioned

§ Data can be replicated at several sites.
§ Data location.

The level of transparency supported by DDBMS varies from system to system. Three levels of
distributed transparency are recognized:

Transparency of fragmentation, of allocation and of language

In absence of transparency, each DBMS accepts its own SQL ‘dialect’: the system is
heterogeneous and the DBMSs do not support a common interoperability standard

Let us assume that the SUPPLIER (SNum, Name, City) relation with two horizontal fragments

SUPPLIER1 = sCity=‘Mumbai SUPPLIER

SUPPLIER2 = sCity=‘ Hydrabad’ SUPPLIER

And the allocation schema:

[email protected]
[email protected]
SUPPLIER2@company. Hydrabad 2.in

Fragmentation transparency

On this level, the programmer should not worry about whether or not the database is distributed
or fragmented For example, consider the following query:

procedure Query1(:snum,:name);
SELECT Name into: name
FROM Supplier
WHERE SNum =: snum;
end procedure;

Here snum and name are found from the fragmentations SUPPLIER1 and SUPPLIER2 but the
user will not know that the data is stored on different nodes.
Sushil Kulkarni 12

Allocation transparency

On this level, the programmer should know the structure of the fragments, but does not have to
indicate their allocation. With replication, the programmer does not have to indicate which copy
is chosen for access (replication transparency) For example, consider the following query:

procedure Query2(:snum,:name);
SELECT Name into :name
FROM Supplier1
WHERE SNum =: snum;
IF :empty then
SELECT Name into :name
FROM Supplier2
WHERE SNum = :snum;
end procedure;

Language transparency

On this level the programmer must indicate in the query both the structure of the fragments and
their allocation

Queries expressed at a high level of transparency are transformed to this level by the distributed
query optimizer, aware of data fragmentation and allocation For example, consider the following
query:

procedure Query 3 (:snum,:name);

SELECT Name into :name
FROM [email protected]
WHERE SNum = :snum;
IF :empty then
SELECT Name into :name
FROM [email protected]
WHERE SNum = :snum;
end procedure;

Optimizations

This application can be made more efficient in two ways:

§ By using parallelism: instead of submitting the two requests in sequence, they can be
processed in parallel, thus saving on the global response time

§ By using the knowledge on the logical properties of fragments (but then the programs are not
flexible)

For example, consider the following query:

procedure Query 4 (:snum,:name,:city);

CASE: city of
Sushil Kulkarni 13

"Mumbai" : SELECT Name into: name

FROM Supplier1
WHERE SNum = :snum;

"Hydrabad": SELECT Name into: name

FROM Supplier2
WHERE SNum = :snum;
end procedure;

Classification Of Transaction

This transparency allows a transaction to update data at several sites. It will also ensure that the
transaction is completed or aborted and thus database integrity is maintained.

In order to understand how the transactions are managed, one should know the classification of
transaction. : remote request, remote transaction, distributed transaction and distributed requests.

Remote request

These are read-only transactions made up of an arbitrary number of SQL queries, addressed to a
single remote DBMS. The remote DBMS can only be queried. Following figure illustrate this
Site 1 Site 2

Network
TP ¬¾ ¾ ¾¾® DP Employee

SELECT * Comment: The request is

FROM Employee made directly to employee
WHERE eno = 123 table at site 2

Remote transactions

It is made up of any number of SQL commands (select, insert, delete, update) directed to a single
remote DBMS and each transaction writes onto only one DP.Consider Customer and Employee
tables situated at site2. The transaction should able to update customer and employee tables. The
transaction can reference to only one DP at a time.
Site 1 Site 2

Network Employee
TP ¬¾ ¾ ¾¾® DP

Customer
begin:
UPDATE Employee
SET salary = salary + 200
WHERE eno > 123;
INSERT
INTO Customer ( cno, cname, age)
VALUES ( 123, ‘Seema’,29);
COMMIT;
Sushil Kulkarni 14

Distributed transactions

It is made up of any number of SQL commands (select, insert, delete, update) directed to an
arbitrary number of remote DP sites, but each SQL command refers to a single DBMS

Let us consider the transaction that points towards two remote sites say 2 and 3. The first request
(SELECT statement) is processed by the DP at the remote site 2, and the next requests
(UPDATE, INSERT statements) are processed by the DP at the remote site 3. Each request can
access only one request at a time.
Site 1 Site 2

Network Employee
¬¾ ¾ ¾¾®
TP DP

begin: Customer
SELECT * DP
FROM Employee
WHERE eno > 104;
Product
UPDATE Customer Site 3
SET cbalance =cbalance -100
WHERE cno = 124;

INSERT
INTO Product ( pno, price)
VALUES ( 321, 50);
COMMIT;

Distributed requests

It is made up of arbitrary transactions, in which each SQL command can refer to any DP that may
contain the fragmentation. This request requires a distributed optimizer.

Let us consider the following examples:

Example 1: Let shop (sno, sname) is at site 2 and customer (cno, cname, bill, sno) and
employee (eno, ename, sno) be at site 3. Fetch the tuples to find sname and cname where sno
= 123. The following figure 1 illustrates this:

Example 2: The distributed request features allow a single request to references a physically
partitioned table. Divide the employee table into two fragments says E1 and E2 located sites 2
and 3. Suppose the end- user wants to obtain all tuples whose salary exceed Rs. 15,000. The
request is illustrated in figure 2
Sushil Kulkarni 15

Site 1 Site 2
Network Shop
¬¾ ¾ ¾¾®
TP DP

begin: DP
Customer
SELECT sname, cname
FROM shop, customer
WHERE sno =123 AND
Customer.sno= shop.sno; Employee
Site 3
UPDATE Customer
SET bill = bill+5100
WHERE cno = 124;

INSERT
INTO Product ( pno, price) Figure 1
VALUES ( 321, 50);
COMMIT;

Site 1
Network Site 2
¬¾ ¾ ¾¾® E1
TP DP

begin: DP
SELECT * E2
FROM employee
WHERE salary >15000;
COMMIT;
Site 3

Figure 2

Distributed Catalog manager

Catalog manager keeps track of all data, which are distributed across different sites by keeping
the name of each replica of each fragment. To preserve local autonomy, information is stored in
the form < local_name, birth_site>. Every site also has the catalog that describes all objects in the
form < fragments, replica> at a site plus it keeps the track of all replica of relations created at this
site. To find any relation it is the best to see birth site catalog which never changes even if the
relation is moved to another site.
Sushil Kulkarni 16

Atomicity problem: Fund transfer transaction

Assume: ACCOUNT (AccNum,Name,Total) with accounts lower than 10000 allocated on

fragment Account1 and accounts above 10000 allocated on fragment Account2 The code can be
developed as follows:

begin transaction
UPDATE Account1
SET Total = Total - 100000
WHERE AccNum = 3154;

UPDATE Account2
SET Total = Total + 100000
WHERE AccNum = 14878;
commit ;
end transaction

The above code is not acceptable because it may violate atomicity that is one of the modifications
is executed while the other is not

Technology of distributed databases

Data distribution does not influence consistency and durability because

§ Consistency of transactions does not depend on data distribution, because integrity

constraints describe only local properties (a limit of the actual DBMS technology)

§ Durability is not a problem that depends on the data distribution, because each system
guarantees local durability by using local recovery mechanisms (logs, checkpoints, and
dumps)

The distributed system require major enhancements in the following concepts:

§ Query optimization
§ Concurrency control
§ Reliability control

We will see these concepts in detail

Distributed query optimization

Besides the factors used in centralized databases, we require additional factors for optimization.
Query optimization is required when a DP receives a distributed request from the site where TP is
located; the DP that is queried is responsible for the ‘global optimization’

DP decides on the breakdown of the query into many sub-queries, each addressed to a specific
DP and it builds a strategy (plan) of distributed execution that consists of coordinating various
execution programs from various DPs and in the exchange of data among them

There are two major cost factors that are involved

§ quantity of data transmitted on the network
Sushil Kulkarni 17

§ unit cost of transmission

The total cost is given using following formula:

C total = C I/O x n I/O + C cpu x n cpu + Ctr x ntr

Where ntr: the quantity of data transmitted on the network

Ctr: unit cost of transmission

The following examples will give us the idea for determining the “best” execution plan

Consider the following relations:

Student (rno, name, address, cno) on site 1

Course (cno, ctitle, duration) on site 2

These relations are not fragmented and have the following information
§ Each record of Student contains 10,000 records and each record is of 100 bytes long.
§ Each record of Course contains 100 records and each record is of 35 bytes long.
6
From above size of Student relation is 100*10,000 = 10 bytes and size of Course relation is
100*35 bytes.

Query: Find the names of students and the course titles that are taken by the students.
Using relational algebra the query can be sited as follows:
s name , cname ( Student join cno Course )

The result of this query will contain 10,000 records, assuming that every student is opted for a
course. Assume further that each record in the query will result in 40 bytes long.

[A] Suppose assume that the query is submitted at a site 2. Site 2 is called resultant site. We have
the following strategies:

(1) Transfer the Student relation to site 2 by executing the query and result is displayed at site 2.
Then 10 6 bytes are transfer from site 1 to2.
(2) Transfer the Course relation to site 1 by executing the query at site 1 and send the result at
site 2. Then 4000,000 + 3500 = 403,500 bytes are transfer for the query.

If minimizing the amount of data transfer is our optimization criteria then we can choose strategy
2.

[B] Suppose assume that the query is submitted at a site 3 that does not contain any one of the
relations given above. Site 3 is called resultant site where we will store the result obtained by the
given query. We have the following strategies:

Student: Site 1
Result site:
Course: Site 2 Site 3
Sushil Kulkarni 18

(1) Transfer Student and Course relations to site 3 and perform the join at site 3. Here total of
10 6 +3500 = 1,003,500 bytes must be transferred.
(2) Transfer the Student relation to site 2, execute the join at site 2 and send the result to site 3.
The size of the query result is 40*10,000 bytes, so 4000,000 + 10 6 = 1,400,000 bytes are
transferred.
(3) Transfer the Course relation to site 1, execute the join at site 1 and send the result to site 3.
The size of the query result is 40*10,000 bytes, so 4000,000 + 3500 = 403,500 bytes are
transferred.

If minimizing the amount of data transfer is our optimization criteria then we can choose strategy
3.

Following is the SQL query to find the ENAME from the relation EMP (ENO, ENAME, AGE)
and ASG (ENO, DUR)

SELECT ENAME
FROM EMP, ASG
WHERE EMP.ENO = ASG.ENO AND DUR > 37

Following are the two possible strategies:

Strategy 2 is better because it avoids Cartesian product. So what is the problem?

Suppose ASG has two fragmentations say ASG1 and ASG2 located at site 1 and 2.
Suppose EMP has two fragmentations say EMP1 and EMP2 located at site3 and 4.
The result is stored at site 5 [see the figure]

Let us assume that: size (EMP) = 400, size (ASG) = 1000 and tuple access cost = 1 unit; tuple
transfer cost = 10 units

For strategy 1

§ Produce ASG': (10+10) * tuple access cost 20

§ Transfer ASG' to the sites of EMP: (10+10) * tuple transfer cost 200
§ Produce EMP ' : (10+10) * tuple access cost * 2 40
§ Transfer EMP ' to result site: (10+10) *tuple transfer cost 200
______
Total cost 460

For Strategy 2
Sushil Kulkarni 19

§ Transfer EMP to site 5: 400 * tuple transfer cost 4,000

§ Transfer ASG to site 5 : 1000 * tuple transfer cost 10,000
§ Produce ASG ' : 1000 * tuple access cost 1,000
§ Join EMP and ASG':400*20*tuple access cost 8,000
______
Total cost 23,000

Thus strategy 1 is better than 2.

Concurrency Control

There are varies techniques to determine which object are to be locked in distributed databases by
obtaining and releasing the locks. The locks are determined by concurrency control protocol.

Lock management can be distributed across sites in the following ways:

Centralized A single site is in charge of handling lock and unlock requests for all objects.

Primary copy One copy of each object is called the primary copy. All requests to lock or unlock
a copy of this object are handled by the lock manager at the site where the primary copy is stored.

Fully distributed Request to lock or unlock a copy of an object stored at a site is handled by the
lock manager of that site where the copy is stored.

The centralized scheme is not useful as well as popular because the failure of the site, which
contains locks, will disturb the functionality of all sites.
Sushil Kulkarni 20

The primary copy scheme avoids this problem, but in general, reading an object requires
communications with two sites, the site where the primary copy resides and the site where the
copy to be read resides. This problem is avoided in the fully distributed scheme, because locking
is done at the site where the copy to be read resides. However, while writing locks must be set at
all sites where copies are modified in the fully distributed scheme, whereas locks need to be set
only at one site in the other two schemes.

Serializability in distributed databases

In a distributed system, a transaction t i can carry out various sub-transactions t i j, where the
second subscript denotes the node of the system on which the sub-transaction works. For
example, let there be two nodes say node 1 and node 2, then the transactions on them can be
performed as follows:

t1 t2
Node 1 Node 2 Node 1 Node 2
Read(x) Read(y)
Write(x) Write(y)
Read(y) Read(x)
Write(y) Write(x)

This can be written as follows

t1 : r11(x) w11(x) r12(y) w12(y)

t2 : r22(y) w22(y) r21(x) w21(x)

The local Serializability within the schedulers is not a sufficient guarantee of serializability.
Consider the two schedules at nodes 1 and 2 [see the table above]:

S1: r11 (x) w11 (x) r21 (x) w21 (x)

S2: r22 (y) w22 (y) r12 (y) w12 (y)

These are locally serializable, but their global conflict graph has a cycle, since for a node 1 t1
precedes t2 and is in conflict with t2 and for node 2, t2 precedes t1 and is in conflict with t1. Thus
we define:

Global serializability of distributed transactions over the nodes of a distributed database requires
the existence of a unique serial schedule S equivalent to all the local schedules Si produced at
each node.

The following properties are valid.

(1) If each scheduler of a distributed database uses the two-phase locking method on each node
and carries out the commit action when all the sub-transactions have acquired all the
resources, then the resulting schedules are globally conflict-serializable

(2) This is imposed by the 2-phase commit protocol

If each distributed transaction acquires a single timestamp and uses it in all requests to all the
schedulers that use concurrency control based on timestamp, the resulting schedules are
globally serial, based on the order imposed by the timestamps
Sushil Kulkarni 21

Lamport Method for assigning time-stamp

The Lamport method for assigning timestamps reflects the precedence among events in a
distributed system. The method is as follows:

• A timestamp is a number characterized by two groups of digits

(1) The least significant digits identify the node at which the event occurs

(2) The most significant digits identify the events that happen at that node. They can be obtained
from a local counter, which is incremented at each event

§ Each time two nodes exchange a message, the timestamps become synchronized.The
receiving event must have a timestamp greater than the timestamp of the sending event and
may require the increasing of the local counter on the receiving node

Following example shows how timestamps can be assignment using the Lamport method

Distributed deadlocks

Distributed deadlocks can be due to circular waiting situations between two or more nodes. A
particular time is given to all nodes to replay and if one of the nodes is not responding in a given
time then dead lock may be possible.

Deadlock resolution can be done with an asynchronous and distributed protocol (implemented in
a distributed version of DB2 by IBM)

Distributed deadlock resolution

Assume that using a remote procedure call, that is, a synchronous call to a procedure that is
remotely executed activates sub-transactions. This model allows for two distinct types of waiting

§ Two sub-transactions of the same transaction can be in waiting in distinct DP s as one waits
for the termination of the other. For instance, if t 11 activates t 12 , it waits for the termination
of t 12
Sushil Kulkarni 22

§ Two different sub-transactions on the same DPS can wait as one blocks a data item to which
the other one requires access. For instance, if t 11 locks an objects requested by t 21, t 21 waits
for the termination of t 11

Following figure illustrates this:

Representation of waiting conditions

The waiting conditions at each DP can be characterized using precedence conditions, Let t i and
t j be two transactions such that t j has to wait till t i is completed. The general format of waiting
condition is give using a wait sequence as:

EXT < t i < t j < EXT

Where EXT represents executions at a remote DP. For example,

§ On DP1 we have: EXT < t21 < t11 < EXT

§ On DP2 we have: EXT < t12 < t22 < EXT

Algorithm can be developed on distributed deadlocks and is given below and is periodically
activated on the various DPs of the system. When it is activated, it:

§ integrates new wait sequences with the local wait conditions as described by the lock
manager

§ analyzes the wait conditions on its DP and detects local deadlocks. It communicates the wait
sequences to other instances of the same algorithm

§ to avoid the situation in which the same deadlock is discovered more than once, the algorithm
sends wait sequences:

– ‘ahead’, towards the DPS which has received the remote procedure call.

– only if, for example, i > j where i and j are the identifiers of the transactions.
Sushil Kulkarni 23

Failures in Distributed Systems

A distributed system is subject to failures, message losses, or network partitioning

• Node failures may occur on any node of the system and be soft or hard, as discussed before
• Message losses leave the execution of a protocol in an uncertain situation

– Each protocol message (msg) is followed by an acknowledgement message (ack)

– The loss of either one leaves the sender uncertain about whether the message has been
received

• Network partitioning. This is a failure of the communication links of the computer network,
which divides it into two sub-networks that have no communication between each other
– A transaction can be simultaneously active in more than one sub-network

Two-phase commit protocol

Commit protocols allow a transaction to reach the correct commit or abort decision at all the
nodes that participate in a transaction

The two-phase commit protocol is similar in essence to a marriage, in that the decision of two
parties is received and registered by a third party, who ratifies the marriage
Sushil Kulkarni 24

The servers, who represent the participants to the marriage, are called resource managers (RM)
and the celebrant (or coordinator), is allocated to a process, called the transaction manager
(TM).
It takes place by means of a rapid exchange of messages between TM and RM and writing of
records into their logs. The TM can use broadcast mechanisms (transmission of the same message
to many nodes, collecting responses arriving from various nodes) and serial communication with
each of the RMs in turn.

Let us see how the records are stored in TM and RM

Records of TM

§ The prepare record contains the identity of all the RM processes (that is, their identifiers of
nodes and processes)

§ The global commit or global abort record describes the global decision. When the TM
writes in its log the global commit or global abort record, it reaches the final decision

§ The complete record is written at the end of the two-phase commit protocol

Records of RM

§ The ready record indicates the irrevocable availability to participate in the two-phase commit
protocol, thereby contributing to a decision to commit. Can be written only when the RM is
“recoverable”, i.e., possesses locks on all resources that need to be written. The identifier
(process identifier and node identifier) of the TM is also written on this record

§ In addition, begin, insert, delete, and update records are written as in centralized servers

§ At any time an RM can autonomously abort a sub-transaction, by undoing the effects, without
participating to the two-phase commit protocol

First phase of the basic protocol

§ The TM writes the prepare record in its log and sends a prepare message to all the RMs. Sets
a timeout indicating the maximum time allocated to the completion of the first phase

§ The recoverable RMs write on their own logs the ready record and transmit to the TM a ready
message, which indicates the positive choice of commit participation

§ The non-recoverable RMs send a not-ready message and end the protocol

§ The TM collects the reply messages from the RMs

– If it receives a positive message from all the RMs, it writes a global commit record on its
log
– If one or more negative messages are received or the timeout expires without the TM
receiving all the messages, it writes a global abort record on its log

Second phase of the basic protocol

§ The TM transmits its global decision to the RMs. It then sets a second time-out
Sushil Kulkarni 25

§ The RMs that are ready receive the decision message, write the commit or abort record on
their own logs, and send an acknowledgement to the TM. Then they implement the commit or
abort by writing the pages to the database as discussed before
§ The TM collects all the ack messages from the RMs involved in the second phase. If the
time-out expires it sets another time-out and repeats the transmission to all the RMs from
which it has not received an ack
§ When all the acks have arrived, the TM writes the complete record on its log

Following figure illustrates this protocol

Two-phase commit protocol in the context of a transaction is shown in the following figure

Blocking, Uncertainty, recovery protocols

• An RM in a ready state loses its autonomy and awaits the decision of the TM. A failure of the
TM leaves the RM in an uncertain state. The resources acquired by using locks are blocked.
Sushil Kulkarni 26

• The interval between the writing on the RM’s log of the ready record and the writing of the
commit or abort record is called the window of uncertainty. The protocol is designed to keep
this interval to a minimum.

• Recovery protocols are performed by the TM or RM after failures; they recover a final state
which depends on the global decision of the TM

Recovery of participants

• Performed by the warm restart protocol. Depends on the last record written in the log:
– When it is an action or abort record, the actions are undone; when it is a commit, the
actions are redone; in both cases, the failure has occurred before starting the commit
protocol
– When the last record written in the log is a ready, the failure has occurred during the two-
phase commit. The participant is in doubt about the result of the transaction

§ During the warm restart protocol, the identifier of the transactions in doubt are collected in
the ready set. For each of them the final transaction outcome must be requested to the TM

§ This can happen as a result of a direct (remote recovery) request from the RM or as a
repetition of the second phase of the protocol

Recovery of the coordinator

§ When the last record in the log is a prepare, the failure of the TM might have placed some
RMs in a blocked situation. Two recovery options:

– Write global abort on the log, and then carry out the second phase of the protocol
– Repeat the first phase, trying to arrive to a global commit

§ When the last record in the log is a global decision, some RMs may have been correctly
informed of the decision and others may have been left in a blocked state. The TM must
repeat the second phase

Message loss and network partitioning

§ The loss of a prepare or ready messages are not distinguishable by the TM. In both cases,
the time-out of the first phase expires and a global abort decision is made

§ The loss of a decision or ack message are also indistinguishable. In both cases, the time-out
of the second phase expires and the second phase is repeated

§ A network partitioning does not cause further problems, in that the transaction will be
successful only if the TM and all the RMs belong to the same partition

Presumed abort protocol

§ The presumed abort protocol is used by most DBMSs

§ Based on the following rule:

Sushil Kulkarni 27

– when a TM receives a remote recovery request from an induct RM and it does not know
the outcome of that transaction, the TM returns a global abort decision as default

§ As a consequence, the force of prepare and global abort records can be avoided, because in
the case of loss of these records the default behavior gives an identical recovery

§ Furthermore, the complete record is not critical for the algorithm, so it needs not be forced; in
some systems, it is omitted. In conclusion the records to be forced are ready, global commit
and commit

Read-only optimization

§ When a participant is found to have carried out only read operations (no write operations)

§ It responds read-only to the prepare message and suspends the execution of the protocol

§ The coordinator ignores read-only participants in the second phase of the protocol

Four-phase commit protocol

§ Created by Tandem, a provider of hardware-software solutions for data management based

on the use of replicated resources to obtain reliability

§ The TM process is replicated by a backup process, located on a different node. At each phase
of the protocol, the TM first informs the backup of its decisions and then communicates with
the RMs

§ The backup can replace the TM in case of failure

§ When a backup becomes TM, it first activates another backup, to which it communicates the
information about its state, and then continues the execution of the transaction

Three-phase commit protocol

Sushil Kulkarni 28

§ The basic idea is to introduce a third pre-commit phase in the standard protocol. If the TM
fails, a participant can be elected as new TM and decide the result of the transaction by
looking at its log

– If the new TM finds ready as last record, no other participants in the protocol can have
gone beyond the pre-commit condition, and thus can make the decision to abort

– If the new TM finds pre-commit as last record, it knows that the other participants are at
least in the ready state, and thus can make the decision to commit

§ The three-phase commit protocol has serious inconveniences and has not been successfully
implemented:
– It lengthens the window of uncertainty
– It is not resilient to network partitioning, unless with additional quorum mechanisms

Interoperability

§ Interoperability is the main problem in the development of heterogeneous applications for

distributed databases

§ It requires the availability of functions of adaptability and conversion, which make it

possible to exchange information between systems, networks and applications, even when
heterogeneous

§ Interoperability is made possible by means of standard protocols such as FTP, SMTP/MIME,

and so on

§ With reference to databases, interoperability is guaranteed by the adoption of suitable

standards