Chapter 16: Concurrency

Control

■Lock-Based Protocols
■Timestamp-Based Protocols
■Validation-Based Protocols
■Multiple Granularity
■Multiversion Schemes
■Deadlock Handling
■Insert and Delete Operations
■Concurrency in Index Structures

Database System Concepts 3rd Edition 16.1 ©Silberschatz, Korth and

 Lock-Based
Protocols

■A lock is a mechanism to control concurrent access to a data item

■Data items can be locked in two modes :
1. exclusive (X) mode. Data item can be both read as well as
written. X-lock is requested using lock-X instruction.
2. shared (S) mode. Data item can only be read. S-lock is
requested using lock-S instruction.

■Lock requests are made to concurrency-control manager.

Transaction can proceed only after request is granted.

Database System Concepts 3rd Edition 16.2 ©Silberschatz, Korth and

 Lock-Based Protocols
(Cont.)
■Lock-compatibility matrix

■A transaction may be granted a lock on an item if the requested lock is

compatible with locks already held on the item by other
transactions
■Any number of transactions can hold shared locks on an item,
but if
any transaction holds an exclusive on the item no other
transaction may hold any lock on the item.
■If a lock cannot be granted, the requesting transaction is made to
wait till
all incompatible locks held by other transactions have been
released. The lock is then granted.

Database System Concepts 3rd Edition 16.3 ©Silberschatz, Korth and

 Lock-Based Protocols
(Cont.)
■ Example of a transaction performing locking:
T2: lock-S(A);
read (A);
unlock(A)
; lock-
S(B);
read (B);
unlock(B);
display(A+B
)

■ Locking get
as above is not sufficient to guarantee serializability — if A and B
updated in-between the read of A and B, the displayed sum would be
wrong.

■A locking protocol is a set of rules followed by all transactions while

requesting and releasing locks. Locking
Database System Concepts 3rd Edition 16.4 protocols restrict the set of Korth and
©Silberschatz,
 Pitfalls of Lock-Based
Protocols
■ Consider the partial schedule

■ Neither T3 nor T4 can make progress — executing lock-S(B) causes T4

to wait for T3 to release its lock on B, while executing lock-X(A) causes
T3 to wait for T4 to release its lock on A.

■ Such a situation is called a deadlock.

 To handle a deadlock one of T3 or T4 must be rolled back
and its locks released.

Database System Concepts 3rd Edition 16.5 ©Silberschatz, Korth and

 Pitfalls of Lock-Based Protocols
(Cont.)

■The potential for deadlock exists in most locking protocols.

Deadlocks are a necessary evil.
■Starvation is also possible if concurrency control manager is badly
designed. For example:
 A transaction may be waiting for an X-lock on an item, while a
sequence of other transactions request and are granted an S-lock
on the same item.
 The same transaction is repeatedly rolled back due to deadlocks.
■Concurrency control manager can be designed to prevent
starvation.

Database System Concepts 3rd Edition 16.6 ©Silberschatz, Korth and

 The Two-Phase Locking
Protocol

■This is a protocol which ensures conflict-serializable schedules.

■Phase 1: Growing Phase
 transaction may obtain locks
 transaction may not release locks

■Phase 2: Shrinking Phase

 transaction may release locks
 transaction may not obtain locks

■The protocol assures serializability. It can be proved that the

transactions can be serialized in the order of their lock points
(i.e. the point where a transaction acquired its final lock).

Database System Concepts 3rd Edition 16.7 ©Silberschatz, Korth and

 The Two-Phase Locking Protocol
(Cont.)

■Two-phase locking does not ensure freedom from deadlocks

■Cascading roll-back is possible under two-phase locking. To avoid
this, follow a modified protocol called strict two-phase locking.
Here a transaction must hold all its exclusive locks till it
commits/aborts.

■Rigorous two-phase locking is even stricter: here all locks are

held till
commit/abort. In this protocol transactions can be serialized
in the order in which they commit.

Database System Concepts 3rd Edition 16.8 ©Silberschatz, Korth and

 The Two-Phase Locking Protocol
(Cont.)

■There can be conflict serializable schedules that cannot be

obtained if two-phase locking is used.
■However, in the absence of extra information (e.g., ordering of access
to data), two-phase locking is needed for conflict
serializability in the following sense:
Given a transaction Ti that does not follow two-phase locking, we
can find a transaction Tj that uses two-phase locking, and a
schedule for Ti and Tj that is not conflict serializable.

Database System Concepts 3rd Edition 16.9 ©Silberschatz, Korth and

 Lock
Conversions
■Two-phase locking with lock conversions:
– First Phase:
 can acquire a lock-S on item
 can acquire a lock-X on item
 can convert a lock-S to a lock-X (upgrade)
– Second Phase:
 can release a lock-S
 can release a lock-X
 can convert a lock-X to a lock-S (downgrade)

■This protocol assures serializability. But still relies on the

programmer to insert the various locking
instructions.

Database System Concepts 3rd Edition 16.10 ©Silberschatz, Korth and

 Automatic Acquisition of
Locks
■ A transaction Ti issues the standard read/write instruction,
without explicit locking calls.

■The operation read(D) is processed as:

if Ti has a lock on D
then
read(D)
else
begin
if necessary wait until no other
transaction has a lock-X on D
grant Ti a

lock-S on D;
Database System Concepts 3rd Edition read(D) 16.11 ©Silberschatz, Korth and
 Automatic Acquisition of Locks
(Cont.)
■ write(D) is processed as: if
Ti has a lock-X on
D
then
write(D)
else
begin
if necessary wait until no other trans. has any lock on D,
if Ti has a lock-S on D
then
upgrade lock on D to lock-
X else
grant Ti a lock-X on D
write(D)
end;

■ All locks are released after commit or abort

Database System Concepts 3rd Edition 16.12 ©Silberschatz, Korth and
 Implementation of
Locking

■A Lock manager can be implemented as a separate process to which

transactions send lock and unlock requests
■The lock manager replies to a lock request by sending a lock
grant
messages (or a message asking the transaction to roll
back, in case of a deadlock)

■The requesting transaction waits until its request is answered

■The lock manager maintains a datastructure called a lock table
to record granted locks and pending requests
■The lock table is usually implemented as an in-memory hash table
indexed on the name of the data item being locked

Database System Concepts 3rd Edition 16.13 ©Silberschatz, Korth and

 Lock Table
■ Black rectangles indicate granted
locks, white ones indicate waiting
requests
■
Lock table also records the type of
lock granted or requested
■ New request is added to the end of the
queue of requests for the data
item, and granted if it is compatible
with all earlier locks
■
Unlock requests result in the
request being deleted, and later
requests are checked to see if they
can now be granted
■ If transaction aborts, all waiting or
granted requests of the transaction
are deleted
 lock manager may keep a list of
locks held by each transaction, to
implement this efficiently

Database System Concepts 3rd Edition 16.14 ©Silberschatz, Korth and

 Graph-Based
Protocols

■Graph-based protocols are an alternative to two-phase locking

■ Impose a partial ordering  on the set D = {d1, d2 ,..., dh} of all
data items.
 If di  dj then any transaction accessing both di and dj must access
di before accessing dj.
 Implies that the set D may now be viewed as a directed acyclic
graph, called a database graph.

■The tree-protocol is a simple kind of graph protocol.

Database System Concepts 3rd Edition 16.15 ©Silberschatz, Korth and

 Tree Protocol

■Only exclusive locks are allowed.

■ The first lock by Ti may be on any data item. Subsequently, a
data Q can be locked by Ti only if the parent of Q is currently
locked by Ti.

■Data items may be unlocked at any time.

Database System Concepts 3rd Edition 16.16 ©Silberschatz, Korth and
 Graph-Based Protocols
(Cont.)
■The tree protocol ensures conflict serializability as well as
freedom from deadlock.
■Unlocking may occur earlier in the tree-locking protocol than in the
two-phase locking protocol.
 shorter waiting times, and increase in concurrency
 protocol is deadlock-free, no rollbacks are required
 the abort of a transaction can still lead to cascading rollbacks.
(this correction has to be made in the book also.)
■However, in the tree-locking protocol, a transaction may have to
lock data items that it does not access.
 increased locking overhead, and additional waiting time
 potential decrease in concurrency
■Schedules not possible under two-phase locking are possible under
tree protocol, and vice versa.

Database System Concepts 3rd Edition 16.17 ©Silberschatz, Korth and

 Timestamp-Based
Protocols
■Each transaction is issued a timestamp when it enters the system. If an old
transaction Ti has time-stamp TS(Ti), a new transaction Tj is
assigned time-stamp TS(Tj) such that TS(Ti) <TS(Tj).

■The protocol manages concurrent execution such that the time-

stamps determine the serializability order.

■In order to assure such behavior, the protocol maintains for each data Q
two timestamp values:
 W-timestamp(Q) is the largest time-stamp of any transaction that
executed write(Q) successfully.
 R-timestamp(Q) is the largest time-stamp of any transaction that
executed read(Q) successfully.

Database System Concepts 3rd Edition 16.18 ©Silberschatz, Korth and

 Timestamp-Based Protocols
(Cont.)
■The timestamp ordering protocol ensures that any conflicting
read and write operations are executed in timestamp order.
■ Suppose a transaction Ti issues a read(Q)

1. If TS(Ti)  W-timestamp(Q), then Ti needs to read a value of Q

that was already overwritten. Hence, the read operation

is rejected, and Ti is rolled back.
2. If TS(Ti) W-timestamp(Q), then the read operation is
executed, and R-timestamp(Q) is set to the maximum of R-
timestamp(Q) and TS(Ti).

Database System Concepts 3rd Edition 16.19 ©Silberschatz, Korth and

 Timestamp-Based Protocols
(Cont.)

■ Suppose that transaction Ti issues write(Q).

■ If TS(Ti) < R-timestamp(Q), then the value of Q that Ti is

producing was needed previously, and the system assumed that
that value would never be produced. Hence, the write operation
is rejected, and Ti is rolled back.
■ If TS(Ti) < W-timestamp(Q), then Ti is attempting to write an
obsolete value of Q. Hence, this write operation is rejected, and
Ti is rolled back.
■Otherwise, the write operation is executed, and W-timestamp(Q) is
set to TS(Ti).

Database System Concepts 3rd Edition 16.20 ©Silberschatz, Korth and

 Example Use of the
Protocol
A partial schedule for several data items for transactions with
timestamps 1, 2, 3, 4, 5

T1 T2 T3 T4 T5
read(X)
read(Y)
read(Y)
write(Y)

write(Z) read(Z)
read(X)
read(X) abort

write(Z) write(Y)
abort
write(Z)

Database System Concepts 3rd Edition 16.21 ©Silberschatz, Korth and

 Correctness of Timestamp-Ordering
Protocol
■The timestamp-ordering protocol guarantees serializability since all the
arcs in the precedence graph are of the form:

transaction transaction
with smaller with larger
timestamp timestamp

Thus, there will be no cycles in the precedence graph

■Timestamp protocol ensures freedom from deadlock as no
transaction ever waits.
■But the schedule may not be cascade-free, and may not even
be recoverable.

Database System Concepts 3rd Edition 16.22 ©Silberschatz, Korth and

 Recoverability and Cascade
Freedom

■Problem with timestamp-ordering protocol:

 Suppose Ti aborts, but Tj has read a data item written by Ti
 Then Tj must abort; if Tj had been allowed to commit earlier, the
schedule is not recoverable.
 Further, any transaction that has read a data item written by Tj must
abort
 This can lead to cascading rollback --- that is, a chain of rollbacks

■ Solution:
 A transaction is structured such that its writes are all performed at
the end of its processing
 All writes of a transaction form an atomic action; no transaction may
execute while a transaction is being written
 A transaction that aborts is restarted with a new timestamp

Database System Concepts 3rd Edition 16.23 ©Silberschatz, Korth and

 Thomas’ Write Rule

■Modified version of the timestamp-ordering protocol in which

obsolete write operations may be ignored under certain
circumstances.
■ When Ti attempts to write data item Q, if TS(Ti) < W-
timestamp(Q), then Ti is attempting to write an obsolete value of
{Q}. Hence, rather than rolling back Ti as the timestamp ordering
protocol would have done, this {write} operation can be
ignored.
■Otherwise this protocol is the same as the timestamp ordering
protocol.

■Thomas' Write Rule allows greater potential concurrency. Unlike

previous protocols, it allows some view-serializable schedules
Database System Concepts 3rd Edition 16.24 ©Silberschatz, Korth and
 Validation-Based
Protocol
■ Execution of transaction Ti is done in three phases.
1. Read and execution phase: Transaction Ti writes only to
temporary local variables
2. Validation phase: Transaction Ti performs a ``validation test''
to determine if local variables can be written without violating
serializability.
3. Write phase: If Ti is validated, the updates are applied to the
database; otherwise, Ti is rolled back.

■The three phases of concurrently executing transactions can be

interleaved, but each transaction must go through the three
phases in that order.
■Also called as optimistic concurrency control since transaction
executes fully in the hope that all will go well during validation

Database System Concepts 3rd Edition 16.25 ©Silberschatz, Korth and

 Validation-Based Protocol
(Cont.)

■ Each transaction Ti has 3 timestamps

 Start(Ti) : the time when Ti started its execution

 Validation(Ti): the time when Ti entered its validation phase

 Finish(Ti) : the time when Ti finished its write phase

■Serializability order is determined by timestamp given at validation
time, to increase concurrency. Thus TS(Ti) is
given
the value of Validation(Ti).

■This protocol is useful and gives greater degree of concurrency if

probability of conflicts is low. That is because the serializability
order is not pre-decided and relatively less transactions will have
to be rolled back.
Database System Concepts 3rd Edition 16.26 ©Silberschatz, Korth and
 Validation Test for Transaction Tj

■ If for all Ti with TS (Ti) < TS (Tj) either one of the following
condition holds:
 finish(Ti) < start(Tj)
 start(Tj) < finish(Ti) < validation(Tj) and the set of data items
written by Ti does not intersect with the set of data items read by
Tj .

then validation succeeds and Tj can be committed.

Otherwise, validation fails and Tj is aborted.
■Justification: Either first condition is satisfied, and there is no
overlapped execution, or second condition is satisfied and
1. the writes of Tj do not affect reads of Ti since they occur
after Ti
has finished its reads.
Database System Concepts 3rd Edition 16.27 ©Silberschatz, Korth and
 Schedule Produced by
Validation

■Example of schedule produced using validation

T14
read(B
) read(B)
T15 B:- B-50
read(A)
A:- A+50
read(A)
(validate)
display
(A+B) (validate)
write (B)
write (A)

Database System Concepts 3rd Edition 16.28 ©Silberschatz, Korth and

 Multiple
Granularity

■Allow data items to be of various sizes and define a hierarchy of

data granularities, where the small granularities are nested
within larger ones
■Can be represented graphically as a tree (but don't confuse with tree-
locking protocol)
■When a transaction locks a node in the tree explicitly, it implicitly
locks all the node's descendents in the same mode.

■Granularity of locking (level in tree where locking is done):

 fine granularity (lower in tree): high concurrency, high locking
overhead
 coarse granularity (higher in tree): low locking overhead, low
concurrency

Database System Concepts 3rd Edition 16.29 ©Silberschatz, Korth and

 Example of Granularity
Hierarchy

The highest level in the example hierarchy is the entire database.

The levels below are of type area, file and record in that order.

Database System Concepts 3rd Edition 16.30 ©Silberschatz, Korth and

 Intention Lock
Modes

■In addition to S and X lock modes, there are three additional lock modes
with multiple granularity:
 intention-shared (IS): indicates explicit locking at a lower level of the
tree but only with shared locks.
 intention-exclusive (IX): indicates explicit locking at a lower level with
exclusive or shared locks
 shared and intention-exclusive (SIX): the subtree rooted by that
node is locked explicitly in shared mode and explicit locking is being
done at a lower level with exclusive-mode locks.
■intention locks allow a higher level node to be locked in S or X mode
without having to check all descendent nodes.

Database System Concepts 3rd Edition 16.31 ©Silberschatz, Korth and

 Compatibility Matrix
with Intention Lock
Modes
■The compatibility matrix for all lock modes is:
IS IX S S IX X
IS     

IX     

S     

S IX     

X     

Database System Concepts 3rd Edition 16.32 ©Silberschatz, Korth and

 Multiple Granularity Locking
Scheme
■ Transaction T can lock a node Q, using the following rules:
i

1. The lock compatibility matrix must be observed.

2. The root of the tree must be locked first, and may be locked
in any mode.
3. A node Q can be locked by Ti in S or IS mode only if the parent
of Q is currently locked by Ti in either IX or IS
mode.
4. A node Q can be locked by Ti in X, SIX, or IX mode only if the
parent of Q is currently locked by Ti in either IX
or SIX mode.
5. Ti can lock a node only if it has not previously unlocked any node
(that is, Ti is two-phase).
6. Ti can unlock a node Q only if none of the children of Q are
currently locked by Ti.
■ Observe that locks are acquired in root-to-leaf order,
whereas they are released in leaf-to-root order.

Database System Concepts 3rd Edition 16.33 ©Silberschatz, Korth and

 Multiversion
Schemes

■Multiversion schemes keep old versions of data item to increase

concurrency.
 Multiversion Timestamp Ordering
 Multiversion Two-Phase Locking
■Each successful write results in the creation of a new version of the
data item written.

■Use timestamps to label versions.

■When a read(Q) operation is issued, select an appropriate
version
of Q based on the timestamp of the transaction, and return
the value of the selected version.
■reads never have to wait as an appropriate version is returned
immediately.

Database System Concepts 3rd Edition 16.34 ©Silberschatz, Korth and

 Multiversion Timestamp
Ordering

■ Each data item Q has a sequence of versions <Q1, Q2,...., Qm>.

Each version Qk contains three data fields:
 Content -- the value of version Qk.
 W-timestamp(Qk) -- timestamp of the transaction that created
(wrote) version Qk
 R-timestamp(Qk) -- largest timestamp of a transaction that
successfully read version Qk

■ when a transaction Ti creates a new version Qk of Q, Qk's W-

timestamp and R-timestamp are initialized to TS(Ti).
■ R-timestamp of Qk is updated whenever a transaction Tj
reads
Qk, and TS(Tj) > R-timestamp(Qk).
Database System Concepts 3rd Edition 16.35 ©Silberschatz, Korth and
 Multiversion Timestamp Ordering
(Cont)
■The multiversion timestamp scheme presented next ensures
serializability.
■ Suppose that transaction Ti issues a read(Q) or write(Q) operation.
Let Qk denote the version of Q whose write timestamp is the largest
write timestamp less than or equal to TS(Ti).
1. If transaction Ti issues a read(Q), then the value returned is the
content of version Qk.
2. If transaction Ti issues a write(Q), and if TS(Ti)
< R- timestamp(Qk), then transaction Ti is rolled
back. Otherwise, if TS(Ti) = W-timestamp(Qk),
the contents of Qk
are overwritten, otherwise a new version of Q is
created.
■ Reads always succeed; a write by Ti is rejected if some other
transaction Tj that (in the serialization order defined by the
timestamp
Database System values) should read T
Concepts 3 Edition
rd
's write, has already©Silberschatz,
16.36 read a Korth version
and
 Multiversion Two-Phase Locking

■Differentiates between read-only transactions and update

transactions
■Update transactions acquire read and write locks, and hold all
locks
up to the end of the transaction. That is, update transactions
follow rigorous two-phase locking.
 Each successful write results in the creation of a new version of the
data item written.
 each version of a data item has a single timestamp whose value is
obtained from a counter ts-counter that is incremented during
commit processing.

■Read-only transactions are assigned a timestamp by reading the current

value of ts-counter before they start execution; they
follow the multiversion timestamp-ordering protocol for
performing reads.

Database System Concepts 3rd Edition 16.37 ©Silberschatz, Korth and

 Multiversion Two-Phase Locking
(Cont.)
■When an update transaction wants to read a data item, it obtains a
shared lock on it, and reads the latest version.
■When it wants to write an item, it obtains X lock on; it then
creates a new version of the item and sets this version's
timestamp to .
■ When update transaction Ti completes, commit processing
occurs:
 Ti sets timestamp on the versions it has created to
ts-counter + 1
 Ti increments ts-counter by 1
■ Read-only transactions that start after Ti increments ts-
counter
will see the values updated by Ti.
■ Read-only transactions that start before Ti increments the
ts-counter will see the value before the updates by Ti.

■Only serializable schedules are produced.

Database System Concepts 3rd Edition 16.38 ©Silberschatz, Korth and
 Deadlock
Handling

■Consider the following two transactions:

T1: write (X) T 2: write(Y)
write(Y)
■Schedule with deadlock
T 1
write(X)
T2

lock-X on X
write (X)
lock-X on Y
write (X)
wait for lock-X on X
wait for lock-X on Y

Database System Concepts 3rd Edition 16.39 ©Silberschatz, Korth and

 Deadlock
Handling

■System is deadlocked if there is a set of transactions such that

every
transaction in the set is waiting for another transaction in the
set.
■Deadlock prevention protocols ensure that the system will never
enter into a deadlock state. Some prevention strategies :
 Require that each transaction locks all its data items before it begins
execution (predeclaration).
 Impose partial ordering of all data items and require that a
transaction can lock data items only in the order specified by the
partial order (graph-based protocol).

Database System Concepts 3rd Edition 16.40 ©Silberschatz, Korth and

 More Deadlock Prevention
Strategies
■Following schemes use transaction timestamps for the sake of
deadlock prevention alone.

■wait-die scheme — non-preemptive

 older transaction may wait for younger one to release data item.
Younger transactions never wait for older ones; they are rolled back
instead.
 a transaction may die several times before acquiring needed data
item

■wound-wait scheme — preemptive

 older transaction wounds (forces rollback) of younger transaction
instead of waiting for it. Younger transactions may wait for older
ones.
 may be fewer rollbacks than wait-die scheme.

Database System Concepts 3rd Edition 16.41 ©Silberschatz, Korth and

 Deadlock prevention
(Cont.)

■Both in wait-die and in wound-wait schemes, a rolled back

transactions is restarted with its original timestamp. Older
transactions thus have precedence over newer ones, and
starvation is hence avoided.

■Timeout-Based Schemes :
 a transaction waits for a lock only for a specified amount of
time.
After that, the wait times out and the transaction is rolled
back.
 thus deadlocks are not possible
 simple to implement; but starvation is possible. Also difficult to
determine good value of the timeout interval.

Database System Concepts 3rd Edition 16.42 ©Silberschatz, Korth and

 Deadlock
Detection

■Deadlocks can be described as a wait-for graph, which consists of a

pair G = (V,E),
 V is a set of vertices (all the transactions in the system)
 E is a set of edges; each element is an ordered pair Ti Tj.
■ If Ti  Tj is in E, then there is a directed edge from Ti to Tj,
implying that Ti is waiting for Tj to release a data item.
■ When Ti requests a data item currently being held by Tj, then the
edge Ti Tj is inserted in the wait-for graph. This edge is removed
only when Tj is no longer holding a data item needed by Ti.

■The system is in a deadlock state if and only if the wait-for graph

has a
cycle. Must invoke a deadlock-detection algorithm
periodically to look for cycles.
Database System Concepts 3rd Edition 16.43 ©Silberschatz, Korth and
 Deadlock Detection
(Cont.)

Wait-for graph without a cycle Wait-for graph with a cycle

Database System Concepts 3rd Edition 16.44 ©Silberschatz, Korth and

 Deadlock
Recovery

■When deadlock is detected :

 Some transaction will have to rolled back (made a victim) to break
deadlock. Select that transaction as victim that will incur minimum
cost.
 Rollback -- determine how far to roll back transaction
 Total rollback: Abort the transaction and then restart it.
 More effective to roll back transaction only as far as necessary to
break deadlock.
 Starvation happens if same transaction is always chosen as victim.
Include the number of rollbacks in the cost factor to avoid starvation

Database System Concepts 3rd Edition 16.45 ©Silberschatz, Korth and

 Insert and Delete
Operations

■If two-phase locking is used :

A delete operation may be performed only if the transaction
deleting the tuple has an exclusive lock on the tuple to be deleted.
 A transaction that inserts a new tuple into the database is given an
X-mode lock on the tuple

■Insertions and deletions can lead to the phantom phenomenon.

 A transaction that scans a relation (e.g., find all accounts in
Perryridge) and a transaction that inserts a tuple in the relation (e.g.,
insert a new account at Perryridge) may conflict in spite of not
accessing any tuple in common.
 If only tuple locks are used, non-serializable schedules can result:
the scan transaction may not see the new account, yet may be
serialized before the insert transaction.

Database System Concepts 3rd Edition 16.46 ©Silberschatz, Korth and

 Insert and Delete Operations
(Cont.)
■The transaction scanning the relation is reading information
that indicates what tuples the relation contains, while a
transaction inserting a tuple updates the same information.
 The information should be locked.

■One solution:
 Associate a data item with the relation, to represent the information
about what tuples the relation contains.
 Transactions scanning the relation acquire a shared lock in the data
item,
 Transactions inserting or deleting a tuple acquire an exclusive lock on
the data item. (Note: locks on the data item do not conflict with locks on
individual tuples.)
■Above protocol provides very low concurrency for
insertions/deletions.
■Index locking protocols provide higher concurrency while
preventing the phantom phenomenon, by requiring locks
on certain index buckets.
Database System Concepts 3rd Edition 16.47 ©Silberschatz, Korth and
 Index Locking
Protocol

■Every relation must have at least one index. Access to a relation must
be made only through one of the indices on the relation.
■ A transaction Ti that performs a lookup must lock all the index
buckets that it accesses, in S-mode.
■ A transaction Ti may not insert a tuple ti into a relation r
without updating all indices to r.
■ Ti must perform a lookup on every index to find all index buckets
that could have possibly contained a pointer to tuple ti, had it
existed already, and obtain locks in X-mode on all these index
buckets. Ti must also obtain locks in X-mode on all index buckets
that it modifies.

■The rules of the two-phase locking protocol must be observed.

Database System Concepts 3rd Edition 16.48 ©Silberschatz, Korth and
 Weak Levels of
Consistency
■Degree-two consistency: differs from two-phase locking in that
S-locks may be released at any time, and locks may be acquired
at any time
 X-locks must be held till end of transaction
 Serializability is not guaranteed, programmer must ensure that no
erroneous database state will occur]

■Cursor stability:
 For reads, each tuple is locked, read, and lock is immediately
released
 X-locks are held till end of transaction
 Special case of degree-two consistency

Database System Concepts 3rd Edition 16.49 ©Silberschatz, Korth and

 Weak Levels of Consistency in
SQL
■SQL allows non-serializable executions
 Serializable: is the default
 Repeatable read: allows only committed records to be read, and
repeating a read should return the same value (so read locks should
be retained)
 However, the phantom phenomenon need not be prevented

– T1 may see some records inserted by T2, but may not see
others inserted by T2
 Read committed: same as degree two consistency,
but most systems implement it as cursor-stability
 Read uncommitted: allows even uncommitted data to be
read

Database System Concepts 3rd Edition 16.50 ©Silberschatz, Korth and

 Concurrency in Index
Structures
■Indices are unlike other database items in that their only job is to help in
accessing data.
■Index-structures are typically accessed very often, much more than
other database items.
■Treating index-structures like other database items leads to low
concurrency. Two-phase locking on an index may result in
transactions executing practically one-at-a-time.
■It is acceptable to have nonserializable concurrent access to an index
as long as the accuracy of the index is maintained.
■In particular, the exact values read in an internal node of a
B+-tree are irrelevant so long as we land up in the correct leaf
node.
■There are index concurrency protocols where locks on internal nodes
are released early, and not in a two-phase fashion.

Database System Concepts 3rd Edition 16.51 ©Silberschatz, Korth and

 Concurrency in Index Structures
(Cont.)
■Example of index concurrency protocol:
■Use crabbing instead of two-phase locking on the nodes of the B+-tree,
as follows. During search/insertion/deletion:
 First lock the root node in shared mode.
 After locking all required children of a node in shared mode, release
the lock on the node.
 During insertion/deletion, upgrade leaf node locks to exclusive
mode.
 When splitting or coalescing requires changes to a parent, lock the
parent in exclusive mode.
■Above protocol can cause excessive deadlocks. Better protocols are
available; see Section 16.9 for one such protocol, the B-link
tree protocol

Database System Concepts 3rd Edition 16.52 ©Silberschatz, Korth and

End of
Chapter
 Partial Schedule Under Two-Phase
Locking

Database System Concepts 3rd Edition 16.54 ©Silberschatz, Korth and

 Incomplete Schedule With a Lock
Conversion

Database System Concepts 3rd Edition 16.55 ©Silberschatz, Korth and

 Lock Table

Database System Concepts 3rd Edition 16.56 ©Silberschatz, Korth and

 Tree-Structured Database Graph

Database System Concepts 3rd Edition 16.57 ©Silberschatz, Korth and

 Serializable Schedule Under the Tree
Protocol

Database System Concepts 3rd Edition 16.58 ©Silberschatz, Korth and

 Schedule 3

Database System Concepts 3rd Edition 16.59 ©Silberschatz, Korth and

 Schedule 4

Database System Concepts 3rd Edition 16.60 ©Silberschatz, Korth and

 Schedule 5, A Schedule Produced by Using
Validation

Database System Concepts 3rd Edition 16.61 ©Silberschatz, Korth and

 Granularity
Hierarchy

Database System Concepts 3rd Edition 16.62 ©Silberschatz, Korth and

 Compatibility
Matrix

Database System Concepts 3rd Edition 16.63 ©Silberschatz, Korth and

 Wait-for Graph With No
Cycle

Database System Concepts 3rd Edition 16.64 ©Silberschatz, Korth and

 Wait-for-graph With A
Cycle

Database System Concepts 3rd Edition 16.65 ©Silberschatz, Korth and

 Nonserializable Schedule with Degree-Two
Consistency

Database System Concepts 3rd Edition 16.66 ©Silberschatz, Korth and

 B+-Tree For account File with n = 3.

Database System Concepts 3rd Edition 16.67 ©Silberschatz, Korth and

 Insertion of “Clearview” Into the B+-Tree of Figure
16.21

Database System Concepts 3rd Edition 16.68 ©Silberschatz, Korth and

 Lock-Compatibility
Matrix

Database System Concepts 3rd Edition 16.69 ©Silberschatz, Korth and

Introduction to Transaction Processing Concepts and Theory
Outline

1 Introduction to Transaction Processing

2 Transaction and System Concepts
3 Desirable Properties of Transactions
4 Characterizing Schedules based on Recoverability
5 Characterizing Schedules based on Serializability
6 Transaction Support in SQL
Introduction to Transaction Processing

• Transaction: An executing program (process) that

includes one or more database access operations
– Read operations (database retrieval, such as SQL SELECT)
– Write operations (modify database, such as SQL INSERT, UPDATE,
DELETE)
– Transaction: A logical unit of database processing
– Example: Bank balance transfer of $100 dollars from a checking
account to a saving account in a BANK database
• Note: Each execution of a program is a distinct transaction with
different parameters
– Bank transfer program parameters: savings account number,
checking account number, transfer amount
Introduction to Transaction Processing
(cont.)
• A transaction (set of operations) may be:
– stand-alone, specified in a high level language like SQL
submitted interactively, or
– consist of database operations embedded within a
program (most transactions)
• Transaction boundaries: Begin and End transaction.
– Note: An application program may contain several
transactions separated by Begin and End transaction
boundaries
Introduction to Transaction Processing
(cont.)
• Transaction Processing Systems: Large multi-user
database systems supporting thousands of
concurrent transactions (user processes) per
minute
• Two Modes of Concurrency
– Interleaved processing: concurrent execution of
processes is interleaved in a single CPU
– Parallel processing: processes are concurrently
executed in multiple CPUs (Figure 21.1)
– Basic transaction processing theory assumes
interleaved concurrency
Introduction to Transaction Processing (cont.)

For transaction processing purposes, a simple

database model is used:
•A database - collection of named data items
•Granularity (size) of a data item - a field (data item
value), a record, or a whole disk block
– TP concepts are independent of granularity
•Basic operations on an item X:
– read_item(X): Reads a database item named X
into a program variable. To simplify our notation,
we assume that the program variable is also
named X.
– write_item(X): Writes the value of program
variable X into the database item named X.
Introduction to Transaction Processing
(cont.)
READ AND WRITE OPERATIONS:
 Basic unit of data transfer from the disk to the
computer main memory is one disk block (or page).
A data item X (what is read or written) will usually
be the field of some record in the database, although
it may be a larger unit such as a whole record or
even a whole block.
 read_item(X) command includes the following
steps:
• Find the address of the disk block that contains item X.
• Copy that disk block into a buffer in main memory (if that
disk block is not already in some main memory buffer).
• Copy item X from the buffer to the program variable named X.
Introduction to Transaction Processing
(cont.)
READ AND WRITE OPERATIONS (cont.):
 write_item(X) command includes the following
steps:
• Find the address of the disk block that contains
item X.
• Copy that disk block into a buffer in main memory
(if it is not already in some main memory buffer).
• Copy item X from the program variable named X
into its correct location in the buffer.
• Store the updated block from the buffer back to
disk (either immediately or at some later point in
time).
 Transaction Notation

• Figure 21.2 (next slide) shows two examples of

transactions
• Notation focuses on the read and write operations
• Can also write in shorthand notation:
– T1: b1; r1(X); w1(X); r1(Y); w1(Y); e1;
– T2: b2; r2(Y); w2(Y); e2;
• bi and ei specify transaction boundaries (begin and
end)
• i specifies a unique transaction identifier (TId)
 Why we need concurrency control

Without Concurrency Control, problems may occur

with concurrent transactions:
• Lost Update Problem.
Occurs when two transactions update the same data
item, but both read the same original value before
update (Figure 21.3(a), next slide)
• The Temporary Update (or Dirty Read) Problem.
This occurs when one transaction T1 updates a
database item X, which is accessed (read) by another
transaction T2; then T1 fails for some reason (Figure
21.3(b)); X was (read) by T2 before its value is
changed back (rolled back or UNDONE) after T1 fails
 Why we need concurrency control
(cont.)

• The Incorrect Summary Problem .

One transaction is calculating an aggregate summary
function on a number of records (for example, sum
(total) of all bank account balances) while other
transactions are updating some of these records (for
example, transferring a large amount between two
accounts, see Figure 21.3(c)); the aggregate function
may read some values before they are updated and
others after they are updated.
 Why we need concurrency control
(cont.)

• The Unrepeatable Read Problem .

A transaction T1 may read an item (say, available
seats on a flight); later, T1 may read the same item
again and get a different value because another
transaction T2 has updated the item (reserved seats
on the flight) between the two reads by T1
 Why recovery is needed

Causes of transaction failure:

1. A computer failure (system crash): A hardware or
software error occurs during transaction execution. If
the hardware crashes, the contents of the computer’s
internal main memory may be lost.
2. A transaction or system error : Some operation in the
transaction may cause it to fail, such as integer overflow
or division by zero. Transaction failure may also occur
because of erroneous parameter values or because of a
logical programming error. In addition, the user may
interrupt the transaction during its execution.
 Why recovery is needed (cont.)

3. Local errors or exception conditions detected by the

transaction:
- certain conditions necessitate cancellation of the
transaction. For example, data for the transaction may
not be found. A condition, such as insufficient account
balance in a banking database, may cause a
transaction, such as a fund withdrawal, to be canceled
- a programmed abort causes the transaction to fail.
4. Concurrency control enforcement: The concurrency
control method may decide to abort the transaction, to
be restarted later, because it violates serializability or
because several transactions are in a state of deadlock
(see Chapter 22).
 Why recovery is needed (cont.)

5. Disk failure: Some disk blocks may lose their data

because of a read or write malfunction or because of a
disk read/write head crash. This kind of failure and
item 6 are more severe than items 1 through 4.
6. Physical problems and catastrophes: This refers to
an endless list of problems that includes power or air-
conditioning failure, fire, theft, sabotage, overwriting
disks or tapes by mistake, and mounting of a wrong
tape by the operator.
Transaction and System Concepts

A transaction is an atomic unit of work that is either

completed in its entirety or not done at all. A transaction
passes through several states (Figure 21.4, similar to
process states in operating systems).
Transaction states:
•Active state (executing read, write operations)
•Partially committed state (ended but waiting for system
checks to determine success or failure)
•Committed state (transaction succeeded)
•Failed state (transaction failed, must be rolled back)
•Terminated State (transaction leaves system)
 Transaction and System Concepts
(cont.)
DBMS Recovery Manager needs system to keep track of the
following operations (in the system log file):
•begin_transaction: Start of transaction execution.
•read or write: Read or write operations on the database
items that are executed as part of a transaction.
•end_transaction: Specifies end of read and write
transaction operations have ended. System may still have
to check whether the changes (writes) introduced by
transaction can be permanently applied to the database
(commit transaction); or whether the transaction has to be
rolled back (abort transaction) because it violates
concurrency control or for some other reason.
 Transaction and System Concepts
(cont.)

Recovery manager keeps track of the following operations

(cont.):
•commit_transaction: Signals successful end of transaction;
any changes (writes) executed by transaction can be safely
committed to the database and will not be undone.
•abort_transaction (or rollback): Signals transaction has
ended unsuccessfully; any changes or effects that the
transaction may have applied to the database must be
undone.
 Transaction and System Concepts
(cont.)

System operations used during recovery (see Chapter

23):
•undo(X): Similar to rollback except that it applies
to a single write operation rather than to a whole
transaction.
•redo(X): This specifies that a write operation of a
committed transaction must be redone to ensure
that it has been applied permanently to the
database on disk.
 Transaction and System Concepts
(cont.)
The System Log File
•Is an append-only file to keep track of all operations of all
transactions in the order in which they occurred. This
information is needed during recovery from failures
•Log is kept on disk - not affected except for disk or
catastrophic failure
•As with other disk files, a log main memory buffer is kept
for holding the records being appended until the whole
buffer is appended to the end of the log file on disk
•Log is periodically backed up to archival storage (tape) to
guard against catastrophic failures
 Transaction and System Concepts
(cont.)
Types of records (entries) in log file:
• [start_transaction,T]: Records that transaction T has
started execution.
• [write_item,T,X,old_value,new_value]: T has changed
the value of item X from old_value to new_value.
• [read_item,T,X]: T has read the value of item X (not
needed in many cases).
• [end_transaction,T]: T has ended execution
• [commit,T]: T has completed successfully, and
committed.
• [abort,T]: T has been aborted.
Transaction and System Concepts (cont.)

The System Log (cont.):

 protocols for recovery that avoid cascading
rollbacks do not require that read operations
be written to the system log; most recovery
protocols fall in this category (see Chapter 23)
 strict protocols require simpler write entries
that do not include new_value (see Section
21.4).
 Transaction and System Concepts
(cont.)
Commit Point of a Transaction:
 Definition: A transaction T reaches its commit point
when all its operations that access the database have
been executed successfully and the effect of all the
transaction operations on the database has been
recorded in the log file (on disk). The transaction is
then said to be committed.
Transaction and System Concepts (cont.)

Commit Point of a Transaction (cont.):

 Log file buffers: Like database files on disk, whole disk blocks
must be read or written to main memory buffers.
 For log file, the last disk block (or blocks) of the file will be in
main memory buffers to easily append log entries at end of file.
 Force writing the log buffer: before a transaction reaches its
commit point, any main memory buffers of the log that have not
been written to disk yet must be copied to disk.
 Called force-writing the log buffers before committing a
transaction.
 Needed to ensure that any write operations by the transaction are
recorded in the log file on disk before the transaction commits
Desirable Properties of Transactions

Called ACID properties – Atomicity,

Consistency, Isolation, Durability:
•Atomicity: A transaction is an atomic unit of
processing; it is either performed in its entirety or
not performed at all.

•Consistency preservation: A correct execution of

the transaction must take the database from one
consistent state to another.
Desirable Properties of Transactions
(cont.)
ACID properties (cont.):
•Isolation: Even though transactions are executing
concurrently, they should appear to be executed in
isolation – that is, their final effect should be as if each
transaction was executed in isolation from start to finish.

•Durability or permanency: Once a transaction is

committed, its changes (writes) applied to the database
must never be lost because of subsequent failure.
Desirable Properties of Transactions
(cont.)

•Atomicity: Enforced by the recovery protocol.

•Consistency preservation: Specifies that each transaction
does a correct action on the database on its own. Application
programmers and DBMS constraint enforcement are
responsible for this.
•Isolation: Responsibility of the concurrency control
protocol.
•Durability or permanency: Enforced by the recovery
protocol.
Schedules of Transactions

• Transaction schedule (or history): When transactions are

executing concurrently in an interleaved fashion, the order of
execution of operations from the various transactions forms
what is known as a transaction schedule (or history).

• Figure 21.5 (next slide) shows 4 possible schedules (A, B, C, D)

of two transactions T1 and T2:
– Order of operations from top to bottom
– Each schedule includes same operations
– Different order of operations in each schedule
Schedules of Transactions (cont.)

• Schedules can also be displayed in more compact notation

• Order of operations from left to right
• Include only read (r) and write (w) operations, with
transaction id (1, 2, …) and item name (X, Y, …)
• Can also include other operations such as b (begin), e (end), c
(commit), a (abort)
• Schedules in Figure 21.5 would be displayed as follows:
– Schedule A: r1(X); w1(X); r1(Y); w1(Y); r2(X); w2(x);
– Schedule B: r2(X); w2(X); r1(X); w1(X); r1(Y); w1(Y);
– Schedule C: r1(X); r2(X); w1(X); r1(Y); w2(X); w1(Y);
– Schedule D: r1(X); w1(X); r2(X); w2(X); r1(Y); w1(Y);
 Schedules of Transactions
(cont.)
• Formal definition of a schedule (or history) S of n
transactions T1, T2, ..., Tn :
An ordering of all the operations of the transactions subject
to the constraint that, for each transaction Ti that participates
in S, the operations of Ti in S must appear in the same order
in which they occur in Ti.

Note: Operations from other transactions Tj can be interleaved

with the operations of Ti in S.
 Schedules of Transactions (cont.)

• For n transactions T1, T2, ..., Tn, where each Ti has mi read
and write operations, the number of possible schedules is (! is
factorial function):
(m1 + m2 + … + mn)! / ( (m1)! * (m2)! * … * (mn)! )

• Generally very large number of possible schedules

• Some schedules are easy to recover from after a failure, while
others are not
• Some schedules produce correct results, while others
produce incorrect results
• Rest of chapter characterizes schedules by classifying them
based on ease of recovery (recoverability) and correctness
(serializability)
 Characterizing Schedules
based on Recoverability
Schedules classified into two main classes:
•Recoverable schedule: One where no committed
transaction needs to be rolled back (aborted).
A schedule S is recoverable if no transaction T in S commits
until all transactions T’ that have written an item that T reads
have committed.
•Non-recoverable schedule: A schedule where a committed
transaction may have to be rolled back during recovery.
This violates Durability from ACID properties (a committed
transaction cannot be rolled back) and so non-recoverable
schedules should not be allowed.
Characterizing Schedules Based
on Recoverability (cont.)
• Example: Schedule A below is non-recoverable because T2
reads the value of X that was written by T1, but then T2
commits before T1 commits or aborts
• To make it recoverable, the commit of T2 (c2) must be
delayed until T1 either commits, or aborts (Schedule B)
• If T1 commits, T2 can commit
• If T1 aborts, T2 must also abort because it read a value that
was written by T1; this value must be undone (reset to its old
value) when T1 is aborted
– known as cascading rollback

• Schedule A: r1(X); w1(X); r2(X); w2(X); c2; r1(Y); w1(Y); c1 (or a1)
• Schedule B: r1(X); w1(X); r2(X); w2(X); r1(Y); w1(Y); c1 (or a1); ...
 Characterizing Schedules
based on Recoverability (cont.)
Recoverable schedules can be further refined:
•Cascadeless schedule: A schedule in which a transaction T2
cannot read an item X until the transaction T1 that last wrote X
has committed.
•The set of cascadeless schedules is a subset of the set of
recoverable schedules.

Schedules requiring cascaded rollback: A schedule in which

an uncommitted transaction T2 that read an item that was
written by a failed transaction T1 must be rolled back.
Characterizing Schedules Based
on Recoverability (cont.)
• Example: Schedule B below is not cascadeless because T2
reads the value of X that was written by T1 before T1 commits
• If T1 aborts (fails), T2 must also be aborted (rolled back)
resulting in cascading rollback
• To make it cascadeless, the r2(X) of T2 must be delayed until
T1 commits (or aborts and rolls back the value of X to its
previous value) – see Schedule C

• Schedule B: r1(X); w1(X); r2(X); w2(X); r1(Y); w1(Y); c1 (or a1);

• Schedule C: r1(X); w1(X); r1(Y); w1(Y); c1; r2(X); w2(X); ...
 Characterizing Schedules
based on Recoverability (cont.)
Cascadeless schedules can be further refined:
•Strict schedule: A schedule in which a transaction T2 can
neither read nor write an item X until the transaction T1 that last
wrote X has committed.
•The set of strict schedules is a subset of the set of cascadeless
schedules.
•If blind writes are not allowed, all cascadeless schedules are
also strict

Blind write: A write operation w2(X) that is not preceded by

a read r2(X).
Characterizing Schedules Based
on Recoverability (cont.)
• Example: Schedule C below is cascadeless and also strict
(because it has no blind writes)
• Schedule D is cascadeless, but not strict (because of the blind
write w3(X), which writes the value of X before T1 commits)
• To make it strict, w3(X) must be delayed until after T1
commits – see Schedule E

• Schedule C: r1(X); w1(X); r1(Y); w1(Y); c1; r2(X); w2(X); …

• Schedule D: r1(X); w1(X); w3(X); r1(Y); w1(Y); c1; r2(X); w2(X); …
• Schedule E: r1(X); w1(X); r1(Y); w1(Y); c1; w3(X); r2(X); w2(X); …
Characterizing Schedules Based
on Recoverability (cont.)
Summary:
• Many schedules can exist for a set of transactions
• The set of all possible schedules can be partitioned into two
subsets: recoverable and non-recoverable
• A subset of the recoverable schedules are cascadeless
• If blind writes are allowed, a subset of the cascadeless
schedules are strict
• If blind writes are not allowed, the set of cascadeless
schedules is the same as the set of strict schedules
Characterizing Schedules
based on Serializability
• Among the large set of possible schedules, we want to
characterize which schedules are guaranteed to give a
correct result
• The consistency preservation property of the ACID
properties states that: each transaction if executed on its
own (from start to finish) will transform a consistent
state of the database into another consistent state
• Hence, each transaction is correct on its own
 Characterizing Schedules
based on Serializability (cont.)
• Serial schedule: A schedule S is serial if, for every
transaction T participating in the schedule, all the
operations of T are executed consecutively (without
interleaving of operations from other transactions) in the
schedule. Otherwise, the schedule is called nonserial.
• Based on the consistency preservation property, any
serial schedule will produce a correct result (assuming no
inter-dependencies among different transactions)
 Characterizing Schedules
based on Serializability (cont.)
• Serial schedules are not feasible for performance
reasons:
– No interleaving of operations
– Long transactions force other transactions to wait
– System cannot switch to other transaction when a
transaction is waiting for disk I/O or any other event
– Need to allow concurrency with interleaving without
sacrificing correctness
 Characterizing Schedules
based on Serializability (cont.)
• Serializable schedule: A schedule S is serializable if it is
equivalent to some serial schedule of the same n
transactions.
• There are (n)! serial schedules for n transactions – a
serializable schedule can be equivalent to any of the
serial schedules
• Question: How do we define equivalence of schedules?
 Equivalence of Schedules

• Result equivalent: Two schedules are called result

equivalent if they produce the same final state of the
database.
• Difficult to determine without analyzing the internal
operations of the transactions, which is not feasible in
general.
• May also get result equivalence by chance for a
particular input parameter even though schedules are
not equivalent in general (see Figure 21.6, next slide)
 Equivalence of Schedules (cont.)

• Conflict equivalent: Two schedules are conflict

equivalent if the relative order of any two conflicting
operations is the same in both schedules.
• Commonly used definition of schedule equivalence
• Two operations are conflicting if:
– They access the same data item X
– They are from two different transactions
– At least one is a write operation
• Read-Write conflict example: r1(X) and w2(X)
• Write-write conflict example: w1(Y) and w2(Y)
 Equivalence of Schedules (cont.)

• Changing the order of conflicting operations generally

causes a different outcome
• Example: changing r1(X); w2(X) to w2(X); r1(X) means
that T1 will read a different value for X
• Example: changing w1(Y); w2(Y) to w2(Y); w1(Y) means
that the final value for Y in the database can be different
• Note that read operations are not conflicting; changing
r1(Z); r2(Z) to r2(Z); r1(Z) does not change the outcome
 Characterizing Scedules Based
on Serializability (cont.)
• Conflict equivalence of schedules is used to determine
which schedules are correct in general (serializable)

A schedule S is said to be serializable if it is conflict

equivalent to some serial schedule S’.
 Characterizing Schedules
based on Serializability (cont.)
• A serializable schedule is considered to be correct
because it is equivalent to a serial schedule, and any
serial schedule is considered to be correct
– It will leave the database in a consistent state.
– The interleaving is appropriate and will result in a
state as if the transactions were serially executed, yet
will achieve efficiency due to concurrent execution
and interleaving of operations from different
transactions.
 Characterizing Schedules based on
Serializability (cont.)
• Serializability is generally hard to check at run-time:
– Interleaving of operations is generally handled by the operating system
through the process scheduler
– Difficult to determine beforehand how the operations in a schedule will be
interleaved
– Transactions are continuously started and terminated
 Characterizing Schedules
Based on Serializability (cont.)
Practical approach:
•Come up with methods (concurrency control protocols)
to ensure serializability (discussed in Chapter 22)
•DBMS concurrency control subsystem will enforce the
protocol rules and thus guarantee serializability of
schedules
•Current approach used in most DBMSs:
– Use of locks with two phase locking (see Section 22.1)
 Characterizing Schedules
based on Serializability (cont.)
Testing for conflict serializability
Algorithm 21.1:
• Looks at only r(X) and w(X) operations in a schedule
• Constructs a precedence graph (serialization graph) – one
node for each transaction, plus directed edges
• An edge is created from Ti to Tj if one of the operations in Ti
appears before a conflicting operation in Tj
• The schedule is serializable if and only if the precedence graph
has no cycles.
 Characterizing Schedules
based on Serializability (cont.)
• View equivalence: A less restrictive definition of
equivalence of schedules than conflict serializability
when blind writes are allowed

• View serializability: definition of serializability based

on view equivalence. A schedule is view serializable if
it is view equivalent to a serial schedule.
 Characterizing Schedules based on
Serializability (cont.)

Two schedules are said to be view equivalent if the following

three conditions hold:
• The same set of transactions participates in S and S’, and S
and S’ include the same operations of those transactions.
• For any operation Ri(X) of Ti in S, if the value of X read was
written by an operation Wj(X) of Tj (or if it is the original
value of X before the schedule started), the same condition
must hold for the value of X read by operation Ri(X) of Ti in
S’.
• If the operation Wk(Y) of Tk is the last operation to write
item Y in S, then Wk(Y) of Tk must also be the last operation
to write item Y in S’.
 Characterizing Schedules based on
Serializability (cont.)

The premise behind view equivalence:

 Each read operation of a transaction reads the result
of the same write operation in both schedules.
 “The view”: the read operations are said to see the
the same view in both schedules.
 The final write operation on each item is the same
on both schedules resulting in the same final
database state in case of blind writes
 Characterizing Schedules
based on Serializability (cont.)
Relationship between view and conflict equivalence:
 The two are same under constrained write
assumption (no blind writes allowed)
 Conflict serializability is stricter than view
serializability when blind writes occur (a schedule
that is view serializable is not necessarily conflict
serialiable.
 Any conflict serializable schedule is also view
serializable, but not vice versa.
 Characterizing Schedules
based on Serializability (cont.)
Relationship between view and conflict equivalence
(cont):
Consider the following schedule of three transactions
T1: r1(X); w1(X); T2: w2(X); and T3: w3(X):
Schedule Sa: r1(X); w2(X); w1(X); w3(X); c1; c2; c3;

In Sa, the operations w2(X) and w3(X) are blind writes, since T2
and T3 do not read the value of X.

Sa is view serializable, since it is view equivalent to the serial

schedule T1, T2, T3. However, Sa is not conflict serializable,
since it is not conflict equivalent to any serial schedule.
 Characterizing Schedules
based on Serializability (cont.)
Other Types of Equivalence of Schedules
 Under special semantic constraints, schedules that
are otherwise not conflict serializable may work
correctly
 Using commutative operations of addition and
subtraction (which can be done in any order) certain
non-serializable transactions may work correctly;
known as debit-credit transactions
 Characterizing Schedules based
on Serializability (cont.)
Other Types of Equivalence of Schedules (cont.)
Example: bank credit/debit transactions on a given item are
separable and commutative.
Consider the following schedule S for the two transactions:
Sh : r1(X); w1(X); r2(Y); w2(Y); r1(Y); w1(Y); r2(X); w2(X);
Using conflict serializability, it is not serializable.
However, if it came from a (read,update, write) sequence as
follows:
r1(X); X := X – 10; w1(X); r2(Y); Y := Y – 20; w2(Y); r1(Y);
Y := Y + 10; w1(Y); r2(X); X := X + 20; w2(X);
Sequence explanation: debit, debit, credit, credit.
It is a correct schedule for the given semantics
Introduction to Transaction Support in
SQL
• A single SQL statement is always considered to be
atomic. Either the statement completes
execution without error or it fails and leaves the
database unchanged.
• With SQL, there is no explicit Begin Transaction
statement. Transaction initiation is done implicitly
when particular SQL statements are encountered.
• Every transaction must have an explicit end
statement, which is either a COMMIT or
ROLLBACK.
 Introduction to Transaction Support
in SQL (cont.)

Characteristics specified by a SET TRANSACTION

statement in SQL:
 Access mode: READ ONLY or READ WRITE. The default is
READ WRITE unless the isolation level of READ
UNCOMITTED is specified, in which case READ ONLY is
assumed.
 Diagnostic size n, specifies an integer value n, indicating
the number of conditions that can be held
simultaneously in the diagnostic area. (To supply run-
time feedback information to calling program for SQL
statements executed in program)
 Transaction Support in SQL (cont.)

Characteristics specified by a SET TRANSACTION

statement in SQL (cont.):
 Isolation level <isolation>, where <isolation> can be
READ UNCOMMITTED, READ COMMITTED, REPEATABLE
READ or SERIALIZABLE. The default is SERIALIZABLE.
If all transactions is a schedule specify isolation
level SERIALIZABLE, the interleaved execution of
transactions will adhere to serializability. However,
if any transaction in the schedule executes at a
lower level, serializability may be violated.
 Transaction Support in SQL (cont.)

Potential problem with lower isolation levels:

 Dirty Read: Reading a value that was written by a
transaction that failed.
 Nonrepeatable Read: Allowing another transaction to
write a new value between multiple reads of one
transaction.
A transaction T1 may read a given value from a table. If
another transaction T2 later updates that value and
then T1 reads that value again, T1 will see a different
value. Example: T1 reads the No. of seats on a flight.
Next, T2 updates that number (by reserving some seats).
If T1 reads the No. of seats again, it will see a different
value.
 Transaction Support in SQL (cont.)

Potential problem with lower isolation levels

(cont.):
 Phantoms: New row inserted after another transaction
accessing that row was started.
A transaction T1 may read a set of rows from a
table (say EMP), based on some condition specified
in the SQL WHERE clause (say DNO=5). Suppose a
transaction T2 inserts a new EMP row whose DNO
value is 5. T1 should see the new row (if equivalent
serial order is T2; T1) or not see it (if T1; T2). The
record that did not exist when T1 started is called a
phantom record.
 Transaction Support in SQL2 (cont.)

Sample SQL transaction:

EXEC SQL whenever sqlerror go to UNDO;
 EXEC SQL SET TRANSACTION
READ WRITE
DIAGNOSTICS SIZE 5
ISOLATION LEVEL SERIALIZABLE;
 EXEC SQL INSERT
INTO EMPLOYEE (FNAME, LNAME, SSN, DNO, SALARY)
VALUES ('Robert','Smith','991004321',2,35000);
EXEC SQL UPDATE EMPLOYEE
SET SALARY = SALARY * 1.1
WHERE DNO = 2;
EXEC SQL COMMIT;
GO TO THE_END;  
UNDO: EXEC SQL ROLLBACK;
THE_END: ...
Chapter 21 Summary

Introduction to Transaction Processing

Transaction and System Concepts

Desirable Properties of Transactions (ACID
properties)

Characterizing Schedules based on
Recoverability

Characterizing Schedules based on
Serializability

Transaction Support in SQL
 Distributed
Transactions
• Transaction may access data at several sites.
• Each site has a local transaction manager responsible
for:
– Maintaining a log for recovery purposes
– Participating in coordinating the concurrent execution of
the transactions executing at that site.
• Each site has a transaction coordinator, which is
responsible for:
– Starting the execution of transactions that originate at the
site.
– Distributing subtransactions at appropriate sites for
execution.
– Coordinating the termination of each transaction that
originates at the site, which may result in the transaction
being committed at all sites or aborted at all sites.

Dept. of CSE, IIT KGP

Distributed
Transactions
• Transaction may access data at several sites.
• Each site has a local transaction manager responsible
for:
– Maintaining a log for recovery purposes
– Participating in coordinating the concurrent execution of
the transactions executing at that site.
• Each site has a transaction coordinator, which is
responsible for:
– Starting the execution of transactions that originate at the
site.
– Distributing subtransactions at appropriate sites for
execution.
– Coordinating the termination of each transaction that
originates at the site, which may result in the transaction
being committed at all sites or aborted at all sites.
 Transaction System
Architecture

Dept. of CSE, IIT KGP

System Failure
Modes
• Failures unique to distributed systems:
– Failure of a site.
– Loss of massages
• Handled by network transmission control protocols
such as TCP-IP
– Failure of a communication link
• Handled by network protocols, by routing messages via
alternative links
– Network partition
• A network is said to be partitioned when it has been split
into two or more subsystems that lack any connection
between them
– Note: a subsystem may consist of a single node
• Network partitioning and site failures are
generally indistinguishable.
 Commit
Protocols
• Commit protocols are used to ensure atomicity across sites
– a transaction which executes at multiple sites must either be
committed at all the sites, or aborted at all the sites.
– not acceptable to have a transaction committed at one site and
aborted at another
• The two-phase commit (2PC) protocol is widely use
• The three-phase commit (3PC) protocol is more complicated and more expensive,
but avoids some drawbacks of two-phase commit protocol. This protocol is not
used in practice.
 Two Phase Commit Protocol
(2PC)
• Assumes fail-stop model – failed sites simply stop
working, and do not cause any other harm, such as
sending incorrect messages to other sites.

• Execution of the protocol is initiated by the coordinator

after the last step of the transaction has been reached.

• The protocol involves all the local sites at which the

transaction executed

• Let T be a transaction initiated at site Si, and let the

transaction coordinator at Si be Ci

Dept. of CSE, IIT KGP

 Phase 1: Obtaining a
Decision
• Coordinator asks all participants to prepare to commit
transaction Ti.
– Ci adds the records <prepare T> to the log and forces log to
stable storage
– sends prepare T messages to all sites at which T executed
• Upon receiving message, transaction manager at site
determines if it can commit the transaction
– if not, add a record <no T> to the log and send abort T
message to Ci
– if the transaction can be committed, then:
– add the record <ready T> to the log
– force all records for T to stable storage
– send ready T message to Ci

Dept. of CSE, IIT KGP

 Phase 2: Recording the
Decision
• T can be committed of C received a ready T message from all
i
the participating sites: otherwise T must be aborted.

• Coordinator adds a decision record, <commit T> or <abort T>,

to the log and forces record onto stable storage. Once the
record stable storage it is irrevocable (even if failures occur)

• Coordinator sends a message to each participant informing it

of the decision (commit or abort)

• Participants take appropriate action locally.

Dept. of CSE, IIT KGP

 Handling of Failures - Site
Failure
When site S recovers, it examines its log to determine the fate
i
of transactions active at the time of the failure.
• Log contain <commit T> record: site executes redo (T)
• Log contains <abort T> record: site executes undo (T)
• Log contains <ready T> record: site must consult Ci to
determine the fate of T.
– If T committed, redo (T)
– If T aborted, undo (T)
• The log contains no control records concerning T replies
that Sk failed before responding to the prepare T
message from Ci
– since the failure of Sk precludes the sending of such a
response C1 must abort T
– Sk must execute undo (T)

Dept. of CSE, IIT KGP

 Handling of Failures- Coordinator
Failure
• If coordinator fails while the commit protocol for T is
executing then participating sites must decide on T’s fate:
1. If an active site contains a <commit T> record in its log, then
T
must be committed.
2. If an active site contains an <abort T> record in its log, then T
must be aborted.
3. If some active participating site does not contain a <ready T>
record in its log, then the failed coordinator Ci cannot have
decided to commit T. Can therefore abort T.
4. If none of the above cases holds, then all active sites must have
a <ready T> record in their logs, but no additional control
records (such as <abort T> of <commit T>). In this case active
sites must wait for Ci to recover, to find decision.
• Blocking problem : active sites may have to wait for failed
coordinator to recover.
Dept. of CSE, IIT KGP
 Handling of Failures - Network
Partition
• If the coordinator and all its participants remain in one
partition, the failure has no effect on the commit
protocol.
• If the coordinator and its participants belong to several
partitions:
– Sites that are not in the partition containing the coordinator
think the coordinator has failed, and execute the protocol to
deal with failure of the coordinator.
• No harm results, but sites may still have to wait for decision
from coordinator.
• The coordinator and the sites are in the same partition as
the coordinator think that the sites in the other partition
have failed, and follow the usual commit protocol.
• Again, no harm results

Dept. of CSE, IIT KGP

 Recovery and Concurrency
Control
• In-doubt transactions have a <ready T>, but neither a
<commit T>, nor an <abort T> log record.

• The recovering site must determine the commit-abort status of

such transactions by contacting other sites; this can slow and
potentially block recovery.

• Recovery algorithms can note lock information in the log.

– Instead of <ready T>, write out <ready T, L> L = list of locks held by
T when the log is written (read locks can be omitted).
– For every in-doubt transaction T, all the locks noted in the
<ready T, L> log record are reacquired.

• After lock reacquisition, transaction processing can resume;

the commit or rollback of in-doubt transactions is
performed concurrently with the execution of new
transactions.
Dept. of CSE, IIT KGP
 Three Phase Commit
(3PC)
• Assumptions:
– No network partitioning
– At any point, at least one site must be up.
– At most K sites (participants as well as coordinator) can fail

• Phase 1: Obtaining Preliminary Decision: Identical to 2PC

Phase 1.
– Every site is ready to commit if instructed to do so

Dept. of CSE, IIT KGP

Three Phase Commit
(3PC)
• Phase 2 of 2PC is split into 2 phases, Phase 2 and Phase 3
of 3PC
– In phase 2 coordinator makes a decision as in 2PC (called the
pre-commit decision) and records it in multiple (at least K)
sites
– In phase 3, coordinator sends commit/abort message to all
participating sites,
• Under 3PC, knowledge of pre-commit decision can be used
to commit despite coordinator failure
– Avoids blocking problem as long as < K sites fail
• Drawbacks:
– higher overheads
– assumptions may not be satisfied in practice
 Chapter 10

Distributed databases
 Concepts

Distributed Database.
A logically interrelated collection of shared data (and a description of this
data), physically distributed over a computer network.

Distributed DBMS.
Software system that permits the management of the distributed database
and makes the distribution transparent to users.
 Concepts

Collection of logically-related shared data.

Data split into fragments.
Fragments may be replicated.
Fragments/replicas allocated to sites.
Sites linked by a communications network.
Data at each site is under control of a DBMS.
DBMSs handle local applications autonomously.
Each DBMS participates in at least one global application.
 e for a
DDBMS

site 1
GDD
DDBMS

DC LDBMS

GDD

Computer Network
DDBMS

DC

site 2 DB

LDBMS : Local DBMS component

DC : Data communication component
GDD : Global Data Dictionary
 The Ideal
Situation

A single application should be able to operate

transparently on data that is:
spread across a variety of different DBMS's
running on a variety of different machines
supported by a variety of different operating systems
connected together by a variety of different
communication networks
The distribution can be geographical or local
 Workable
definition

A distributed database system consists of a collection of sites connected

together via some kind of communications network, in which :
each site is a database system site in its own right;
the sites agree to work together, so that a user at any site can access data anywhere in
the network exactly as if the data were all stored at the user's own site
It is a logical union of real databases
It can be seen as a kind of partnership among individual local DBMS's

Difference with remote access or distributed processing systems

Temporary assumption: strict homogeneity
Distributed
DBMS

5
 Distributed
Processing

A centralized database that can be accessed over a

computer network.

6
 Parallel
DBMS

A DBMS running across multiple processors and disks

designed to execute operations in parallel, whenever
possible, to improve performance.
Based on premise that single processor systems can no
longer meet requirements for cost-effective scalability,
reliability, and performance.
Parallel DBMSs link multiple, smaller machines to achieve
same throughput as single, larger machine, with greater
scalability and reliability.
 Parallel
DBMS

Main architectures for parallel DBMSs are:

a: Shared memory.
b: Shared disk.
c: Shared nothing.
Parallel
DBMS

9
 Advantages
of DDBMSs

Organizational Structure
Shareability and Local Autonomy
Improved Availability
Improved Reliability
Improved Performance
Economics
Modular Growth
 Disadvanta
ges of
DDBMSs

Complexity
Cost
Security
Integrity Control More Difficult
Lack of Standards
Lack of Experience
Database Design More Complex
 Types of
DDBMS

Homogeneous DDBMS
Heterogeneous DDBMS
 Homogene
ous
DDBMS

All sites use same DBMS product.

Much easier to design and manage.
Approach provides incremental growth and allows
increased performance.
 Heterogene
ous
DDBMS

Sites may run different DBMS products, with possibly

different underlying data models.
Occurs when sites have implemented their own databases
and integration is considered later.
Translations required to allow for:
Different hardware.
Different DBMS products.
Different hardware and different DBMS products.
Typical solution is to use gateways.
 Open
Database
Access and
Interoperabili
ty

Open Group has formed a Working Group

to provide specifications that will create
database infrastructure environment where
there is:
Common SQL API that allows client
applications to be written that do not need
to know vendor of DBMS they are
accessing.
Common database protocol that enables DBMS
from one vendor to communicate directly with
 Multidataba
se System
(MDBS)

DDBMS in which each site maintains complete autonomy.

DBMS that resides transparently on top of existing

database and file systems and presents a single database
to its users.
Allows users to access and share data without requiring
physical database integration.
Non-federated MDBS (no local users) and federated MDBS
(FMDBS).
 Functions
of a
DDBMS

Expect DDBMS to have at least the functionality of a

DBMS.
Also to have following functionality:
Extended communication services.
Extended Data Dictionary.
Distributed query processing.
Extended concurrency control.
Extended recovery services.
 Reference
Architectu
re for
DDBMS

Due to diversity, no universally accepted architecture such

as the ANSI/SPARC 3-level architecture.
A reference architecture consists of:
Set of global external schemas.
Global conceptual schema (GCS).
Fragmentation schema and allocation schema.
Set of schemas for each local DBMS conforming to 3-
level ANSI/SPARC .
Some levels may be missing, depending on levels of
transparency supported.
Reference
Architectu
re for
DDBMS
 Reference
Architectu
re for
MDBS

In DDBMS, GCS is union of all local conceptual schemas.

In FMDBS, GCS is subset of local conceptual schemas
(LCS), consisting of data that each local system agrees to
share.
GCS of tightly coupled system involves integration of
either parts of LCSs or local external schemas.
FMDBS with no GCS is called loosely coupled.
Reference
Architectur
e for
Tightly-
Coupled
Federated
MDBS
DDBMS

25
 Distributed
Database
Design

Three key issues:

Fragmentation.
Allocation
Replication
 Distributed
Database
Fragmentation
Design
Relation may be divided into a number of sub-relations, which are then distributed.

Allocation
Each fragment is stored at site with "optimal" distribution.

Replication
Copy of fragment may be maintained at several sites.
 Fragmentat
ion

Definition and allocation of fragments carried out

strategically to achieve:
Locality of Reference
Improved Reliability and Availability
Improved Performance
Balanced Storage Capacities and Costs
Minimal Communication Costs.
Involves analyzing most important applications, based on
quantitative/qualitative information.
 Fragmentat
ion

Quantitative information may include:

frequency with which an application is run;
site from which an application is run;
performance criteria for transactions and
applications.
Qualitative information may include transactions that are
executed by application, type of access (read or write),
and predicates of read operations.
 Data
Allocation

Four alternative strategies regarding placement of data:

Centralized
Partitioned (or Fragmented)
Complete Replication
Selective Replication
 Data
Allocation

Centralized
Consists of single database and DBMS stored at one
site with users distributed across the network.

Partitioned
Database partitioned into disjoint fragments, each
fragment assigned to one site.
 Data
Allocation

Complete Replication
Consists of maintaining complete copy of database at
each site.

Selective Replication
Combination of partitioning, replication, and
centralization.
Compariso
n of
Strategies
for Data
Distribution

33
 Why
Fragment?

Usage
Applications work with views rather than entire
relations.
Efficiency
Data is stored close to where it is most frequently
used.
Data that is not needed by local applications is not
stored.
 Why
Fragment?

Parallelism
With fragments as unit of distribution, transaction
can be divided into several subqueries that operate
on fragments.
Security
Data not required by local applications is not stored
and so not available to unauthorized users.

Disadvantages
Performance
Integrity.
 Correctnes
s of
Fragmentat
ion

Three correctness rules:

Completeness
Reconstruction
Disjointness.
 Correctnes
s of
Fragmentat
ion

Completeness
If relation R is decomposed into fragments R1,
R2, ... Rn, each data item that can be found in R must
appear in at least one fragment.

Reconstruction
Must be possible to define a relational operation that will
reconstruct R from the fragments.
Reconstruction for horizontal fragmentation is Union
operation and Join for vertical .
 Correctnes
s of
Fragmentat
ion

Disjointness
If data item di appears in fragment Ri, then it should not
appear in any other fragment.
Exception: vertical fragmentation, where primary key
attributes must be repeated to allow reconstruction.
For horizontal fragmentation, data item is a tuple
For vertical fragmentation, data item is an attribute.
 Types of
Fragmentat
ion

Four types of fragmentation:

Horizontal
Vertical
Mixed
Derived.

Other possibility is no fragmentation:

If relation is small and not updated frequently, may

be better not to fragment relation.
Horizontal
and Vertical
Fragmentat
ion

41
Mixed
Fragmentat
ion
 Horizontal
Fragmentat
ion

This strategy is determined by looking at predicates used

by transactions.
Involves finding set of minimal (complete and relevant)
predicates.
Set of predicates is complete, if and only if, any two tuples
in same fragment are referenced with same probability by
any application.
Predicate is relevant if there is at least one application
that accesses fragments differently.
 Transparen
cies in a
DDBMS

Distribution Transparency

Fragmentation Transparency
Location Transparency
Replication Transparency
Local Mapping Transparency
Naming Transparency
 Transparen
cies in a
DDBMS

Transaction Transparency

Concurrency Transparency
Failure Transparency

Performance Transparency

DBMS Transparency
 Distribution
Transparen
cy

Distribution transparency allows user to perceive

database as single, logical entity.
If DDBMS exhibits distribution transparency, user does not
need to know:
data is fragmented (fragmentation transparency),
location of data items (location transparency),
otherwise call this local mapping transparency.
With replication transparency, user is unaware of
replication of fragments .
 Naming
Transparen
cy

Each item in a DDB must have a unique name.

DDBMS must ensure that no two sites create a database
object with same name.
One solution is to create central name server. However,
this results in:
loss of some local autonomy;
central site may become a bottleneck;
low availability; if the central site fails, remaining
sites cannot create any new objects.
 Transaction
Transparen
cy

Ensures that all distributed transactions maintain

distributed database’s integrity and consistency.
Distributed transaction accesses data stored at more than
one location.
Each transaction is divided into number of sub-
transactions, one for each site that has to be accessed.
DDBMS must ensure the indivisibility of both the global
transaction and each subtransactions.
 Concurrenc
y
Transparen
cy

All transactions must execute independently and be

logically consistent with results obtained if transactions
executed one at a time, in some arbitrary serial order.
Same fundamental principles as for centralized DBMS.
DDBMS must ensure both global and local transactions do
not interfere with each other.
Similarly, DDBMS must ensure consistency of all sub-
transactions of global transaction.
 Concurrenc
y
Transparen
cy

Replication makes concurrency more complex.

If a copy of a replicated data item is updated, update must
be propagated to all copies.
Could propagate changes as part of original transaction,
making it an atomic operation.
However, if one site holding copy is not reachable, then
transaction is delayed until site is reachable.
 Concurrenc
y
Transparen
cy

Could limit update propagation to only those sites

currently available. Remaining sites updated when they
become available again.
Could allow updates to copies to happen asynchronously,
sometime after the original update. Delay in regaining
consistency may range from a few seconds to several
hours.
 Failure
Transparen
cy

DDBMS must ensure atomicity and durability of global

transaction.
Means ensuring that sub-transactions of global
transaction either all commit or all abort.
Thus, DDBMS must synchronize global transaction to
ensure that all sub-transactions have completed
successfully before recording a final COMMIT for global
transaction.
Must do this in presence of site and network failures.
 Performanc
e
Transparen
cy

DDBMS must perform as if it were a centralized DBMS.

DDBMS should not suffer any performance
degradation due to distributed architecture.
DDBMS should determine most cost-effective
strategy to execute a request.
 Performanc
e
Transparen
cy

Distributed Query Processor (DQP) maps data request into

ordered sequence of operations on local databases.
Must consider fragmentation, replication, and allocation
schemas.
DQP has to decide:
which fragment to access;
which copy of a fragment to use;
which location to use.
 Performanc
e
Transparen
cy

DQP produces execution strategy optimized with respect

to some cost function.
Typically, costs associated with a distributed request
include:

I/O cost;
CPU cost;
communication cost.
 Date’s 12
Rules for a
DDBMS

0. Fundamental Principle
To the user, a distributed system should
look exactly like a non-distributed system.
1. Local Autonomy
2. No Reliance on a Central Site
3. Continuous Operation
4. Location Independence
5. Fragmentation Independence
6. Replication Independence
 Date’s 12
Rules for a
DDBMS

7. Distributed Query Processing

8. Distributed Transaction Processing
9. Hardware Independence
10. Operating System Independence
11. Network Independence
12. Database Independence

Last four rules are ideals.

Introduction to Transaction Processing Concepts and Theory
Outline

1 Introduction to Transaction Processing

2 Transaction and System Concepts
3 Desirable Properties of Transactions
4 Characterizing Schedules based on Recoverability
5 Characterizing Schedules based on Serializability
6 Transaction Support in SQL
Introduction to Transaction Processing

• Transaction: An executing program (process) that

includes one or more database access operations
– Read operations (database retrieval, such as SQL SELECT)
– Write operations (modify database, such as SQL INSERT, UPDATE,
DELETE)
– Transaction: A logical unit of database processing
– Example: Bank balance transfer of $100 dollars from a checking
account to a saving account in a BANK database
• Note: Each execution of a program is a distinct transaction with
different parameters
– Bank transfer program parameters: savings account number,
checking account number, transfer amount
Introduction to Transaction Processing
(cont.)
• A transaction (set of operations) may
be:
– stand-alone, specified in a high level language like SQL
submitted interactively, or
– consist of database operations embedded within a
program (most transactions)
• Transaction boundaries: Begin and End transaction.
– Note: An application program may contain several
transactions separated by Begin and End transaction
boundaries
Introduction to Transaction Processing
(cont.)
• Transaction Processing Systems: Large multi-user
database systems supporting thousands of
concurrent transactions (user processes) per
minute
• Two Modes of Concurrency
– Interleaved processing: concurrent execution of
processes is interleaved in a single CPU
– Parallel processing: processes are concurrently
executed in multiple CPUs (Figure 21.1)
– Basic transaction processing theory assumes
interleaved concurrency
Introduction to Transaction Processing (cont.)

For transaction processing purposes, a simple

database model is used:
•A database - collection of named data items
•Granularity (size) of a data item - a field (data item
value), a record, or a whole disk block
– TP concepts are independent of granularity
•Basic operations on an item X:
– read_item(X): Reads a database item named X
into a program variable. To simplify our notation,
we assume that the program variable is also
named X.
– write_item(X): Writes the value of program
variable X into the database item named X.
Introduction to Transaction Processing
(cont.)
READ AND WRITE OPERATIONS:
 Basic unit of data transfer from the disk to the
computer main memory is one disk block (or page).
A data item X (what is read or written) will usually
be the field of some record in the database, although
it may be a larger unit such as a whole record or
even a whole block.
 read_item(X) command includes the following
steps:
• Find the address of the disk block that contains item X.
• Copy that disk block into a buffer in main memory (if that
disk block is not already in some main memory buffer).
• Copy item X from the buffer to the program variable named X.
Introduction to Transaction Processing
(cont.)
READ AND WRITE OPERATIONS (cont.):
 write_item(X) command includes the following
steps:
• Find the address of the disk block that contains
item X.
• Copy that disk block into a buffer in main memory
(if it is not already in some main memory buffer).
• Copy item X from the program variable named X
into its correct location in the buffer.
• Store the updated block from the buffer back to
disk (either immediately or at some later point in
time).
 Transaction Notation

• Figure 21.2 (next slide) shows two examples of

transactions
• Notation focuses on the read and write operations
• Can also write in shorthand notation:
– T1: b1; r1(X); w1(X); r1(Y); w1(Y); e1;
– T2: b2; r2(Y); w2(Y); e2;
• bi and ei specify transaction boundaries (begin and
end)
• i specifies a unique transaction identifier (TId)
 Why we need concurrency control

Without Concurrency Control, problems may occur

with concurrent transactions:
• Lost Update Problem.
Occurs when two transactions update the same data
item, but both read the same original value before
update (Figure 21.3(a), next slide)
• The Temporary Update (or Dirty Read) Problem.
This occurs when one transaction T1 updates a
database item X, which is accessed (read) by another
transaction T2; then T1 fails for some reason (Figure
21.3(b)); X was (read) by T2 before its value is
changed back (rolled back or UNDONE) after T1 fails
 Why we need concurrency control
(cont.)

• The Incorrect Summary Problem .

One transaction is calculating an aggregate summary
function on a number of records (for example, sum
(total) of all bank account balances) while other
transactions are updating some of these records (for
example, transferring a large amount between two
accounts, see Figure 21.3(c)); the aggregate function
may read some values before they are updated and
others after they are updated.
 Why we need concurrency control
(cont.)

• The Unrepeatable Read Problem .

A transaction T1 may read an item (say, available
seats on a flight); later, T1 may read the same item
again and get a different value because another
transaction T2 has updated the item (reserved seats
on the flight) between the two reads by T1
 Why recovery is needed

Causes of transaction failure:

1. A computer failure (system crash): A hardware or
software error occurs during transaction execution. If
the hardware crashes, the contents of the computer’s
internal main memory may be lost.
2. A transaction or system error : Some operation in the
transaction may cause it to fail, such as integer overflow
or division by zero. Transaction failure may also occur
because of erroneous parameter values or because of a
logical programming error. In addition, the user may
interrupt the transaction during its execution.
 Why recovery is needed (cont.)

3. Local errors or exception conditions detected by the

transaction:
- certain conditions necessitate cancellation of the
transaction. For example, data for the transaction may
not be found. A condition, such as insufficient account
balance in a banking database, may cause a
transaction, such as a fund withdrawal, to be canceled
- a programmed abort causes the transaction to fail.
4. Concurrency control enforcement: The concurrency
control method may decide to abort the transaction, to
be restarted later, because it violates serializability or
because several transactions are in a state of deadlock
(see Chapter 22).
 Why recovery is needed (cont.)

5. Disk failure: Some disk blocks may lose their data

because of a read or write malfunction or because of a
disk read/write head crash. This kind of failure and
item 6 are more severe than items 1 through 4.
6. Physical problems and catastrophes: This refers to
an endless list of problems that includes power or air-
conditioning failure, fire, theft, sabotage, overwriting
disks or tapes by mistake, and mounting of a wrong
tape by the operator.
Transaction and System Concepts

A transaction is an atomic unit of work that is either

completed in its entirety or not done at all. A
transaction passes through several states (Figure 21.4,
similar to process states in operating systems).
Transaction states:
• Active state (executing read, write operations)
• Partially committed state (ended but waiting for
system checks to determine success or failure)
• Committed state (transaction succeeded)
• Failed state (transaction failed, must be rolled back)
• Terminated State (transaction leaves system)
 Transaction and System Concepts
(cont.)
DBMS Recovery Manager needs system to keep track of the
following operations (in the system log file):
•begin_transaction: Start of transaction execution.
•read or write: Read or write operations on the database
items that are executed as part of a transaction.
•end_transaction: Specifies end of read and write
transaction operations have ended. System may still have
to check whether the changes (writes) introduced by
transaction can be permanently applied to the database
(commit transaction); or whether the transaction has to be
rolled back (abort transaction) because it violates
concurrency control or for some other reason.
 Transaction and System Concepts
(cont.)

Recovery manager keeps track of the following operations

(cont.):
•commit_transaction: Signals successful end of transaction;
any changes (writes) executed by transaction can be safely
committed to the database and will not be undone.
•abort_transaction (or rollback): Signals transaction has
ended unsuccessfully; any changes or effects that the
transaction may have applied to the database must be
undone.
 Transaction and System Concepts
(cont.)

System operations used during recovery (see Chapter

23):
•undo(X): Similar to rollback except that it applies
to a single write operation rather than to a whole
transaction.
•redo(X): This specifies that a write operation of a
committed transaction must be redone to ensure
that it has been applied permanently to the
database on disk.
 Transaction and System Concepts
(cont.)
The System Log File
•Is an append-only file to keep track of all operations of all
transactions in the order in which they occurred. This
information is needed during recovery from failures
•Log is kept on disk - not affected except for disk or
catastrophic failure
•As with other disk files, a log main memory buffer is kept
for holding the records being appended until the whole
buffer is appended to the end of the log file on disk
•Log is periodically backed up to archival storage (tape) to
guard against catastrophic failures
 Transaction and System Concepts
(cont.)
Types of records (entries) in log file:
• [start_transaction,T]: Records that transaction T has
started execution.
• [write_item,T,X,old_value,new_value]: T has changed
the value of item X from old_value to new_value.
• [read_item,T,X]: T has read the value of item X (not
needed in many cases).
• [end_transaction,T]: T has ended execution
• [commit,T]: T has completed successfully, and
committed.
• [abort,T]: T has been aborted.
Transaction and System Concepts (cont.)

The System Log (cont.):

 protocols for recovery that avoid cascading
rollbacks do not require that read operations
be written to the system log; most recovery
protocols fall in this category (see Chapter 23)
 strict protocols require simpler write entries
that do not include new_value (see Section
21.4).
 Transaction and System Concepts
(cont.)
Commit Point of a Transaction:
 Definition: A transaction T reaches its commit point
when all its operations that access the database have
been executed successfully and the effect of all the
transaction operations on the database has been
recorded in the log file (on disk). The transaction is
then said to be committed.
Transaction and System Concepts (cont.)

Commit Point of a Transaction (cont.):

 Log file buffers: Like database files on disk, whole disk blocks
must be read or written to main memory buffers.
 For log file, the last disk block (or blocks) of the file will be in
main memory buffers to easily append log entries at end of file.
 Force writing the log buffer: before a transaction reaches its
commit point, any main memory buffers of the log that have not
been written to disk yet must be copied to disk.
 Called force-writing the log buffers before committing a
transaction.
 Needed to ensure that any write operations by the transaction are
recorded in the log file on disk before the transaction commits
Desirable Properties of Transactions

Called ACID properties – Atomicity,

Consistency, Isolation, Durability:
•Atomicity: A transaction is an atomic unit of
processing; it is either performed in its entirety or
not performed at all.

•Consistency preservation: A correct execution of

the transaction must take the database from one
consistent state to another.
Desirable Properties of Transactions
(cont.)
ACID properties (cont.):
•Isolation: Even though transactions are executing
concurrently, they should appear to be executed in
isolation – that is, their final effect should be as if each
transaction was executed in isolation from start to finish.

•Durability or permanency: Once a transaction is

committed, its changes (writes) applied to the database
must never be lost because of subsequent failure.
Desirable Properties of Transactions
(cont.)

• Atomicity: Enforced by the recovery protocol.

• Consistency preservation: Specifies that each transaction
does a correct action on the database on its own.
Application programmers and DBMS constraint
enforcement are responsible for this.
• Isolation: Responsibility of the concurrency control
protocol.
• Durability or permanency: Enforced by the recovery
protocol.
Schedules of Transactions

• Transaction schedule (or history): When transactions are

executing concurrently in an interleaved fashion, the order of
execution of operations from the various transactions forms
what is known as a transaction schedule (or history).

• Figure 21.5 (next slide) shows 4 possible schedules (A, B, C, D)

of two transactions T1 and T2:
– Order of operations from top to bottom
– Each schedule includes same operations
– Different order of operations in each schedule
Schedules of Transactions (cont.)

• Schedules can also be displayed in more compact notation

• Order of operations from left to right
• Include only read (r) and write (w) operations, with
transaction id (1, 2, …) and item name (X, Y, …)
• Can also include other operations such as b (begin), e (end), c
(commit), a (abort)
• Schedules in Figure 21.5 would be displayed as follows:
– Schedule A: r1(X); w1(X); r1(Y); w1(Y); r2(X); w2(x);
– Schedule B: r2(X); w2(X); r1(X); w1(X); r1(Y); w1(Y);
– Schedule C: r1(X); r2(X); w1(X); r1(Y); w2(X); w1(Y);
– Schedule D: r1(X); w1(X); r2(X); w2(X); r1(Y); w1(Y);
 Schedules of Transactions
(cont.)
• Formal definition of a schedule (or history) S of n
transactions T1, T2, ..., Tn :
An ordering of all the operations of the transactions subject
to the constraint that, for each transaction Ti that participates
in S, the operations of Ti in S must appear in the same order
in which they occur in Ti.

Note: Operations from other transactions Tj can be interleaved

with the operations of Ti in S.
 Schedules of Transactions (cont.)

• For n transactions T1, T2, ..., Tn, where each Ti has mi read
and write operations, the number of possible schedules is (! is
factorial function):
(m1 + m2 + … + mn)! / ( (m1)! * (m2)! * … * (mn)! )

• Generally very large number of possible schedules

• Some schedules are easy to recover from after a failure, while
others are not
• Some schedules produce correct results, while others
produce incorrect results
• Rest of chapter characterizes schedules by classifying them
based on ease of recovery (recoverability) and correctness
(serializability)
 Characterizing Schedules
based on Recoverability
Schedules classified into two main classes:
• Recoverable schedule: One where no committed
transaction needs to be rolled back (aborted).
A schedule S is recoverable if no transaction T in S commits
until all transactions T’ that have written an item that T reads
have committed.
• Non-recoverable schedule: A schedule where a
committed transaction may have to be rolled back during
recovery.
This violates Durability from ACID properties (a committed
transaction cannot be rolled back) and so non-recoverable
schedules should not be allowed.
Characterizing Schedules Based
on Recoverability (cont.)
• Example: Schedule A below is non-recoverable because T2
reads the value of X that was written by T1, but then T2
commits before T1 commits or aborts
• To make it recoverable, the commit of T2 (c2) must be
delayed until T1 either commits, or aborts (Schedule B)
• If T1 commits, T2 can commit
• If T1 aborts, T2 must also abort because it read a value that
was written by T1; this value must be undone (reset to its old
value) when T1 is aborted
– known as cascading rollback

• Schedule A: r1(X); w1(X); r2(X); w2(X); c2; r1(Y); w1(Y); c1 (or a1)
• Schedule B: r1(X); w1(X); r2(X); w2(X); r1(Y); w1(Y); c1 (or a1); ...
 Characterizing Schedules
based on Recoverability (cont.)
Recoverable schedules can be further refined:
•Cascadeless schedule: A schedule in which a transaction T2
cannot read an item X until the transaction T1 that last wrote X
has committed.
•The set of cascadeless schedules is a subset of the set of
recoverable schedules.

Schedules requiring cascaded rollback: A schedule in which

an uncommitted transaction T2 that read an item that was
written by a failed transaction T1 must be rolled back.
Characterizing Schedules Based
on Recoverability (cont.)
• Example: Schedule B below is not cascadeless because T2
reads the value of X that was written by T1 before T1 commits
• If T1 aborts (fails), T2 must also be aborted (rolled back)
resulting in cascading rollback
• To make it cascadeless, the r2(X) of T2 must be delayed until
T1 commits (or aborts and rolls back the value of X to its
previous value) – see Schedule C
• Schedule B: r1(X); w1(X); r2(X); w2(X); r1(Y); w1(Y); c1 (or a1);
• Schedule C: r1(X); w1(X); r1(Y); w1(Y); c1; r2(X); w2(X); ...
 Characterizing Schedules
based on Recoverability (cont.)
Cascadeless schedules can be further refined:
•Strict schedule: A schedule in which a transaction T2 can
neither read nor write an item X until the transaction T1 that last
wrote X has committed.
•The set of strict schedules is a subset of the set of cascadeless
schedules.
•If blind writes are not allowed, all cascadeless schedules are
also strict

Blind write: A write operation w2(X) that is not preceded by

a read r2(X).
Characterizing Schedules Based
on Recoverability (cont.)
• Example: Schedule C below is cascadeless and also strict
(because it has no blind writes)
• Schedule D is cascadeless, but not strict (because of the blind
write w3(X), which writes the value of X before T1 commits)
• To make it strict, w3(X) must be delayed until after T1
commits – see Schedule E
• Schedule C: r1(X); w1(X); r1(Y); w1(Y); c1; r2(X); w2(X); …
• Schedule D: r1(X); w1(X); w3(X); r1(Y); w1(Y); c1; r2(X); w2(X); …
• Schedule E: r1(X); w1(X); r1(Y); w1(Y); c1; w3(X); r2(X); w2(X); …
Characterizing Schedules Based
on Recoverability (cont.)
Summary:
• Many schedules can exist for a set of transactions
• The set of all possible schedules can be partitioned into two
subsets: recoverable and non-recoverable
• A subset of the recoverable schedules are cascadeless
• If blind writes are allowed, a subset of the cascadeless
schedules are strict
• If blind writes are not allowed, the set of cascadeless
schedules is the same as the set of strict schedules
 Characterizing Schedules
based on Serializability
• Among the large set of possible schedules, we want to characterize which
schedules are guaranteed to give a correct result
• The consistency preservation property of the ACID properties states that: each
transaction if executed on its own (from start to finish) will transform a consistent
state of the database into another consistent state
• Hence, each transaction is correct on its own
 Characterizing Schedules
based on Serializability (cont.)
• Serial schedule: A schedule S is serial if, for every transaction T participating in the
schedule, all the operations of T are executed consecutively (without interleaving
of operations from other transactions) in the schedule. Otherwise, the schedule is
called nonserial.
• Based on the consistency preservation property, any serial schedule will produce a
correct result (assuming no inter-dependencies among different transactions)
 Characterizing Schedules
based on Serializability (cont.)
• Serial schedules are not feasible for performance reasons:
– No interleaving of operations
– Long transactions force other transactions to wait
– System cannot switch to other transaction when a
transaction is waiting for disk I/O or any other event
– Need to allow concurrency with interleaving without
sacrificing correctness
 Characterizing Schedules
based on Serializability (cont.)
• Serializable schedule: A schedule S is serializable if it is equivalent to some serial
schedule of the same n transactions.
• There are (n)! serial schedules for n transactions – a serializable schedule can be
equivalent to any of the serial schedules
• Question: How do we define equivalence of schedules?
 Equivalence of Schedules

• Result equivalent: Two schedules are called result equivalent if they produce the
same final state of the database.
• Difficult to determine without analyzing the internal operations of the
transactions, which is not feasible in general.
• May also get result equivalence by chance for a particular input parameter even
though schedules are not equivalent in general (see Figure 21.6, next slide)
 Equivalence of Schedules (cont.)

• Conflict equivalent: Two schedules are conflict equivalent if the relative order of
any two conflicting operations is the same in both schedules.
• Commonly used definition of schedule equivalence
• Two operations are conflicting if:
– They access the same data item X
– They are from two different transactions
– At least one is a write operation
• Read-Write conflict example: r1(X) and w2(X)
• Write-write conflict example: w1(Y) and w2(Y)
 Equivalence of Schedules (cont.)

• Changing the order of conflicting operations generally causes a different outcome

• Example: changing r1(X); w2(X) to w2(X); r1(X) means that T1 will read a different
value for X
• Example: changing w1(Y); w2(Y) to w2(Y); w1(Y) means that the final value for Y in
the database can be different
• Note that read operations are not conflicting; changing r1(Z); r2(Z) to r2(Z); r1(Z)
does not change the outcome
 Characterizing Scedules Based
on Serializability (cont.)
• Conflict equivalence of schedules is used to determine which schedules are
correct in general (serializable)

A schedule S is said to be serializable if it is conflict equivalent to some serial schedule

S’.
 Characterizing Schedules
based on Serializability (cont.)
• A serializable schedule is considered to be correct
because it is equivalent to a serial schedule, and any
serial schedule is considered to be correct
– It will leave the database in a consistent state.
– The interleaving is appropriate and will result in a
state as if the transactions were serially executed, yet
will achieve efficiency due to concurrent execution
and interleaving of operations from different
transactions.
 Characterizing Schedules based on
Serializability (cont.)
• Serializability is generally hard to check at run-time:
– Interleaving of operations is generally handled by the operating system
through the process scheduler
– Difficult to determine beforehand how the operations in a schedule will be
interleaved
– Transactions are continuously started and terminated
 Characterizing Schedules
Based on Serializability (cont.)
Practical approach:
•Come up with methods (concurrency control protocols)
to ensure serializability (discussed in Chapter 22)
•DBMS concurrency control subsystem will enforce the
protocol rules and thus guarantee serializability of
schedules
•Current approach used in most DBMSs:
– Use of locks with two phase locking (see Section 22.1)
 Characterizing Schedules
based on Serializability (cont.)
Testing for conflict serializability
Algorithm 21.1:
• Looks at only r(X) and w(X) operations in a schedule
• Constructs a precedence graph (serialization graph) – one
node for each transaction, plus directed edges
• An edge is created from Ti to Tj if one of the operations in Ti
appears before a conflicting operation in Tj
• The schedule is serializable if and only if the precedence graph
has no cycles.
 Characterizing Schedules
based on Serializability (cont.)
• View equivalence: A less restrictive definition of
equivalence of schedules than conflict serializability
when blind writes are allowed

• View serializability: definition of serializability based

on view equivalence. A schedule is view serializable if
it is view equivalent to a serial schedule.
 Characterizing Schedules based on
Serializability (cont.)

Two schedules are said to be view equivalent if the following

three conditions hold:
• The same set of transactions participates in S and S’, and S
and S’ include the same operations of those transactions.
• For any operation Ri(X) of Ti in S, if the value of X read was
written by an operation Wj(X) of Tj (or if it is the original
value of X before the schedule started), the same condition
must hold for the value of X read by operation Ri(X) of Ti in
S’.
• If the operation Wk(Y) of Tk is the last operation to write
item Y in S, then Wk(Y) of Tk must also be the last operation
to write item Y in S’.
 Characterizing Schedules based on
Serializability (cont.)

The premise behind view equivalence:

 Each read operation of a transaction reads the result
of the same write operation in both schedules.
 “The view”: the read operations are said to see the
the same view in both schedules.
 The final write operation on each item is the same
on both schedules resulting in the same final
database state in case of blind writes
 Characterizing Schedules
based on Serializability (cont.)
Relationship between view and conflict equivalence:
 The two are same under constrained write
assumption (no blind writes allowed)
 Conflict serializability is stricter than view
serializability when blind writes occur (a schedule
that is view serializable is not necessarily conflict
serialiable.
 Any conflict serializable schedule is also view
serializable, but not vice versa.
 Characterizing Schedules
based on Serializability (cont.)
Relationship between view and conflict equivalence
(cont):
Consider the following schedule of three transactions
T1: r1(X); w1(X); T2: w2(X); and T3: w3(X):
Schedule Sa: r1(X); w2(X); w1(X); w3(X); c1; c2; c3;

In Sa, the operations w2(X) and w3(X) are blind writes, since T2
and T3 do not read the value of X.

Sa is view serializable, since it is view equivalent to the serial

schedule T1, T2, T3. However, Sa is not conflict serializable,
since it is not conflict equivalent to any serial schedule.
 Characterizing Schedules
based on Serializability (cont.)
Other Types of Equivalence of Schedules
 Under special semantic constraints, schedules that
are otherwise not conflict serializable may work
correctly
 Using commutative operations of addition and
subtraction (which can be done in any order) certain
non-serializable transactions may work correctly;
known as debit-credit transactions
 Characterizing Schedules based
on Serializability (cont.)
Other Types of Equivalence of Schedules (cont.)
Example: bank credit/debit transactions on a given item are
separable and commutative.
Consider the following schedule S for the two transactions:
Sh : r1(X); w1(X); r2(Y); w2(Y); r1(Y); w1(Y); r2(X); w2(X);
Using conflict serializability, it is not serializable.
However, if it came from a (read,update, write) sequence as
follows:
r1(X); X := X – 10; w1(X); r2(Y); Y := Y – 20; w2(Y); r1(Y);
Y := Y + 10; w1(Y); r2(X); X := X + 20; w2(X);
Sequence explanation: debit, debit, credit, credit.
It is a correct schedule for the given semantics
Introduction to Transaction Support in
SQL
• A single SQL statement is always considered to be
atomic. Either the statement completes
execution without error or it fails and leaves the
database unchanged.
• With SQL, there is no explicit Begin Transaction
statement. Transaction initiation is done implicitly
when particular SQL statements are encountered.
• Every transaction must have an explicit end
statement, which is either a COMMIT or
ROLLBACK.
 Introduction to Transaction Support
in SQL (cont.)

Characteristics specified by a SET TRANSACTION

statement in SQL:
 Access mode: READ ONLY or READ WRITE. The default is
READ WRITE unless the isolation level of READ
UNCOMITTED is specified, in which case READ ONLY is
assumed.
 Diagnostic size n, specifies an integer value n, indicating
the number of conditions that can be held
simultaneously in the diagnostic area. (To supply run-
time feedback information to calling program for SQL
statements executed in program)
 Transaction Support in SQL (cont.)

Characteristics specified by a SET TRANSACTION

statement in SQL (cont.):
 Isolation level <isolation>, where <isolation> can be
READ UNCOMMITTED, READ COMMITTED, REPEATABLE
READ or SERIALIZABLE. The default is SERIALIZABLE.
If all transactions is a schedule specify isolation
level SERIALIZABLE, the interleaved execution of
transactions will adhere to serializability. However,
if any transaction in the schedule executes at a
lower level, serializability may be violated.
 Transaction Support in SQL (cont.)

Potential problem with lower isolation levels:

 Dirty Read: Reading a value that was written by a
transaction that failed.
 Nonrepeatable Read: Allowing another transaction to
write a new value between multiple reads of one
transaction.
A transaction T1 may read a given value from a table. If
another transaction T2 later updates that value and
then T1 reads that value again, T1 will see a different
value. Example: T1 reads the No. of seats on a flight.
Next, T2 updates that number (by reserving some seats).
If T1 reads the No. of seats again, it will see a different
value.
 Transaction Support in SQL (cont.)

Potential problem with lower isolation levels

(cont.):
 Phantoms: New row inserted after another transaction
accessing that row was started.
A transaction T1 may read a set of rows from a
table (say EMP), based on some condition specified
in the SQL WHERE clause (say DNO=5). Suppose a
transaction T2 inserts a new EMP row whose DNO
value is 5. T1 should see the new row (if equivalent
serial order is T2; T1) or not see it (if T1; T2). The
record that did not exist when T1 started is called a
phantom record.
 Transaction Support in SQL2 (cont.)

Sample SQL transaction:

EXEC SQL whenever sqlerror go to UNDO;
 EXEC SQL SET TRANSACTION
READ WRITE
DIAGNOSTICS SIZE 5
ISOLATION LEVEL SERIALIZABLE;
 EXEC SQL INSERT
INTO EMPLOYEE (FNAME, LNAME, SSN, DNO, SALARY)
VALUES ('Robert','Smith','991004321',2,35000);
EXEC SQL UPDATE EMPLOYEE
SET SALARY = SALARY * 1.1
WHERE DNO = 2;
EXEC SQL COMMIT;
GO TO THE_END;  
UNDO: EXEC SQL ROLLBACK;
THE_END: ...
Chapter 21 Summary

Introduction to Transaction Processing

Transaction and System Concepts

Desirable Properties of Transactions (ACID
properties)

Characterizing Schedules based on
Recoverability

Characterizing Schedules based on
Serializability

Transaction Support in SQL
 Distributed
Transactions
• Transaction may access data at several sites.
• Each site has a local transaction manager responsible for:
– Maintaining a log for recovery purposes
– Participating in coordinating the concurrent execution of the
transactions executing at that site.
• Each site has a transaction coordinator, which is responsible
for:
– Starting the execution of transactions that originate at the site.
– Distributing subtransactions at appropriate sites for execution.
– Coordinating the termination of each transaction that originates
at the site, which may result in the transaction being committed
at all sites or aborted at all sites.

Dept. of CSE, IIT KGP

 Distributed
•Transactions
Transaction may access data at several sites.
• Each site has a local transaction manager responsible for:
– Maintaining a log for recovery purposes
– Participating in coordinating the concurrent execution of the
transactions executing at that site.
• Each site has a transaction coordinator, which is responsible
for:
– Starting the execution of transactions that originate at the site.
– Distributing subtransactions at appropriate sites for execution.
– Coordinating the termination of each transaction that originates
at the site, which may result in the transaction being committed
at all sites or aborted at all sites.
 Transaction System
Architecture

Dept. of CSE, IIT KGP

System Failure
Modes
• Failures unique to distributed systems:
– Failure of a site.
– Loss of massages
• Handled by network transmission control protocols such as
TCP-IP
– Failure of a communication link
• Handled by network protocols, by routing messages via
alternative links
– Network partition
• A network is said to be partitioned when it has been split into
two or more subsystems that lack any connection between
them
– Note: a subsystem may consist of a single node
• Network partitioning and site failures are generally
indistinguishable.
 Commit
Protocols
• Commit protocols are used to ensure atomicity across sites
– a transaction which executes at multiple sites must either be
committed at all the sites, or aborted at all the sites.
– not acceptable to have a transaction committed at one site and
aborted at another
• The two-phase commit (2PC) protocol is widely use
• The three-phase commit (3PC) protocol is more complicated and more expensive,
but avoids some drawbacks of two-phase commit protocol. This protocol is not
used in practice.
 Two Phase Commit Protocol
(2PC)
• Assumes fail-stop model – failed sites simply stop working, and
do not cause any other harm, such as sending incorrect
messages to other sites.

• Execution of the protocol is initiated by the coordinator after

the last step of the transaction has been reached.

• The protocol involves all the local sites at which the

transaction executed

• Let T be a transaction initiated at site Si, and let the transaction

coordinator at Si be Ci

Dept. of CSE, IIT KGP

 Phase 1: Obtaining a
Decision
• Coordinator asks all participants to prepare to commit
transaction Ti.
– Ci adds the records <prepare T> to the log and forces log to stable
storage
– sends prepare T messages to all sites at which T executed
• Upon receiving message, transaction manager at site determines
if it can commit the transaction
– if not, add a record <no T> to the log and send abort T
message to Ci
– if the transaction can be committed, then:
– add the record <ready T> to the log
– force all records for T to stable storage
– send ready T message to Ci

Dept. of CSE, IIT KGP

 Phase 2: Recording the
Decision
• T can be committed of C received a ready T message from all the
i
participating sites: otherwise T must be aborted.

• Coordinator adds a decision record, <commit T> or <abort T>, to the

log and forces record onto stable storage. Once the record stable
storage it is irrevocable (even if failures occur)

• Coordinator sends a message to each participant informing it of the

decision (commit or abort)

• Participants take appropriate action locally.

Dept. of CSE, IIT KGP

 Handling of Failures - Site
Failure
When site S recovers, it examines its log to determine the fate of
i
transactions active at the time of the failure.
• Log contain <commit T> record: site executes redo (T)
• Log contains <abort T> record: site executes undo (T)
• Log contains <ready T> record: site must consult Ci to determine the
fate of T.
– If T committed, redo (T)
– If T aborted, undo (T)
• The log contains no control records concerning T replies that Sk
failed before responding to the prepare T message from Ci
– since the failure of Sk precludes the sending of such a response C1 must
abort T
– Sk must execute undo (T)

Dept. of CSE, IIT KGP

 Handling of Failures- Coordinator
Failure
• If coordinator fails while the commit protocol for T is executing then
participating sites must decide on T’s fate:
1. If an active site contains a <commit T> record in its log, then T
must be committed.
2. If an active site contains an <abort T> record in its log, then T
must be aborted.
3. If some active participating site does not contain a <ready T> record in its
log, then the failed coordinator Ci cannot have decided to commit T. Can
therefore abort T.
4. If none of the above cases holds, then all active sites must have a <ready
T> record in their logs, but no additional control records (such as <abort
T> of <commit T>). In this case active sites must wait for Ci to recover, to
find decision.
• Blocking problem : active sites may have to wait for failed coordinator
to recover.

Dept. of CSE, IIT KGP

 Handling of Failures - Network
Partition
• If the coordinator and all its participants remain in one partition,
the failure has no effect on the commit protocol.
• If the coordinator and its participants belong to several
partitions:
– Sites that are not in the partition containing the coordinator think
the coordinator has failed, and execute the protocol to deal with
failure of the coordinator.
• No harm results, but sites may still have to wait for decision from
coordinator.
• The coordinator and the sites are in the same partition as the
coordinator think that the sites in the other partition have failed,
and follow the usual commit protocol.
• Again, no harm results

Dept. of CSE, IIT KGP

 Recovery and Concurrency
Control
• In-doubt transactions have a <ready T>, but neither a
<commit T>, nor an <abort T> log record.

• The recovering site must determine the commit-abort status of such

transactions by contacting other sites; this can slow and potentially
block recovery.

• Recovery algorithms can note lock information in the log.

– Instead of <ready T>, write out <ready T, L> L = list of locks held by
T when the log is written (read locks can be omitted).
– For every in-doubt transaction T, all the locks noted in the
<ready T, L> log record are reacquired.

• After lock reacquisition, transaction processing can resume; the commit

or rollback of in-doubt transactions is performed concurrently with the
execution of new transactions.

Dept. of CSE, IIT KGP

 Three Phase Commit
(3PC)
• Assumptions:
– No network partitioning
– At any point, at least one site must be up.
– At most K sites (participants as well as coordinator) can fail

• Phase 1: Obtaining Preliminary Decision: Identical to 2PC Phase 1.

– Every site is ready to commit if instructed to do so

Dept. of CSE, IIT KGP

 Three Phase Commit
(3PC)
• Phase 2 of 2PC is split into 2 phases, Phase 2 and Phase 3 of 3PC
– In phase 2 coordinator makes a decision as in 2PC (called the pre-
commit decision) and records it in multiple (at least K) sites
– In phase 3, coordinator sends commit/abort message to all
participating sites,
• Under 3PC, knowledge of pre-commit decision can be used to
commit despite coordinator failure
– Avoids blocking problem as long as < K sites fail
• Drawbacks:
– higher overheads
– assumptions may not be satisfied in practice

Dept. of CSE, IIT KGP

Chapter 12: Query Processing
 Chapter 12:
Query
Overview Processing
Measures of Query Cost
Selection Operation
Sorting
Join Operation
Other Operations
Evaluation of Expressions
 Basic Steps
in Query
1. Processing
Parsing and translation
2. Optimization
3. Evaluation
Basic Steps in Query Processing
(Cont.)
Parsing and translation
translate the query into its internal form. This is then translated into
relational algebra.
Parser checks syntax, verifies relations
Evaluation
The query-execution engine takes a query-evaluation plan, executes
that plan, and returns the answers to the query.
 Basic Steps in Query Processing :
Optimization
A relational algebra expression may have many equivalent expressions
E.g., salary75000(salary(instructor)) is equivalent to
salary(salary75000(instructor))
Each relational algebra operation can be evaluated using one of several
different algorithms
Correspondingly, a relational-algebra expression can be evaluated in
many ways.
Annotated expression specifying detailed evaluation strategy is called an
evaluation-plan.
E.g., can use an index on salary to find instructors with salary < 75000,
or can perform complete relation scan and discard instructors with salary
 75000
 Basic
Steps:
Optimizatio
n (Cont.)

Query Optimization: Amongst all equivalent

evaluation plans choose the one with lowest cost.
Cost is estimated using statistical information
from the
database catalog
e.g. number of tuples in each relation, size
of tuples, etc.
In this chapter we study
How to measure query costs
Algorithms for evaluating relational algebra
operations
 Measures
of Query
Cost
Cost is generally measured as total elapsed time for answering query
Many factors contribute to time cost
disk accesses, CPU, or even network communication
Typically disk access is the predominant cost, and is also relatively easy to
estimate. Measured by taking into account
Number of seeks * average-seek-cost
Number of blocks read * average-block-read-cost
Number of blocks written * average-block-write-cost
Cost to write a block is greater than cost to read a block
data is read back after being written to ensure that the write
was successful
 Measures
of Query
For simplicity we just useCost
the number of block transfers from disk and the
number of seeks as the cost measures
T
(Cont.)
t – time to transfer one block
tS – time for one seek
Cost for b block transfers plus S seeks
b * tT + S * tS
We ignore CPU costs for simplicity
Real systems do take CPU cost into account
We do not include cost to writing output to disk in our cost formulae
 Measures
of Query
Cost
Several algorithms can reduce disk IO by using extra buffer space
Amount of real memory available to buffer depends on other
(Cont.)
concurrent queries and OS processes, known only during execution
We often use worst case estimates, assuming only the minimum
amount of memory needed for the operation is available
Required data may be buffer resident already, avoiding disk I/O
But hard to take into account for cost estimation
 Selection
Operation
File scan
Algorithm A1 (linear search). Scan each file block and test all records to see
whether they satisfy the selection condition.
Cost estimate = br block transfers + 1 seek
br denotes number of blocks containing records from relation r
If selection is on a key attribute, can stop on finding record
cost = (br /2) block transfers + 1 seek
Linear search can be applied regardless of
selection condition or
ordering of records in the file, or
availability of indices
Note: binary search generally does not make sense since data is not stored
consecutively
except when there is an index available,
and binary search requires more seeks than index search
 Selections
Using
Indices
Index scan – search algorithms that use an index
selection condition must be on search-key of index.
A2 (primary index, equality on key). Retrieve a single record that satisfies
the corresponding equality condition
Cost = (hi + 1) * (tT + tS)
A3 (primary index, equality on nonkey) Retrieve multiple records.
Records will be on consecutive blocks
Let b = number of blocks containing matching records
Cost = hi * (tT + tS) + tS + tT * b
 Selections
Using
Indices
A4 (secondary index, equality on nonkey).
Retrieve a single record if the search-key is a candidate key
Cost = (hi + 1) * (tT + tS)
Retrieve multiple records if search-key is not a candidate key
each of n matching records may be on a different block
Cost = (hi + n) * (tT + tS)
Can be very expensive!
 Selections
Involving
Can implement selections of the form  (r) or  (r) by using
a linear file scan, Compariso AV AV

ns
or by using indices in the following ways:
A5 (primary index, comparison). (Relation is sorted on A)
For A  V(r) use index to find first tuple  v and scan relation sequentially
from there
For AV (r) just scan relation sequentially till first tuple > v; do not use
index
A6 (secondary index, comparison).
For A  V(r) use index to find first index entry  v and scan index
sequentially from there, to find pointers to records.
For AV (r) just scan leaf pages of index finding pointers to records, till first
entry > v
In either case, retrieve records that are pointed to
requires an I/O for each record
Linear file scan may be cheaper
 Implementa
tion of
Conjunction: 1 2. . . Complex
n(r)

Selections
A7 (conjunctive selection using one index).
Select a combination of i and algorithms A1 through A7 that results in the
least cost for i (r).
Test other conditions on tuple after fetching it into memory buffer.
A8 (conjunctive selection using composite index).
Use appropriate composite (multiple-key) index if available.
A9 (conjunctive selection by intersection of identifiers).
Requires indices with record pointers.
Use corresponding index for each condition, and take intersection of all the
obtained sets of record pointers.
Then fetch records from file
If some conditions do not have appropriate indices, apply test in memory.
 Algorithms
for
Disjunction:1 2 . . Complex
. n (r).
Selections
A10 (disjunctive selection by union of identifiers).
Applicable if all conditions have available indices.
Otherwise use linear scan.
Use corresponding index for each condition, and take union of all the
obtained sets of record pointers.
Then fetch records from file
Negation: (r)
Use linear scan on file
If very few records satisfy , and an index is applicable to 
Find satisfying records using index and fetch from file
 Sorting

We may build an index on the relation, and then use

the index to read the relation in sorted order. May
lead to one disk block access for each tuple.
For relations that fit in memory, techniques like
quicksort can be used. For relations that don’t fit in
memory, external
sort-merge is a good choice.
 External
Sort-Merge
Let M denote memory size (in pages).
1. Create sorted runs. Let i be 0 initially.
Repeatedly do the following till the end of the relation:
(a) Read M blocks of relation into memory
(b) Sort the in-memory blocks
(c) Write sorted data to run Ri; increment i.
Let the final value of i be N
2. Merge the runs (next slide)…..
 External
Sort-Merge
(Cont.)

2.Merge the runs (N-way merge). We assume (for

now) that N < M.
1. Use N blocks of memory to buffer input
runs, and 1 block to buffer output. Read the
first block of each run into its buffer page
2. repeat
1. Select the first record (in sort order)
among all buffer pages
2. Write the record to the output buffer. If
the output buffer is full write it to disk.
3. Delete the record from its input buffer
 External
Sort-Merge
(Cont.)
If N  M, several merge passes are required.
In each pass, contiguous groups of M - 1 runs are merged.
A pass reduces the number of runs by a factor of M -1, and creates
runs longer by the same factor.
E.g. If M=11, and there are 90 runs, one pass reduces the number
of runs to 9, each 10 times the size of the initial runs
Repeated passes are performed till all runs have been merged into
one.
Example: External Sorting Using Sort-Merge
a 19 a 19
g 24 d 31 a 14
b 14
a 19 g 24 a 19
c 33
d 31 b 14
b 14 d 31
c 33 c 33
c 33 e 16
b 14 d 7
e 16 g 24
e 16 d 21
r 16 d 21 d 31
a 14
d 21 m 3 e 16
d 7
m 3 r 16 g 24
d 21
p 2 m 3
m 3
d 7 a 14 p 2
p 2
a 14 d 7 r 16
r 16
p 2
initial sorted
relation runs runs output
create merge merge
runs pass–1 pass–2
 External
Merge Sort
(Cont.)

Cost analysis:
1 block per run leads to too many seeks during
merge
Instead use bb buffer blocks per run
 read/write bb blocks at a time
Can merge M/bb–1 runs in one pass
Total number of merge passes required: log M/bb–
1(br/M).
Block transfers for initial run creation as well as in
each pass is 2br
for final pass, we don’t count write cost
we ignore final write cost for all operations
since the output of an operation may be
 External
Merge Sort
(Cont.)

Cost of seeks
During run generation: one seek to read each
run and one seek to write each run
2 br / M
During the merge phase
Need 2 br / bb seeks for each merge pass
except the final one which does not
require a write
Total number of seeks:
2 br / M + br / bb (2 logM/bb–1(br / M)
-1)
 Join
Operation

Several different algorithms to implement joins

Nested-loop join
Block nested-loop join
Indexed nested-loop join
Merge-join
Hash-join
Choice based on cost estimate
Examples use the following information
Number of records of student: 5,000 takes:
10,000
Number of blocks of student: 100 takes:
 Nested-
Loop Join
To compute the theta join r  s
for each tuple tr in r do begin
for each tuple ts in s do begin
test pair (tr,ts) to see if they satisfy the join condition 
if they do, add tr • ts to the result.
end
end
r is called the outer relation and s the inner relation of the join.
Requires no indices and can be used with any kind of join condition.
Expensive since it examines every pair of tuples in the two relations.
 Nested-
Loop Join
(Cont.)
In the worst case, if there is enough memory only to hold one block of each relation,
the estimated cost is
nr  bs + br block transfers, plus
nr + b r seeks
If the smaller relation fits entirely in memory, use that as the inner relation.
Reduces cost to br + bs block transfers and 2 seeks
Assuming worst case memory availability cost estimate is
with student as outer relation:
5000  400 + 100 = 2,000,100 block transfers,
5000 + 100 = 5100 seeks
with takes as the outer relation
10000  100 + 400 = 1,000,400 block transfers and 10,400 seeks
If smaller relation (student) fits entirely in memory, the cost estimate will be 500 block
transfers.
Block nested-loops algorithm (next slide) is preferable.
 Block
Nested-
Loop
Variant of nested-loop join in which Join
every block of inner relation is
paired with every block of outer relation.
for each block Br of r do begin
for each block Bs of s do begin
for each tuple tr in Br do begin
for each tuple ts in Bs do begin
Check if (tr,ts) satisfy the join condition
if they do, add tr • ts to the result.
end
end
end
end
 Block
Nested-
Worst case estimate: b  bLoop
r s+ b blockJoin
r transfers + 2 * b seeks
r
Each block in the inner relation s is read once for each block in the
outer relation (Cont.)
Best case: br + bs block transfers + 2 seeks.
Improvements to nested loop and block nested loop algorithms:
In block nested-loop, use M — 2 disk blocks as blocking unit for outer
relations, where M = memory size in blocks; use remaining two blocks
to buffer inner relation and output
Cost = br / (M-2)  bs + br block transfers +
2 br / (M-2) seeks
If equi-join attribute forms a key or inner relation, stop inner loop on
first match
Scan inner loop forward and backward alternately, to make use of the
blocks remaining in buffer (with LRU replacement)
Use index on inner relation if available (next slide)
 Indexed
Nested-
Index lookups can replaceLoop file scans Join
if
join is an equi-join or natural join and
an index is available on the inner relation’s join attribute
Can construct an index just to compute a join.
For each tuple tr in the outer relation r, use the index to look up tuples in s
that satisfy the join condition with tuple tr.
Worst case: buffer has space for only one page of r, and, for each tuple in r,
we perform an index lookup on s.
Cost of the join: br (tT + tS) + nr  c
Where c is the cost of traversing index and fetching all matching s tuples for one
tuple or r
c can be estimated as cost of a single selection on s using the join condition.
If indices are available on join attributes of both r and s,
use the relation with fewer tuples as the outer relation.
 Example of
Nested-
Loop Join
Costs

Compute student takes, with student as the outer

relation.
Let takes have a primary B+-tree index on the attribute ID,
which contains 20 entries in each index node.
Since takes has 10,000 tuples, the height of the tree is 4,
and one more access is needed to find the actual data
student has 5000 tuples
Cost of block nested loops join
400*100 + 100 = 40,100 block transfers + 2 * 100 =
200 seeks
assuming worst case memory
may be significantly less with more memory
Cost of indexed nested loops join
 Merge-Join
1. Sort both relations on their join attribute (if not already sorted on the
join attributes).
2. Merge the sorted relations to join them
1. Join step is similar to the merge stage of the sort-merge algorithm.
2. Main difference is handling of duplicate values in join attribute —
every pair with same value on join attribute must be matched
3. Detailed algorithm in book
 Merge-Join
(Cont.)
Can be used only for equi-joins and natural joins
Each block needs to be read only once (assuming all tuples for any given value of
the join attributes fit in memory
Thus the cost of merge join is:
br + bs block transfers + br / bb + bs / bb seeks
+ the cost of sorting if relations are unsorted.
hybrid merge-join: If one relation is sorted, and the other has a secondary B+-
tree index on the join attribute
Merge the sorted relation with the leaf entries of the B+-tree .
Sort the result on the addresses of the unsorted relation’s tuples
Scan the unsorted relation in physical address order and merge with
previous result, to replace addresses by the actual tuples
Sequential scan more efficient than random lookup
 Hash-Join

Applicable for equi-joins and natural joins.

A hash function h is used to partition tuples of both relations
h maps JoinAttrs values to {0, 1, ..., n}, where JoinAttrs denotes the common
attributes of r and s used in the natural join.
r0, r1, . . ., rn denote partitions of r tuples
Each tuple tr  r is put in partition ri where i = h(tr [JoinAttrs]).
r0,, r1. . ., rn denotes partitions of s tuples
Each tuple ts s is put in partition si, where i = h(ts [JoinAttrs]).

Note: In book, ri is denoted as Hri, si is denoted as Hsi and

n is denoted as nh.
Hash-Join
(Cont.)
 Hash-Join
(Cont.)
r tuples in ri need only to be compared with s tuples in si Need not
be compared with s tuples in any other partition, since:
an r tuple and an s tuple that satisfy the join condition will have
the same value for the join attributes.
If that value is hashed to some value i, the r tuple has to be in ri
and the s tuple in si.
 Hash-Join
Algorithm
The hash-join of r and s is computed as follows.
1. Partition the relation s using hashing function h. When partitioning
a relation, one block of memory is reserved as the output buffer for
each partition.
2. Partition r similarly.
3. For each i:
(a)Load si into memory and build an in-memory hash index on it
using the join attribute. This hash index uses a different hash
function than the earlier one h.
(b) Read the tuples in ri from the disk one by one. For each
tuple tr locate each matching tuple ts in si using the in-memory
hash index. Output the concatenation of their attributes.

Relation s is called the build input and r is called the probe input.
 Hash-Join
algorithm
The value n and the hash (Cont.)
function h is chosen such that each si should fit
in memory.
Typically n is chosen as bs/M * f where f is a “fudge factor”,
typically around 1.2
The probe relation partitions si need not fit in memory
Recursive partitioning required if number of partitions n is greater than
number of pages M of memory.
instead of partitioning n ways, use M – 1 partitions for s
Further partition the M – 1 partitions using a different hash function
Use same partitioning method on r
Rarely required: e.g., with block size of 4 KB, recursive partitioning
not needed for relations of < 1GB with memory size of 2MB, or
relations of < 36 GB with memory of 12 MB
 Handling of
Overflows
Partitioning is said to be skewed if some partitions have significantly more tuples
than some others
Hash-table overflow occurs in partition si if si does not fit in memory. Reasons could
be
Many tuples in s with same value for join attributes
Bad hash function
Overflow resolution can be done in build phase
Partition si is further partitioned using different hash function.
Partition ri must be similarly partitioned.
Overflow avoidance performs partitioning carefully to avoid overflows during build
phase
E.g. partition build relation into many partitions, then combine them
Both approaches fail with large numbers of duplicates
Fallback option: use block nested loops join on overflowed partitions
 Cost of
Hash-Join
If recursive partitioning is not required: cost of hash join is
3(br + bs) +4  nh block transfers +
2( br / bb + bs / bb) seeks
If recursive partitioning required:
number of passes required for partitioning build relation s to less than
M blocks per partition is logM/bb–1(bs/M)
best to choose the smaller relation as the build relation.
Total cost estimate is:
2(br + bs) logM/bb–1(bs/M) + br + bs block transfers +
2(br / bb + bs / bb) logM/bb–1(bs/M)  seeks
If the entire build input can be kept in main memory no partitioning is
required
Cost estimate goes down to br + bs.
 Example of
Cost of
instructor teaches
Hash-Join
Assume that memory size is 20 blocks
binstructor= 100 and bteaches = 400.
instructor is to be used as build input. Partition it into five partitions, each
of size 20 blocks. This partitioning can be done in one pass.
Similarly, partition teaches into five partitions,each of size 80. This is also
done in one pass.
Therefore total cost, ignoring cost of writing partially filled blocks:
3(100 + 400) = 1500 block transfers +
2( 100/3 + 400/3) = 336 seeks
 Hybrid
Hash–Join
Useful when memory sized are relatively large, and the build input is bigger
than memory.
Main feature of hybrid hash join:
Keep the first partition of the build relation in memory.
E.g. With memory size of 25 blocks, instructor can be partitioned into five
partitions, each of size 20 blocks.
Division of memory:
The first partition occupies 20 blocks of memory
1 block is used for input, and 1 block each for buffering the other 4
partitions.
teaches is similarly partitioned into five partitions each of size 80
the first is used right away for probing, instead of being written out
Cost of 3(80 + 320) + 20 +80 = 1300 block transfers for
hybrid hash join, instead of 1500 with plain hash-join.
Hybrid hash-join most useful if M >>

bs
 Complex
Joins
Join with a conjunctive condition:
r 1  2...   n s
Either use nested loops/block nested loops, or
Compute the result of one of the simpler joins r i s
final result comprises those tuples in the intermediate result that
satisfy the remaining conditions
1  . . .  i –1  i +1  . . .  n
Join with a disjunctive condition
r 1  2 ...  n s
Either use nested loops/block nested loops, or
Compute as the union of the records in individual joins r  i s:
(r 1 s)  (r 2 s)  . . .  (r n s)
 Other
Operations
Duplicate elimination can be implemented via hashing or sorting.
On sorting duplicates will come adjacent to each other, and all but one
set of duplicates can be deleted.
Optimization: duplicates can be deleted during run generation as well as
at intermediate merge steps in external sort-merge.
Hashing is similar – duplicates will come into the same bucket.
Projection:
perform projection on each tuple
followed by duplicate elimination.
 Other
Operations
: in a manner similar to duplicate elimination.
Aggregation can be implemented
Sorting or hashing can be used to bring tuples in the same group
together, and then theAggregatio
aggregate functions can be applied on each group.
n
Optimization: combine tuples in the same group during run generation
and intermediate merges, by computing partial aggregate values
For count, min, max, sum: keep aggregate values on tuples found so
far in the group.
When combining partial aggregate for count, add up the
aggregates
For avg, keep sum and count, and divide sum by count at the end
 Other
Operations
Set operations (,  and ): : Setcan either use variant of merge-join after
sorting, or variant of hash-join.
Operations
E.g., Set operations using hashing:
1. Partition both relations using the same hash function
2. Process each partition i as follows.
1. Using a different hashing function, build an in-memory hash index
on ri.
2. Process si as follows
 r  s:
1. Add tuples in si to the hash index if they are not already in
it.
2. At end of si add the tuples in the hash index to the result.
 Other
Operations
E.g., Set operations using:hashing:
Set
1. as before partition r and s,
2. as before, processOperations
each partition i as follows
1. build a hash index on ri
2. Process si as follows
 r  s:
1. output tuples in si to the result if they are already
there in the hash index
 r – s:
1. for each tuple in si, if it is there in the hash index,
delete it from the index.
2. At end of si add remaining tuples in the hash index to
the result.
 Other
Operations
: Outer
Outer join can be computed either as Join
A join followed by addition of null-padded non-participating tuples.
by modifying the join algorithms.
Modifying merge join to compute r s
In r s, non participating tuples are those in r – R(r s)
Modify merge-join to compute r s:
During merging, for every tuple tr from r that do not match any
tuple in s, output tr padded with nulls.
Right outer-join and full outer-join can be computed similarly.
 Other
Operations
: Outer
Modifying hash join to compute r sJoin
If r is probe relation, output non-matching r tuples padded with nulls
If r is build relation, when probing keep track of which
r tuples matched s tuples. At end of si output
non-matched r tuples padded with nulls
 Evaluation
of
Expression
s

So far: we have seen algorithms for individual

operations
Alternatives for evaluating an entire expression tree
Materialization: generate results of an
expression whose inputs are relations or are
already computed, materialize (store) it on disk.
Repeat.
Pipelining: pass on tuples to parent operations
even as an operation is being executed
We study above alternatives in more detail
 Materializat
ion
Materialized evaluation: evaluate one operation at a time, starting at the
lowest-level. Use intermediate results materialized into temporary relations
to evaluate next-level operations.
E.g., in figure below, compute and store

 building"Watson " (department )

then compute the store its join with instructor, and finally compute the
projection on name.
 Materializat
ion (Cont.)

Materialized evaluation is always applicable

Cost of writing results to disk and reading them back
can be quite high
Our cost formulas for operations ignore cost of
writing results to disk, so
Overall cost = Sum of costs of individual
operations +
cost of writing intermediate
results to disk
Double buffering: use two output buffers for each
operation, when one is full write it to disk while the
 Pipelining

Pipelined evaluation : evaluate several operations simultaneously, passing

the results of one operation on to the next.
E.g., in previous expression tree, don’t store result of

 building"Watson " ( department )

instead, pass tuples directly to the join.. Similarly, don’t store result of
join, pass tuples directly to projection.
Much cheaper than materialization: no need to store a temporary relation to
disk.
Pipelining may not always be possible – e.g., sort, hash-join.
For pipelining to be effective, use evaluation algorithms that generate output
tuples even as tuples are received for inputs to the operation.
Pipelines can be executed in two ways: demand driven and producer driven
 Pipelining
(Cont.)
In demand driven or lazy evaluation
system repeatedly requests next tuple from top level operation
Each operation requests next tuple from children operations as required, in order
to output its next tuple
In between calls, operation has to maintain “state” so it knows what to return next
In producer-driven or eager pipelining
Operators produce tuples eagerly and pass them up to their parents
Buffer maintained between operators, child puts tuples in buffer, parent
removes tuples from buffer
if buffer is full, child waits till there is space in the buffer, and then generates
more tuples
System schedules operations that have space in output buffer and can process
more input tuples
Alternative name: pull and push models of pipelining
 Pipelining
(Cont.)
Implementation of demand-driven pipelining
Each operation is implemented as an iterator implementing the following
operations
open()
E.g. file scan: initialize file scan
state: pointer to beginning of file
E.g.merge join: sort relations;
state: pointers to beginning of sorted relations
next()
E.g. for file scan: Output next tuple, and advance and store file
pointer
E.g. for merge join: continue with merge from earlier state till
next output tuple is found. Save pointers as iterator state.
close()
 Evaluation
Algorithms
Some algorithms are not forable to output results even as they get input tuples
E.g. merge join, or hash join
Pipelining
intermediate results written to disk and then read back
Algorithm variants to generate (at least some) results on the fly, as input
tuples are read in
E.g. hybrid hash join generates output tuples even as probe relation tuples in the
in-memory partition (partition 0) are read in
Double-pipelined join technique: Hybrid hash join, modified to buffer partition 0
tuples of both relations in-memory, reading them as they become available, and
output results of any matches between partition 0 tuples
When a new r0 tuple is found, match it with existing s0 tuples, output
matches, and save it in r0
Symmetrically for s0 tuples
End of Chapter
Figure
12.02
 Selection
Operation
(Cont.)
Old-A2 (binary search). Applicable if selection is an equality comparison on
the attribute on which file is ordered.
Assume that the blocks of a relation are stored contiguously
Cost estimate (number of disk blocks to be scanned):
cost of locating the first tuple by a binary search on the blocks
log2(br) * (tT + tS)
If there are multiple records satisfying selection
Add transfer cost of the number of blocks containing records
that satisfy selection condition
Will see how to estimate this cost in Chapter 13