Edited with the trial version of

Foxit Advanced PDF Editor

To remove this notice, visit:
www.foxitsoftware.com/shopping

CS DATABASE MANAGEMENT SYSTEMS

6302
NOTES

INTRODUCTION TO DATABASE
MANAGEMENT SYSTEMS
Learning Objectives
 To Learn the difference between File systems and Database systems
 To learn about various Data models
 To Study the Architecture of DBMS
 To learn about Various Data Independence
 To Learn various Data Modeling Techniques

INTRODUCTION
A database is a collection of data elements (facts) stored in a computer in
such a systematic way that a computer program can consult it to answer
questions. The answers to those questions become information that can be used
to make decisions that may not be made with the data elements alone. The
computer program used to manage and query a database is known as a database
management system (DBMS).
A database is a collection of related data that we can use for
 Defining (specifying types of data)
 Constructing (storing & populating)
 Manipulating (querying, updating, reporting)
A Database Management System (DBMS) is a software package to
facilitate the creation and maintenance of a computerized database. ADatabase
System (DB) is a DBMS together with the data.
Features of a database
 It is a persistent (stored) collection of related data.
 The data is input (stored) only once.
 The data is organized (in some fashion).
 The data is accessible and can be queried (effectively and efficiently).
1 CS 6302
 Edited with the trial version of
Foxit Advanced PDF Editor
To remove this notice, visit:
www.foxitsoftware.com/shopping

CS DATABASE MANAGEMENT SYSTEMS

6302
NOTES FILE SYSTEMS VERSUS DATABASE SYSTEMS
DBMS are expensive to create in terms of software, hardware, and time
invested. Why should we use them? Why couldn’t we just keep all our data in
files, and use word-processors to edit the files appropriately to insert, delete, or
update data? And we could write our own programs to query the data! This
solution is called maintaining data in flat files. However, flat files have the
following limitations.
• Uncontrolled redundancy
• Inconsistent data
• Inflexibility
• Limited data sharing
• Poor enforcement of standards
• Low programmer productivity
• Excessive program maintenance
• Excessive data maintenance
Drawbacks of file systems :
Drawbacks of using file systems to store data are:
Data redundancy and inconsistency
Due to availability of multiple file formats, storage in files may
cause duplication of information in different files.
Difficulty in accessing data
In order to retrieve, access and use stored data, we need to write
a new program to carry out-each new task
Data isolation
To isolate data we need to store them in multiple files and
different Formats.
Integrity problems
Integrity constraints (e.g. account balance > 0) become part of
program code which has to be written everytime. It is hard to add
new constraints or to change existing ones.
Atomicity of updates
Failures of files may leave database in an inconsistent state with
partial updates carried out
E.g. transfer of funds from one account to another should happen

CS 6302 2
 Edited with the trial version of
Foxit Advanced PDF Editor
To remove this notice, visit:
www.foxitsoftware.com/shopping

CS DATABASE MANAGEMENT SYSTEMS

6302 eit
he
r
co
m
ple
tel
y
or
Pa
rti
all
y.

3 CS 6302
 Edited with the trial version of
Foxit Advanced PDF Editor
To remove this notice, visit:
www.foxitsoftware.com/shopping

CS DATABASE MANAGEMENT SYSTEMS

6302
Concurrent access by multiple users
NOTES
Concurrent access of files is needed for better performance and it
is also true that Uncontrolled concurrent accesses of files can
lead to inconsistencies
E.g. two people reading a balance and updating it at the same time
Several Security related problems caused in file system.
Database Management System Approach
The Data maintained in the form of DBMS has the following advantages
Controlled redundancy
o consistency of data & integrity constraints
Integration of data
o self-contained & represents semantics of application
Data and operation sharing
o multiple interfaces
Services & Controls
o security & privacy controls
o backup & recovery
o enforcement of standards
Flexibility
o data independence
o data accessibility
o reduced program maintenance
Ease of application and development
Limitations of Database Management System
Database Management System is
 more expensive
 more complex
 general
 simple
 stringent real-time
 single user
 static
NOTES When not to use a DBMS
As DBMS include the following costs like high initial cost, (possibly) cost
of extra hardware , cost of entering data , cost of training people to use DBMS
,cost of maintaining DBMS , it becomes tough to use DBMS.
DBMS becomes unnecessary
 if access to data by multiple users is not required
 If data set is small
 if database and application are simple, well-defined, and if they are
not expected to change
Levels of Abstraction
 Physical level : It is also known as internal schema which
describes data storage structures and access paths. It typically
uses a physical data model and the details are hidden from
database users. It describes how a record (e.g. customer) is stored.
 Logical level: It is also known as conceptual schema which
describes the structure and constraints for the whole database. It
uses a conceptual or implementation model. It describes data stored
in database, and the relationships among the data.
 type customer = record
name :
string; street
: string; city :
integer;
end;

 View level: It is also known as External schema, whichdescribes

various user views. Usually uses the same data model as
conceptual level. As shown in Fig 1.1 , Different views may have
different interpretations of the same data item. Application
programs hide details of data types. Views can also hide
information (e.g. salary) for security purposes.
 The Figure 1.1 in following section clearly depicts the levels.
Diagram for Levels of Data abstraction
NOTES

Fig.1.1 Three levels of Data Abstraction

Instances and Schemas
 Similar to types and variables in programming languages which we
already know.
 Schema is the logical structure of the database
o e.g. the database consists of information about a set of customers
and accounts and the relationship between them
o Analogous to type information of a variable in a program
o Physical schema: database design at the physical level
o Logical schema: database design at the logical level
 Instance is the actual content of the database at a particular point of time
o Analogous to the value of a variable
 Physical Data Independence – the ability to modify the physical schema
without changing the logical schema
o Applications depend on the logical schema
o In general, the interfaces between the various levels and
components should be well defined so that changes in some parts
do not seriously influence others.
Keywords:
Data, Database, Files, Redundancy, Consistency, Data Independence, Physical
Schema, Logical Schema, Abstraction.
NOTES Short Questions:
1. State the properties of Database.
2. State the approaches of Database system.
3. What are the levels of Abstraction in database?
4. When to prefer to use Database and when not to ?
5. What do you mean by instance and schema?
Descriptive Questions:
1. Compare the File System Model and DBMS.
2. Explain Database Systems Approach.
DATA MODELS
Model
o A structure that demonstrates all the required features of the parts
of the real world which is of interest to the users of the
information in the model.
o Representation and reflection of the real world (Universe of
Discourse)
1 Data Model
o A set of concepts that can be used to describe the structure of a
database: the data types, relationships, constraints, semantics and
operational behaviour.
o It is a tool for data abstraction
o A collection of tools for describing
 data
 data relationships
 data semantics
 data constraints
o Entity-Relationship model
o Relational model
o Other models:
 object-oriented model
 semi-structured data models
  Older models: network model and hierarchical model
NOTES
2 A model is described by the schema which is held in the data dictionary.
 Student(studno,name,address)
 Course(courseno,lecturer)  Schema
 Student(123,Bloggs,Woolton)  Instance
 (321,Jones,Owens)
Network and Hierarchical Data Models

Definition:

Data models are a collection of conceptual tools for describing data,

data relationships, data semantics and data constraints.

There are three different groups:

1. Object-based Logical Models.

2. Record-based Logical Models.

3. Physical Data Models.

Example: Consider the following database of a bank and its accounts.

Table 1. “Account” contains details of Table 2. “Customer” contains details

of the customer of a bank the bank account

Customer Account

Name Area City Account Number Balance

The Network Model

 Data are represented by collections of records.
 Relationships among data are represented by links.
 Organization is that of an arbitrary graph and represented by Network dia-
gram.
The Following figure shows a sample network database that is the equivalent of
the relational database of previous figure.
NOTES

Figure.1.2. A Sample Network Database

 The CODASYL/DBTG database was derived on this
model. Constraints in the Network Model
1. Insertion Constraints: Specifies what should happen when a record is
inserted
2. Retention Constraints: Specifies whether a record must exist on its own
or always be related to an owner as a member of some set instance.
3. Set Ordering Constraints: Specifies how a record is ordered inside the
data- base.
4. Set Selection Constraints: Specifies how a record can be selected from
the database.
1. The Hierarchical Model
 This model is Similar to the network model and the concepts are derived
from the Information Management System and System-200.
 Organization of the records is as a collection of trees, rather than
arbitrary graphs.
 Schema represented by a Hierarchical Diagram.
o One record type, called Root, does not participate as a child
record type.
o Every record type except the root participates as a child record
type in exactly one type.
o Leaf is a record that does not participate in any record types.
o A record can act as a Parent for any number of records.
 NOTES

Figure.1.3. A Sample Hierarchical Database

The relational model does not use pointers or links, but relates records by
the values they contain. This allows a formal mathematical foundation to be
defined. As shown in fig 1.4. different users have their own views of accessing the
same database. Different tables combine to form data to be converted to utilizable
format like printing.

Database Management System

Database

Fig 1.4. DBMS

NOTES Data Independence

Data independence means that any part of data is not dependent on any
other part of the data in the DBMS. They are no way dependent upon their
physical storage and logical arrangements. As shown in Fig. 1.5, they depend only
upon various exter- nal factors such as new hardware, new users, new technology,
new linkages and link- ages to other databases.
o Logical data independence
o change the conceptual schema without having to change the
external schemas
o Physical data independence
o change the internal schema without having to change the
conceptual schema

New
hardware New functions
Change in
New use
users
Database New data
User's
view

New storage Change in

techniques Linkage to other technology
databases

Fig 1.5. Data Independence

Key Terms:
Record, Link, Arbitrary Graph, Set Retention, Set Ordering, Set Selection, Tree,
Root, Leaf, Pointer.
Short Questions:
1. State the properties of a Network Model.
2. State the constraints in Network Model.
3. Draw the Network Diagram for any given database.
4. Draw the Hierarchical Diagram for any given database.
5. What do you mean by Root Record and Leaf Record?
 Descriptive Questions:
NOTES
1. Compare the Hierarchical Model and Network Model.
2. Design a database using Hierarchical Model.
3. Design a database using Network Model.

DBMS ARCHITECTURE
Overall System Structure

Fig 1.6 Overall System Structure

The DBMS architecture constitutes the Disk storage, Storage manager,

Query processor and Database users, connected through various tools and
applications like query and application tools and application programs and
application interfaces.
NOTES Database Users
Users are differentiated by the way they expect to interact with the system
o Application programmers: They interact with system through DML calls
o Sophisticated users: They form requests in a database query language
o Specialized users: They write specialized database applications that do not
fit into the traditional data processing framework
o Naïve users: They invoke one of the permanent application programs that
have been written previously
E.g. people accessing database over the web, bank tellers, clerical staff
Database Administrator
Database Administrator Coordinates all the activities of the database
system; he has a good understanding of the enterprise’s information
resources and needs.
o Database administrator’s duties include:
o Schema definition
o Storage structure and access method definition
o Schema and physical organization modification
o Granting user authority to access the database
o Specifying integrity constraints
o Acting as liasion with users
o Monitoring performance and responding to changes in requirements
Transaction Management within Storage Manager
o A transaction is a collection of operations that performs a single
logical function in a database application
o Transaction-management component ensures that the database
remains in a consistent (correct) state despite system failures (e.g.,
power failures and operating system crashes) and transaction
failures.
o Concurrency-control manager controls the interaction among the
concurrent transactions, to ensure the consistency of the database.
 o This process is performed by a system inside Storage manager.
NOTES
Storage Manager

o Storage manager is a program module that provides the interface

between the low-level data stored in the database and the
application programs and queries submitted to the system.

o It executes on the basis of data obtained from query execution

engine.

o It consists of Buffer manager, File manager, authentication and

integrity manager and transaction manager.

o The storage manager is responsible for the following tasks:

o Interaction within various managers.

o Efficient storing, retrieving and updating of data

Disk Storage

Disk storage consists of Data in the form logical tables, indices, data
dictionary and statistical data. Data Dictionary stores the data about data i.e. its
structure, etc. Indices are used for easy searching in a data base. Statistical data
is the log storage details about various transactions which occur on the database.

Query processor

As described in Fig.1.6, the query processor consists of the following

layers and the functionalities of each layer are described well with Fig. 1.7.

The users submit query which passes to optimizer where the query is
optimized, the physical execution plan goes to execution engine.

Query Execution engine passes the request to index/ file / record

manager. That in turn passes the request to buffer manager requesting it to
allocate memory to store pages. Buffer manager in turn sends the pages to
storage manager which takes care of physical storage.

The resulting data out of physical storage comes in reverse order. The
catalog is the Data Dictionary which contains statistics and schema. Every query
execution which takes place in execution engine is logged and recovered when
required.
NOTES API/GUI
Query
Optimizer
Stats
Physical
plan Logging, recovery
Exec. Engine
SchemasData/etc Requests
Catalog
Index/file/rec Mgr
Data/etc Requests
Buffer Mgr =
Pages Pages logical
-  physical
Storage Mgr
Data Requests
Storage
Fig 1.7 Query processing layers

Application Architectures

Fig 1.8 Architectures

A single user may use a simple accounting database on personal
computer to manage a small business. At other extreme, a very large company
may have databases at numerous locations around the counter that are linked
together. There are two generic database architectures namely centralized and
distributed. There are also some applications which run on servers with two tier
architecture and three tier architecture.
  Two-tier architecture: E.g. client programs using ODBC/JDBC
to communicate with a database
NOTES
 Three-tier architecture: E.g. web-based applications, and
applications built using “middleware”
DATA MODELING USING ENTITY-RELATIONSHIP DIAGRAMS
Definitions
Entity
An object that exists in the real world that has certain properties and
that is distinguishable from other objects. It may be an object, place, person,
concept or activity about which an enterprise records data. It is an object which
can have instances or occurrences. Each instance should be capable of being
uniquely identified. Each entity has certain properties or attributes associated with
it and operations applicable to it. It is represented by a rectangle in the ER model.
Example
o Physical Object - Employee, machine , book client , student , Item
o Abstract Object – Account , Department
o Event – Application, Reservation, Invoice, Contract
o Location – Building , City, State
Attribute
Attributes are data elements that describe an entity. They are the properties
of entities and relationships.
Example
o Employee: Employee No., Name, Title, Salary
o Project: Project No., Name, Budget
o Book : ISBN, Title, Author, Price
Relationship
Associations between two or more entities are represented by a diamond
in ER diagram.
Example
o Manage: Employee manage projects
o Work: Employee work in projects
NOTES Entity Types
Entity type is an abstraction which defines the properties (attributes) of a
simi- lar set of entities.
Example:
o Employee: Name, Title, Salary
o Project: Number, Budget, Location
Entity Instances
Entity instances are instantiations of types
Example:
o Employee: Joe, Jim
o Project: Compiler design, Accounting,
o An entity instance can have multiple entity types
Example:
If we want to have an EMPLOYEE entity type, then every engineer is an
employee.
Entity class
Entity class (or entity set) is a set of entity instances that are of the same type.
Similar arguments can be made for relationships.

Fig 1.9 Relationship

Attributes
o They describe the properties of entities and relationships.
o An instance of an attribute is a value, drawn from given domain, which
presents the set of possible values of the attribute.
Types
NOTES
o Single vs Multi-valued Attribute
 Single: Social insurance number
 Multi: Lecturers of a course
o Simple vs Composite Attribute
 Composite: Address consisting of Apt#, Street, City,
Zip
o Stored vs Derived Attribute
 Stored: Individual mark of a student
 Derived: Average mark in a class

ER Notations
NOTES

Example:

Fig 1.10 Employee – Project Example

Participation Constraints
They are used to check whether or not the existence of an entity depends on
its being related to another entity via the relationship type.
o Total: If entity Ei is in total participation with relation R, then every entity
in- stance of Ei has to participate via relation R to an entity instance of
another entity type Ej.
o Partial: Only some entity instances participate.

Strong entities
o The instances of the entity class can exist on
their own, without participating in any
relationship.
o It is also called non-obligatory membership.
 Weak entities Fig
o Entity does not have a primary key. 1.11
Enhanc
o Each instance of the entity class has to ed ER
participate in a relationship in order to model
exist.
o Keys are imported from dependent entity.
o It is also called obligatory membership.
o There is a special type of total participation.
Enhanced E-R (EER) Models
o An entity type E1 is a specialization of
another entity type E2 if E1 has the same
properties of E2 and perhaps even more.
o E1 IS-A E2
Inheritance allows one class to incorporate the attributes ad
behaviours one
or more classes.
Various sub classes are specializations of one or more super classes.
Specialization :
Object Oriented Applications are structured to perform work on generic
classes (e.g.Vehicle) and at runtime invoke behaviors appropriate for the specific
vehicle being executed upon (e.g. Boeing 747)
Aggregation :
Consider the ternary relationship works-on, which we saw earlier. Suppose
we want to record managers for tasks performed by an employee at a branch.
Then Fig
1.11 illustrates the EER situation of it.
NOTES
NOTES Generalization
 Abstracting the common properties of two or more entities to produce a
“higher- level” entity is called Generalization.

Fig. 1.12 Generalization

Subclass/Super class

 The subclass is specialization of the super class and it adds additional data or
behaviours or overrides behaviours of the super class.
 Super classes are generalization of their sub classes.
 This is related to instances of entities that are involved in a specialization
/generali- zation relationship
 If E1 specializes E2, then each instance of E1 is also an instance of E2.
 Therefore Class(E1)  Class(E2)
Example

Class (Manager)  Class (Employee)

Class (Employee) = Class (Engineer)  Class (Secretary)  Class
(Salesper- son)
Constraints
 Constraint on which entities can be members of a given lower-level entity set.
 condition-defined
 E.g. all customers above the age of 65 are members of senior-citizen
entity set; senior-citizen ISA person.
 user-defined
 Constraint on whether the entities may belong to more than one lower-level
NOTES
entity set within a single generalization.
 Disjoint
 An entity can belong to only one lower-level entity set.
 It is noted in E-R diagram by writing disjoint next to the ISA triangle.
 Overlapping
 An entity can belong to more than one lower-level entity set.
 Completeness constraint — It specifies whether or not an entity in the
higher-level entity set must belong to at least one of the lower-level entity
sets within a generalization.
 total : an entity must belong to one of the lower-level entity sets
 partial: an entity need not belong to one of the lower-level entity sets
Notations
d
•Disjoint, Total

•Disjoint, Partial d

O
•Overlapping, Total

•Overlapping, Partial O

Example:

Fig. 1.13 Example of Constraints

 NOTES Keywords:
Entity, Attribute, Relationship, Entity Type, Entity Instance, Entity Class,
Participation Constraints, Weak Entity, Strong Entity, Generalization, Specialization,
Subclass, Super Class.

Short Questions:
1. Define Entity, Attribute, Relationship, Entity Type, Entity Instance,
Entity Class.
2. Differentiate Weak and Strong Entity Set.
3. What are the various notations for ER Diagrams?
4. What is participation constraint? Mention different types of
participation constraint.
5. Define Generalization and Specialization.

Descriptive Questions:
1. Explain various types of Attributes with suitable examples.
2. State different types of Participation Constraint and explain with a
diagram- matic example.

EER MODEL DESIGN EXAMPLE

Scenario
• http://www.imdb.com wants to store information about movies
• Three steps:
– Requirements Analysis: Discover what information needs to be
stored, how the stored information will be used, etc. It is taught
in a course on system analysis and design
– Conceptual Database Design: High level description of data to be
stored (ER model)
– Logical Database Design: Translation of ER diagram to a
relational database schema (description of tables)
Example Requirements
• http://www.imdb.com wants to store information about films
• For actors and directors, we want to store their name, a unique
identification number, address and birthday (why not age?)
• For actors, we also want to store a photograph
 •For films, we want to store the title, year of production and type (thriller,
NOTES
comedy, etc.)
•We want to know who directed and who acted in each film. Every film has
one director. We store the salary of each actor for each film
•An actor can receive an award for his part in a film. We store information
about who got which award for which film, along with the name of the
award and year.
•We also store the name and telephone number of the organization who gave
the award. Two different organizations can give an award with the same
name. A single organization does not give more than one award with a
particular name per year.
Let us see a sample ER Model before discussing it in detail

address
id
birthday
Movie
phone Person name
name
number

Organization IS
A

Give picture Actor Director

Won
salary
Acted Directed
Award In

year Film
year nam
e

title type

Fig. 1.14 Sample ER Model

NOTES Entities, Entity Sets
• Entity : An object in the world that can be distinguished from other objects
– Examples of entities: mango , banana , guava
– Examples of things that are not entities: taste , red
• Entity set : A set of similar entities
– Examples of entity sets: fruit , flower , man
 Entity sets are drawn as rectangles
• Attributes : Used to describe entities
– All entities in the set have the same attributes.
– A minimal set of attributes that uniquely identify an entity is
called a key.
– An attribute contains a single piece of information (and not a
list of data).
• Examples of attributes: color , taste , size
•
Examples of things that cannot be attributes: shop , box
Attributes are drawn using ovals.
 The names of the attributes which make up a key are underlined.
Example

id birthday

Actor

name address

Another Option for a Key?

id birthday

Actor

name address
Representation of Composite and multi valued
attributes NOTES

 Composite attributes are flattened out by creating a separate attribute for

each component attribute.
E.g. given entity set customer with composite attribute name with
component attributes first-name and last-name the table corresponding to
the entity set has two attributes
name. First-name and name. last-name
 A multivalued attribute M of an entity E is represented by a separate table
EM. Table EM has attributes corresponding to the primary key of E and an
attribute corresponding to multi valued attribute M
E.g. Multi valued attribute dependent-names of employee is represented
by a table
employee-dependent-names( employee-id, dname)
 Each value of the multi valued attribute maps to a separate row of the table
EM.
E.g., an employee entity with primary key John and dependents Johnson
and Johndotir maps to two rows: (John, Johnson) and (John, Johndotir)
Another Option for a Key?

id birthday

Actor

name address

Relationships, Relationship Sets

•Relationship : Association among two or more entities
– Relationships may have attributes
• Relationship Set : Set of similar relationships
 Relationship sets are drawn using diamonds
NOTES Example title
birthday

id Actor Acted Film year

In
name

address
type

Fig. 1.15 Relationship Example

Where does the salary attribute belong?
Salary

Recursive Relationships
• An entity set can participate more than once in a relationship.
• In this case, we add a description of the role to the ER-diagram.

phone
number
manager

id
Employee Manage
name worker s

address

Fig. 1.16 Recursive Relationship

ry Relationship
NOTES
•An n-ary relationship R set involves exactly n entity sets: E1, …, En.
•Each relationship in R involves exactly n entities: e1 E1, …, en 
En
•Formally, R E1x …x En

name
id Director

id
Actor Film title
Produced
name

Fig. 1.17 n-ary relationship

Important Note
•The entities in a relationship set must identify the relationship.
•Attributes of the relationship set cannot be used for identification!
•Imagine that we want to store the role of an actor in a film.
How do we store information about a person who acted in one
film in several roles?

id Actor Film
Acted In title
name

Key Constraints
•Key constraints specify whether an entity can participate in one or more
than one relationships in a relationship set.
•When there is no key constraint an entity can participate any number of times.
•When there is a key constraint, the entity can participate one time at most.
 Key constraints are drawn using an arrow from the entity set to the relationship set.
•We express cardinality constraints by drawing either a directed line (),
signifying “one,” or an undirected line (—), signifying “many,” between
the relationship set and the entity set.
NOTES One-to-Many
 A film is directed by one director at most.
 A director can direct any number of films.

id
Directed Film title
Director
name

Director Directed Film

Many-to-Many
 A film is directed by any number of directors.
 A director can direct any number of films.

id Director Film
Directed title
name
 Edited with the trial version of
Foxit Advanced PDF Editor
To remove this notice, visit:
www.foxitsoftware.com/shopping

CS DATABASE MANAGEMENT SYSTEMS

6302
NOTES

Director Directed Film

One-to-One
 A film is directed by one director at most.
 A director can direct one film at most

id
Directed title
Director Film

name

Director Directed Film

 Edited with the trial version of
Foxit Advanced PDF Editor
To remove this notice, visit:
www.foxitsoftware.com/shopping

CS DATABASE MANAGEMENT SYSTEMS

6302
NOTES Another Example
Where would you put the arrow?

age
father

id
Person
FatherOf
child
name

Key Constraints in Ternary Relationships

Actor name
id

id
Director Film title
produced
name

What does this mean?

Participation Constraints
• Participation constraints specify whether or not an entity must participate
in a relationship set.
• When there is no participation constraint, it is possible that an entity will
not participate in a relationship set.
• When there is a participation constraint, the entity must participate at
least once.
 Participation constraints are drawn using a thick line from the entity set to the
relationship set.
Example (1)
• A film has at least one director.
• A director can direct any number of films.
 id Director Film
Directed title
name

Director Directed Film

Do you think that there should be a participation constraint from Director to Directed?

Example (2)
•We can combine key and participation constraints.
•What does this diagram mean?

id
Director Directed Film title

name

Keywords:
Entity, Entity Set, Attribute, Relation, Relationship set, Key Constraint, One to
one, one to many, Many to many, Recursive relations.
 NOTES Short Questions:
1. Define the following:
a. Entity
b. Entity Set
c. Attribute
d. Key
e. Relation
f. Relationship
2. What is n-ary Relationship?
3. Draw the diagram of Participation Constraint.

Descriptive Questions:
1. Explain various Relationship types with examples.
2. Compare the Recursive Relations with Participation Constraint with example.

RELATION SCHEMES AND INSTANCES

Relational scheme
 A relation scheme is the definition; i.e. a set of attributes
 A relational database scheme is a set of relation schemes: i.e. a set of sets
of attributes
Relation instance (simply relation)
 A relation is an instance of a relation scheme.
 A relation r over a relation scheme R = {A1, ..., An} is a subset of the
Cartesian product of the domains of all attributes, i.e.
r = D (A1), D (A2, … , D (An)
Domains
 A domain is a set of acceptable values for a variable.
 Example: Names of Canadians, Salaries of professors
Simple/Composite domains
Address = Street Name + Street Number + City + Province +Postal Code
Domain compatibility
 Binary operations (e.g. comparison to one another, addition, etc) can
be performed on them.
 Full support for domains is not provided in many current relational DBMS.
 Edited with the trial version of
Foxit Advanced PDF Editor
To remove this notice, visit:
www.foxitsoftware.com/shopping

CS DATABASE MANAGEMENT SYSTEMS

6302
Example of a Relation Scheme
NOTES
EMP (ENO, ENAME, TITLE, SAL)
PROJ (PNO, PNAME, BUDGET)
WORKS (ENO, PNO, RESP, DUR)
1 Underlined attributes are relation keys.
2 Tabular form

Example of a Relational Instance

Properties
 Based on the set theory
 No ordering among attributes and tuples
 No duplicate tuples allowed
 Value-oriented: tuples are identified by the attributes values
 All attribute values are atomic
 No repeating groups
Degree
It is the number of attributes that is available in a relation is called Degree of the
relation.
Cardinality
It is the number of tuples available in a relation.
 Cardinality constraints are specified in the form l..h, where l denotes the
minimum and h the maximum number of relationships an entity can
participate.
 Beware: the positioning of the constraints is exactly the reverse of
 Edited with the trial version of
Foxit Advanced PDF Editor
To remove this notice, visit:
www.foxitsoftware.com/shopping

CS DATABASE MANAGEMENT SYSTEMS

6302 the positioning of constraints in E-R diagrams.
NOTES  A Sample Relation – Mark Sheet
Rno. Name DOB Percentage
4001 Babu 10.01.1981 99
4003 Chitra 03.04.1984 89
4008 Rohini 11.11.1982 100

 The cardinality is 3
Integrity Constraints
# Key Constraints
Referring to Mark sheet relation as shown above
 Key: Aset of attributes that uniquely identifies tuples ( e.g. : Rno , Name )
 A super key of an entity set is a set of one or more attributes whose
values uniquely determine each entity.(e.g. : Rno , Name & DOB , Rno
& Name , Rno & Name & DOB)
 A candidate key of an entity set is a minimal super key (e.g. : Rno ,
Name & DOB)
 Although several candidate keys may exist, one of the candidate keys with
minimum number of attributes is selected to be the primary key. ( e.g.
:Rno )
# Data Constraints
~ Check constraints (e.g.: to check percentage
>80) # Others
~ Entity Integrity Constraint: No primary key value can be null. (e.g.:
Rno cannot be null)
~ Referential Integrity Constraint: A tuple in one relation must refer to
an existing tuple.
Characteristics of Relations
 Ordering of Tuples in a Relation
 Ordering of values within a tuple and alternative definition of a relation
 Values in tuples: Each value in a tuple is an atomic value.
Advantages over other models
 Simple concepts
 Solid mathematical foundation
 Set theory
 Powerful query languages
  Efficient query optimization strategies NO TES
 Design theory
 Industry standard SQL language

Keywords:
Relation, Relational Scheme, Relational Instance, Domain, Degree, Cardinality,
Integrity Constraints, Candidate Key, Primary Key.

Short Questions:
1. Define the following:
a. Relation
b. Relational Scheme
c. Relational Instance
d. Domain
e. Degree
f. Cardinality
g. Integrity Constraints
h. Candidate Key
i. Primary Key
2. What is a Domain Compatibility?
3. State the properties of a Relational Model.

Descriptive Questions:
1. Design a database in Relational Model and represent it in Relational
Schema.
2. Explain various constraints in Relational Model.
Summary
 DBMS is used to maintain and query large datasets.
 Benefits include recovery from system crashes, concurrent access, quick
application development, data integrity and security.
 Levels of abstraction give data independence.
 A DBMS typically has a layered architecture.
 DBAs hold responsible jobs and are well-paid!
 DBMS R&D is one of the broadest, most exciting areas in CS.
 Conceptual design follows requirements analysis, Yields a high-level
description of data to be stored
NOTES  ER model popular for conceptual design Constructs are expressive, close
to the way people think about their applications.
 Basic constructs: entities, relationships, and attributes (of entities and
relationships).
 Some additional constructs: weak entities, ISA hierarchies, and aggregation.
 Note: There are many variations on ER model.
 Several kinds of integrity constraints can be expressed in the ER model:
key constraints, participation constraints, and overlap/covering
constraints for ISA Hierarchies. Some foreign key constraints are also
implicit in the definition of a relationship set.
 Some constraints (notably, functional dependencies) cannot be expressed in
the ER model.
 Constraints play an important role in determining the best database design
for an enterprise.
 ER design is subjective. There are often many ways to model a given
scenario!
 Analyzing alternatives can be tricky, especially for a large enterprise.
 Common choices include: Entity vs. attribute, entity vs. relationship, binary
or nary relationship, whether or not to use ISA hierarchies, and whether or
not to use aggregation.
 Ensuring good database design: resulting relational schema should be
analyzed and refined further. FD information and normalization techniques
are especially useful.
References :
1. Ramez Elamasri and shankant B.Navathe, “ Fundamentals of Database
Systems”, Third Edition , Pearson Education, Delhi, 2002.
2. Abraham Silberschatz, Henry F.Korth and S.Sundarshan, “Database
System concepts”, Fourth Edition, McGrawHill, 2002.
3. C.J.Date,”An Introduction to Database Systems”, Seventh Edition,
Pearson Education, Delhi, 2002.
 NOTES

STORAGE STRUCTURES
Learning Objectives
 To Learn about Secondary Storage Devices
 To learn about RAID Technology
 To Study about File Operations
 To learn about Hashing Techniques
 To Learn about Indexing ( Both Single and Multi level )
 To study B+ Tree and Indexes on Multiple keys

INTRODUCTION
Storage Structure
 A database file is partitioned into fixed-length storage units called
blocks. Blocks are units of both storage allocation and data transfer.
 Database system seeks to minimize the number of block
transfers between the disk and memory. We can reduce the
number of disk accesses by keeping as many blocks as possible
in main memory.
 Secondary storage devices are used to store data permanently.
 Buffer – portion of main memory available to store copies of
disk blocks.
 Buffer manager – subsystem responsible for allocating buffer space
in main memory.
 Volatile storage:
o does not survive system crashes
examples: main memory, cache
memory
 Nonvolatile storage:
o survives system crashes
examples: disk, tape, flash memory,
non-volatile (battery backed up) RAM
NOTES  Stable storage:
o a mythical form of storage that survives all failures
o approximated by maintaining multiple copies on
distinct nonvolatile media
2.1.1.1. Physical Storage Media
Physical storage media are the devices on which data are stored. It is
classified into different categories. It is based on the following aspects.
H Speed with which data can be accessed
H Cost per unit of data
H Reliability
4 Data loss on power failure or system crash

4 Physical failure of the storage device

H Life of storage
4 Volatile storage: It loses contents when power is switched off

4 Non-volatile storage:

4 It’s contents persist even when power is switched off.

4 It includes secondary and tertiary storage, as well as batter-backed up

main- memory.
2.1.1.2 Categories of Physical Storage Media
Cache
4 The fastest and most costly form of storage
4 Volatile
4 Managed by the computer system hardware.

Main memory
4 Fast access (10s to 100s of nanoseconds; 1 nanosecond = 10–9 seconds)
4 Generally too small (or too expensive) to store the entire database
4 Capacities of a few Gigabytes are widely used currently.
4 Capacities have gone up and per-byte costs have decreased steadily and
rapidly.
4 Volatile: Contents of main memory are usually lost if a power failure or
system crash occurs.

Flash memory
4 Data survives power failure.
4 Data can be written at a location only once, but location can be erased
and written again.
 It can support only a limited number of write/erase cycles.
4
NOTES
4 Erasure of memory has to be done to an entire bank of memory.
4 Reads are roughly as fast as main memory.
4 But writes are slow (few microseconds), erasure is slower.
4 Cost per unit of storage is roughly similar to main memory.
4 It is widely used in embedded devices like digital cameras.
4 It is also known as EEPROM.

Magnetic-disk
4 Data is stored on spinning disk, and it is read/written magnetically.
4 Primary medium for the long-term storage of data; typically stores entire
data base.
4 Data must be moved from disk to main memory for access, and written
back for storage.
4 Much slower access than main memory (more on this later)

4 Direct-access: It is possible to read data on disk in any order, unlike

magnetic tape
4 Hard disks vs. floppy disks
4 Capacities range upto roughly 100 GB currently.
4 Much larger capacity and cost/byte than main memory/flash memory

4 Growing constantly and rapidly with technology improvements

4 Survives power failures and system crashes

4 Disk failure can destroy data, but is very rare

Optical storage
4 Non-volatile, data is read optically from a spinning disk using a laser
4 CD-ROM (640 MB) and DVD (4.7 to 17 GB) most popular forms
4 Write-one, read-many (WORM) optical disks used for archival storage
(CD-R and DVD-R)
4 Multiple write versions also available (CD-RW, DVD-RW, and DVD-RAM)
4 Reads and writes are slower than with magnetic disk

4 Juke-box systems, with large numbers of removable disks, a few drives, and a

mechanism for automatic loading/unloading of disks available for storing large

volumes of data
Optical Disks
Compact disk-read only memory (CD-ROM)
NOTES 4 Disks can be loaded into or removed from a drive.
4 High storage capacity (640 MB per disk)
4 High seek times or about 100 msec (optical read head is heavier and
slower)
4 Higher latency (3000 RPM) and lower data-transfer rates (3-6 MB/s)
com-
pared to magnetic disks

Digital Video Disk (DVD)

4 DVD-5 holds 4.7 GB and DVD-9 holds 8.5 GB.
4 DVD-10 and DVD-18 are double sided formats with capacities of 9.4 GB
and 17 GB.
4 Other characteristics similar to CD-ROM.
4 Record once versions (CD-R and DVD-R).
4 Data can only be written once, and cannot be erased.
4 High capacity and long lifetime; used for archival storage.
4 Multi-write versions (CD-RW, DVD-RW and DVD-RAM) also available.

Tape storage
4 Non-volatile, used primarily for backup (to recover from disk failure), and
for archival data
4 Sequential-access – much slower than disk
4 Very high capacity (40 to 300 GB tapes available)
4 Tape can be removed from drive  storage costs much cheaper than disk,
but drives are expensive
4 Tape jukeboxes available for storing massive amounts of data
4 Hundreds of terabytes (1 terabyte = 109 bytes) to even a petabyte (1
petabyte
= 1012 bytes)

Magnetic Tapes
n Hold large volumes of data and provide high transfer rates

H Few GB for DAT (Digital Audio Tape) format, 10-40 GB with DLT

(Digital Linear Tape) format, 100 GB+ with Ultrium format, and 330 GB
with Ampex helical scan format
H Transfer rates from few to 10s of MB/s
n Currently the cheapest storage medium
H Tapes are cheap, but cost of drives is very high.
n Very slow access time in comparison with magnetic disks and optical disks
 Limited to sequential access.
H
NOTES
n Used mainly for backup, for storage of infrequently used information, and as an
off- line medium for transferring information from one system to another.
n Tape jukeboxes used for very large capacity storage
H (Terabyte (1012 bytes) to Petabye (1015 bytes)

Storage Hierarchy
Primary storage: Fastest media but volatile (cache, main memory).
Secondary storage: next level in hierarchy, non-volatile, moderately fast access
time.
Also called on-line storage
4 E.g. flash memory, magnetic disks
Tertiary storage: lowest level in hierarchy, non-volatile, slow access time
4 Also called off-line storage
4 E.g. magnetic tape, optical storage

Magnetic Hard Disk Mechanism

Read-write head
H Positioned very close to the platter surface (almost touching it)

H Reads or writes magnetically encoded information

n Surface of platter divided into circular tracks

H Over 17,000 tracks per platter on typical hard disks

n Each track is divided into sectors.

H A sector is the smallest unit of data that can be read or written.

H Sector size typically 512 bytes

H Typical sectors per track: 200 (on inner tracks) to 400 (on outer tracks)

n To read/write a sector

H Disk arm swings to position head on right track

H Platter spins continually; data is read/written as sector passes under head

n Head-disk assemblies

H Multiple disk platters on a single spindle (typically 2 to 4)

H One head per platter, mounted on a common arm.

th
n Cylinder i consists of i track of all the platters

Performance Measures of Disks

n Access Time – the time it takes from when a read or write request is issued to

when data transfer begins. It consists of the following.

NOTES H Seek Time – time it takes to reposition the arm over the correct track.
4 Average Seek Time is 1/2 the worst case seek time.

– Would be 1/3 if all tracks had the same number of sectors, and we

ignore the time to start and stop arm movement

4 4 to 10 milliseconds on typical disks

H Rotational latency – time it takes for the sector to be accessed to appear

under the head.
4 Average latency is 1/2 of the worst-case latency.

4 4 to 11 milliseconds on typical disks (5400 to 15000 r.p.m.)

n Data-Transfer Rate – the rate at which data can be retrieved from or stored to

the disk.
H 4 to 8 MB per second is typical
4 Multiple disks may share a controller, so rate that controller can handle is

also important
Mean Time To Failure (MTTF) – the average time the disk is expected to run
continuously without any failure.
4 Typically 3 to 5 years
4 Probability of failure of new disks is quite low, corresponding to a
“theoretical MTTF” of 30,000 to 1,200,000 hours for a new disk.

RAID
RAID: Redundant Arrays of Independent Disks
o Disk organization techniques that manage a large numbers of
disks, provide a view of a single disk of
 high capacity and high speed by using multiple disks
in parallel, and
 high reliability by storing data redundantly, so that data
can be recovered even if a disk fails
 The chance that some disk out of a set of N disks will fail is much higher
than the chance that a specific single disk will fail.
 E.g. a system with 100 disks, each with MTTF of
100,000 hours (approx. 11 years), will have a system
MTTF of 1000 hours (approx. 41 days)
o Techniques for using redundancy to avoid data loss are critical
with large numbers of disks
 Originally a cost-effective alternative to large, expensive disks
NOTES
o I in RAID originally stood for “inexpensive’’
o Today RAIDs are used for their higher reliability and bandwidth.
 The “I” is interpreted as independent
 Improvement of Reliability via Redundancy
 Redundancy – stores extra information that can be used to
rebuild information lost in a disk failure
 E.g. Mirroring (or shadowing)
o Duplicate every disk. Logical disk consists of two
physical disks.
o Every write is carried out on both disks
 Reads can take place from either disk
o If one disk in a pair fails, data still available in the other
 Data loss would occur only if a disk fails, and its
mirror disk also fails before the system is repaired
 Probability of combined event is very small
o Except for dependent failure modes
such as fire or building collapse or
electrical power surges
 Mean time to data loss depends on mean time to
failure, and mean time to repair
o E.g. MTTF of 100,000 hours, mean time to repair of 10
hours gives mean time to data loss of 500*106 hours (or
57,000 years) for a mirrored pair of disks (ignoring
dependent failure modes)
 Improvement in Performance via Parallelism
o Two main goals of parallelism in a disk system:
1. Load balance multiple small accesses to increase throughput
2. Parallelize large accesses to reduce response time.
o Improve transfer rate by striping data across multiple disks.
o Bit-level striping – split the bits of each byte across multiple disks
 In an array of eight disks, write bit i of each byte to disk i.
 Each access can read data at eight times the rate of a single disk.
 But seek/access time worse than for a single disk
 Bit level striping is not used much any more
o Block-level striping – with n disks, block i of a file goes to disk (i
mod n)
+1
NOTES  Requests for different blocks can run in parallel if the blocks
reside on different disks
 A request for a long sequence of blocks can utilize all disks
inparallel
RAID Levels
o Schemes to provide redundancy at lower cost by using disk
striping combined with parity bits
 Different RAID organizations, or RAID levels, have
differing cost, performance and reliability characteristics
 RAID Level 0: Block striping; non-redundant.
o Used in high-performance applications where loss of data is
not critical.
 RAID Level 1: Mirrored disks with block striping
o Offers best write performance.
o Popular for applications such as storing log files in a
database system.
 RAID Level 2: Memory-Style Error-Correcting-Codes (ECC) with
bit striping.
 RAID Level 3: Bit-Interleaved Parity
o a single parity bit is enough for error correction, not just
detection, since we know which disk has failed
 When writing data, corresponding parity bits must also
be computed and written to a parity bit disk.
To recover data in a damaged disk, compute XOR of
bits from other disks (including parity bit disk).
o Faster data transfer than with a single disk, but fewer I/Os
per second since every disk has to participate in every I/O.
o Subsumes Level 2 (provides all its benefits at lower cost).
 RAID Level 4: Block-Interleaved Parity; uses block-level striping, and
keeps a parity block on a separate disk for corresponding blocks
from N other disks.
o When writing data block, corresponding block of parity bits
must also be computed and written to parity disk.
o To find the value of a damaged block, compute XOR of bits
from corresponding blocks (including parity block) of other
disks.
o P han Level 3
r
o
v
i
d
e
s
h
i
g
h
e
r
I
/
O

r
a
t
e
s
f
o
r
i
n
d
e
p
e
n
d
e
n
t
b
l
o
c
k
r
e
a
d
s
t
  Block read goes to a single disk, so blocks stored on
NOTES
different disks can be read in parallel.
o Provides high transfer rates for reads of multiple blocks than no-
strip- ing
o Before writing a block, parity data must be computed.
 Can be done by using old parity block, old value of
current block and new value of current block (2 block
reads + 2 block writes)
 Or by recomputing the parity value using the new values
of blocks corresponding to the parity block
n More efficient for writing large amounts of
data sequentially
o Parity block becomes a bottleneck for independent block writes
since every block write also writes to parity disk.
 RAID Level 5: Block-Interleaved Distributed Parity; partitions data and
parity among all N + 1 disks, rather than storing data in N disks and
parity in 1 disk.
o E.g. with 5 disks, parity block for nth set of blocks is stored on
disk (n mod 5) + 1, with the data blocks stored on the other 4
disks.
o Higher I/O rates than Level 4.
 Block writes occur in parallel if the blocks and their parity
blocks are on different disks.
o Subsumes Level 4: provides same benefits, but avoids bottleneck
of parity disk.
 RAID Level 6: P+Q Redundancy scheme; similar to Level 5, but stores
extra redundant information to guard against multiple disk failures.
o Better reliability than Level 5 at a higher cost; It is not used
widely.
NOTES

Fig 2.1 RAID Levels

Choice of RAID Level
o Factors in choosing RAID level
 Monetary cost
 Performance: Number of I/O operations per second, and
bandwidth during normal operation
 Performance during failure
 Performance during rebuild of failed disk
 Including time taken to rebuild failed disk
o RAID 0 is used only when data safety is not important
  E.g. data can be recovered quickly from other sources
NOTES
o Level 2 and 4 are never used since they are subsumed by 3 and 5.
o Level 3 is not used anymore since bit-striping forces single block reads
to access all disks, wasting disk arm movement, which block striping
(level 5) avoids.
o Level 6 is rarely used since levels 1 and 5 offer adequate safety for
almost all applications.
o So competition is between 1 and 5 only.
 Level 1 provides much better write performance than level 5
o Level 5 requires at least 2 block reads and 2 block writes to write a
single block, whereas Level 1 only requires 2 block writes.
o Level 1 is preferred for high update environments such as log disks
 Level 1 has higher storage cost than level 5
o Disk drive capacities increase rapidly (50%/year) whereas disk access
times decrease much less (x 3 in 10 years)
o I/O requirements have increased greatly, e.g. for Web servers
o When enough disks are bought to satisfy required rate of I/O, they often
have spare storage capacity.
 so there is often no extra monetary cost for Level 1!
 Level 5 is preferred for applications with low update rate, and large amounts
of data.
 Level 1 is preferred for all other applications.
Hardware Issues
 Software RAID: RAID implementations are done entirely in software, with
no special hardware support
 Hardware RAID: RAID implementations with special hardware
o Use non-volatile RAM to record writes that are being executed
o Beware: power failure during write can result in corrupted disk
 E.g. failure after writing one block but before writing the second
in a mirrored system
 Such corrupted data must be detected when power is restored
 Recovery from corruption is similar to recovery from
failed disk
 NV-RAM helps to efficiently detect potentially corrupted
blocks
NOTES o Otherwise all blocks of disk must be read and
compared with mirror/parity block
 Hot swapping: replacement of disk while system is running, without power down
o Supported by some hardware RAID systems,
o reduces time to recovery, and improves availability greatly
 Many systems maintain spare disks which are kept online, and used as
replace- ments for failed disks immediately on detection of failure
o Reduces time to recovery greatly
 Many hardware RAID systems ensure that a single point of failure will not stop
the functioning of the system by using
o Redundant power supplies with battery backup
o Multiple controllers and multiple interconnections to guard against
controller/ interconnection failures
FILE OPERATIONS
 The database is stored as a collection of files. Each file is a sequence of
records. A record is a sequence of fields.
 One approach:
o assume that record size is fixed
o each file has records of one particular type only.
o different files are used for different relations.
 This case is easiest to implement; we will consider variable length records
later.
Fixed-Length Records
 Simple approach:
o Store record i starting from byte n  (i – 1), where n is the size of
each record.
o Record access is simple but records may cross blocks
 Modification: do not allow records to cross block boundaries
 Deletion of record I: alternatives:
o move records i + 1, . . ., n to i, . . . , n – 1
o move record n to i
o do not move records, but link all free records on a free list
 NOTES

Fig 2.2 Fixed length records

 Free Lists
o Store the address of the first deleted record in the file header.
o Use this first record to store the address of the second
deleted record, and so on
o Can think of these stored addresses as pointers since they “point”
to the location of a record.
o More space efficient representation: reuse space for normal at-
tributes of free records to store pointers. (No pointers stored in
in- use records.)
 Fig 2.3 Free list
Variable-Length Records
o Variable-length records arise in database systems in several ways:
 Storage of multiple record types in a file.
NOTES  Record types that allow variable lengths for one or more
fields.
 Record types that allow repeating fields (used in some
older data models).
o Byte string representation
 Attach an end-of-record () control character to the end
of each record
 Difficulty with deletion
 Difficulty with growth
 Variable-Length Records: Slotted Page Structure

Fig 2.4 Slotted page structure

 Slotted page header contains:
o number of record entries
o end of free space in the block
o location and size of each record
 Records can be moved around within a page to keep them contiguous
with no empty space between them; entry in the header must be updated.
 Pointers should not point directly to record — instead they should point to
the entry for the record in header.
 Fixed-length representation:
o reserved space
o pointers
Reserved space – can use fixed-length records of a known maximum length;
unused space in shorter records filled with a null or end-of-record symbol.
Pointer Method
NOTES

 Pointer method
 A variable-length record is represented by a list of fixed-length
records, chained together via pointers.
 It can be used even if the maximum record length is not known.
 Disadvantage to pointer structure; space is wasted in all records except the
first in a a chain.
 Solution is to allow two kinds of block in file:
 Anchor block – contains the first records of chain
 Overflow block – contains records other than those that are the first
records of chairs.

Organization of Records in Files

 Heap – a record can be placed anywhere in the file where there is
space
 Sequential – store records in sequential order, based on the value of
the search key of each record
 Hashing – a hash function computed on some attribute of each
record;
NOTES the result specifies in which block of the file the record should be placed
 Records of each relation may be stored in a separate file. In a
clustering file organization, records of several different relations can be
stored in the same file
 Motivation: store related records on the same block to minimize I/O

Sequential File Organization

 Suitable for applications that require sequential processing of the
entire file
 The records in the file are ordered by a search-key.
 Deletion – use pointer chains
 Insertion –locate the position where the record is to be inserted
 if there is free space insert there.
 if no free space, insert the record in an overflow block.
 In either case, pointer chain must be updated
 Need to reorganize the file from time to time to restore sequential order

Clustering File Organization

o Simple file structure stores each relation in a separate file
o It can instead store several relations in one file using a clustering
file organization
o E.g. clustering organization of customer and depositor:
 Edited with the trial version of
Foxit Advanced PDF Editor
To remove this notice, visit:
www.foxitsoftware.com/shopping

CS DATABASE MANAGEMENT SYSTEMS

6302

o good for queries involving depositor customer, and for

queries involving one single customer and his accounts
o bad for queries involving only customer
o results in variable size records
Mapping of Objects to Files
 Mapping objects to files is similar to mapping tuples to files in a
relational system; object data can be stored using file structures.
 Objects in O-O databases may lack uniformity and may be very
large; such objects have to be managed differently from records in a
relational system.
o Set fields with a small number of elements may be
implemented using data structures such as linked lists.
o Set fields with a larger number of elements may be
implemented as separate relations in the database.
o Set fields can also be eliminated at the storage level by
normali- zation.
 Similar to conversion of multivalued attributes of
E-R diagrams to relations
 Objects are identified by an object identifier (OID); the storage system
needs a mechanism to locate an object given its OID (this action is
called dereferencing).
o logical identifiers do not directly specify an object’s physical
location; they must maintain an index that maps an OID to
the object’s actual location.
o physical identifiers encode the location of the object and so
the object can be found directly. Physical OIDs typically
have the following parts:
1. a volume or file identifier
 Edited with the trial version of
Foxit Advanced PDF Editor
To remove this notice, visit:
www.foxitsoftware.com/shopping

CS DATABASE MANAGEMENT SYSTEMS

6302
NOTES 2. a page identifier within the volume or file NOTES
3. an offset within the page
HASHING
It is a type of file organization in which the address of each record is deter-
mined using the hashing algorithm. Ahashing algorithm is a routine that
converts a primary key value into a relative record number. As a result of
this the records are located non-sequentially in the file.
o Example : The hash function returns the sum of the individual
digits as single digit
o E.g. h(123) = 6 h(792) = h(18)=9 h(111) = 3
Hashing is a hash function computed on some attribute of each record; the
result specifies in which block of the file the record should be placed.
 Static and Dynamic Hashing are the types of Hashing.
Static Hashing
 A bucket is a unit of storage containing one or more records (a
bucket is typically a disk block).
 In a hash file organization we obtain the bucket of a record directly
from its search-key value using a hash function.
 Hash function h is a function from the set of all search-key values
K to the set of all bucket addresses B.
 Hash function is used to locate records for access, insertion as
well as deletion.
 Records with different search-key values may be mapped to the
same bucket; thus entire bucket has to be searched sequentially to
locate a record.
Example of Hash File Organization
 Hash file organization of account file, using branch-name as key
o There are 10 buckets,
o The binary representation of the ith character is assumed to be
the integer i.
o The hash function returns the sum of the binary representations of
the characters modulo 10
o E.g. h(Perryridge) = 5 h(Round Hill) = 3 h(Brighton) = 3
o Hash file organization of account file, using branch-name as key
 Fig 2.5 Hashing

Hash Functions

o Worst Hash function maps all search-key values to the same bucket;
this makes access time proportional to the number of search-key
values in the file.
o An ideal hash function is uniform, i.e. each bucket is assigned the
same number of search-key values from the set of all possible values.
o Ideal hash function is random, so each bucket will have the same
number of records assigned to it irrespective of the actual
distribution of search- key values in the file.
o Typical hash functions perform computation on the internal binary
repre- sentation of the search-key.
o For example, for a string search-key, the binary
representations of all the characters in the string could be
added and the sum modulo the number of buckets could be
returned. .
Handling of Bucket Overflows
o Bucket overflow can occur because of
 Insufficient buckets
NOTES  Skew in distribution of records. This can occur due to
two reasons:
 multiple records have same search-key value.
 chosen hash function produces non-uniform
distribution of key values.
o Although the probability of bucket overflow can be reduced, it cannot
be eliminated; it is handled by using overflow buckets.
o Overflow chaining – the overflow buckets of a given bucket are
chained together in a linked list.
o Above scheme is called closed hashing.
o An alternative, called open hashing, which does not use overflow
buckets, is not suitable for database applications.

Fig 2.6 Bucket hashing

Hash Indices
o Hashing can be used not only for file organization, but also for
index- structure creation.
o A hash index organizes the search keys, with their associated
record pointers, into a hash file structure.
o Strictly speaking, hash indices are always secondary indices.
o If the file itself is organized using hashing, a separate primary hash
index on it using the same search-key is unnecessary.
o However, we use the term hash index to refer to both secondary
index structures and hash organized files.
 Example of Hash Index
 Fig 2.7 Example of Hash Index
Deficiencies of Static Hashing
o In static hashing, function h maps search-key values to a fixed set of
B of bucket addresses.
o Databases grow with time. If initial number of buckets is too small,
per- formance will degrade due to too much overflows.
o If file size at some point in the future is anticipated and number of
buckets allocated accordingly, significant amount of space will be
wasted initially.
o If database shrinks, space will be wasted again.
o One option is periodic re-organization of the file with a new hash
function, but it is very expensive.
o These problems can be avoided by using techniques that allow the
number of buckets to be modified dynamically.
Dynamic Hashing
 Good for database that grows and shrinks in size
 Allows the hash function to be modified dynamically
Extendable hashing – one form of dynamic hashing
 Hash function generates values over a large range — typically
b- bit integers, with b = 32.
 At any time use only a prefix of the hash function to index
into a table of bucket addresses.
 Let the length of the prefix be i bits, 0  i  32.
 Bucket address table size = 2i. Initially i = 0
 Value of i grows and shrinks as the size of the database grows
and shrinks.
NOTES  Multiple entries in the bucket address table may point to a
bucket.
 Thus, actual number of buckets is < 2i
 The number of buckets also changes dynamically due
to coalescing and splitting of buckets.
Use of Extendable Hash Structure
o Each bucket j stores a value ij; all the entries that point to
the same bucket have the same values on the first ij bits.
o To locate the bucket containing search-key Kj:
o 1. Compute h(Kj) = X
o 2. Use the first i high order bits of X as a displacement
into bucket address table, and follow the pointer to
appropriate bucket
o To insert a record with search-key value Kj,
follow same procedure as look-up and locate the bucket, say j.
o If there is room in the bucket j, insert record in the bucket.
o Otherwise the bucket must be split and insertion re-attempted.
 Overflow buckets used instead in some cases.
Example of Extendable Hash Structure
In this structure, i2 = i3 = i, whereas i1 = i – 1

Fig 2.8 Extendable Hash Structure

Updates in Extendable Hash Structure
o To split a bucket j when inserting record with search-key value
Kj:
 Edited with the trial version of
Foxit Advanced PDF Editor
To remove this notice, visit:
www.foxitsoftware.com/shopping

CS DATABASE MANAGEMENT SYSTEMS

6302
o If i > ij (more than one pointer to bucket j)
NOTES
o Allocate a new bucket z, and set ij and iz to the old ij -+ 1.
o Make the second half of the bucket address table entries
pointing to j to point to z
o Remove and reinsert each record in bucket j.
o Recompute new bucket for Kj and insert record in the bucket
(further splitting is required if the bucket is still full)
o If i = ij (only one pointer to bucket j)
o Increment i and double the size of the bucket address table.
o Replace each entry in the table by two entries that point to
the same bucket.
o Recompute new bucket address table entry for Kj.
Now i > ij so use the first case above.
o When inserting a value, if the bucket is full after several splits
(that is, i reaches some limit b) create an overflow bucket
instead of splitting bucket entry table further.
o To delete a key value,
locate it in its bucket and remove it.
o The bucket itself can be removed if it becomes empty (with
appro- priate updates to the bucket address table).
o Coalescing of buckets can be done (can coalesce only with a
“buddy” bucket having same value of ij and same ij –1 prefix,
if it is present)
o Decreasing bucket address table size is also possible.
o Note: decreasing bucket address table size is an expensive
opera- tion and should be done only if number of buckets
becomes much smaller than the size of the table
 Use of Extendable Hash Structure: Example
NOTES

Initial Hash structure, bucket size = 2

 Hash structure after insertion of one Brighton and two Downtown

records

 Hash structure after insertion of Mianus record

 NOTES

Fig 2.9 Hash structure after insertion of three Perryridge

records

Fig 2.10 Hash structure after insertion of Redwood and Round Hill records

Extendable Hashing vs. Other Schemes

 Benefits of extendable hashing:
o Hash performance does not degrade with growth of file
NOTES o Minimal space overhead
 Disadvantages of extendable hashing
o Extra level of indirection to find desired record
o Bucket address table itself may become very big (larger than
memory).
 Need a tree structure to locate desired record in
the structure!
o Changing size of bucket address table is an expensive operation.
 Linear hashing is an alternative mechanism which avoids these
disadvan- tages at the possible cost of more bucket overflows
INDEXING
Needs of Indexing
 Indexing mechanisms used to speed up access to desired data.
o E.g., author catalog in library
 Search Key - attribute to set of attributes used to look up
records in a file.
 An index file consists of records (called index entries) of this form
 Index files are typically much smaller than the original file
Two basic kinds of indices:
o Ordered indices: search keys are stored in sorted order
o Hash indices: search keys are distributed uniformly
across “buckets” using a “hash function”.
 Index Evaluation Metrics
o Access types supported efficiently.
o Records with a specified value in the attribute
o Records with an attribute value falling in a specified range
of values.
o Access time
o Insertion time
o Deletion time
o Space overhead
 Ordered Indices
o Indexing techniques evaluated on basis of:
 In an ordered index, index entries are stored and
sorted on the search key value. E.g. author catalog
in library.
 Primary index: in a sequentially ordered file, the
index whose search key specifies the sequential
order of the file.
  Also called clustering index
NOTES
 The search key of a primary index is usually but
not necessarily the primary key.
 Secondary index: an index whose search key
specifies an order different from the sequential
order of the file. Also called non-clustering index.
 Index-sequential file: ordered sequential file with a
pri- mary index.
 Dense Index Files
o Dense index — Index record appears for every search-
key value in the file.

Fig 2.11 Dense index

Sparse Index Files
 Sparse Index: contains index records for only some search-key values.
 Applicable when records are sequentially ordered on search-key
 To locate a record with search-key value K: we
find index record with largest search-key value <
K
 Search file sequentially starting at the record to which the index
record point
 Less space and less maintenance overhead for insertions and deletions.
 Generally slower than dense index for locating records.
 Good tradeoff: sparse index with an index entry for every block in
file, corresponding to least search-key value in the block.
 Edited with the trial version of
Foxit Advanced PDF Editor
To remove this notice, visit:
www.foxitsoftware.com/shopping

CS DATABASE MANAGEMENT SYSTEMS

6302
NOTES

Fig 2.12 Example of Sparse Index Files

Multilevel Index
 If primary index does not fit in memory, access becomes expensive.
 To reduce number of disk accesses to index records, treat
primary index kept on disk as a sequential file and construct a
sparse index on it.
o outer index – a sparse index of primary index
o inner index – the primary index file
 If even outer index is too large to fit in main memory, yet another
level of index can be created, and so on.
 Indices at all levels must be updated on insertion or deletion from
the
file.

 Index Update: Deletion

 If deleted record is the only record in the file with its
particular search-key value, the search-key is deleted from
the index also.
 Edited with the trial version of
Foxit Advanced PDF Editor
To remove this notice, visit:
www.foxitsoftware.com/shopping

CS DATABASE MANAGEMENT SYSTEMS

6302
 Single-level index deletion:
 Edited with the trial version of
Foxit Advanced PDF Editor
To remove this notice, visit:
www.foxitsoftware.com/shopping

CS DATABASE MANAGEMENT SYSTEMS

6302
NOTES o Dense indices – deletion of search-key is similar to
NOTES
file record deletion.
o Sparse indices – if an entry for the search key exists in
the index, it is deleted by replacing the entry in the
index with the next search-key value in the file (in
search-key order). If the next search-key value already
has an index entry, the entry is deleted instead of being
replaced.
 Index Update: Insertion
o Single-level index insertion:
o Perform a lookup using the search-key value appearing
in the record to be inserted.
o Dense indices – if the search-key value does not
appear in the index, insert it.
o Sparse indices – if index stores an entry for each
block of the file, no change needs to be made to the
index unless a new block is created. In this case, the
first search-key value appearing in the new block is
inserted into the index.
o Multilevel insertion (as well as deletion) algorithms
are simple extensions of the single-level algorithms.
Secondary Indices
o Frequently, one wants to find all the records whose
values in a certain field (which is not the search-key of
the primary index) satisfy some conditions.
o Example 1: In the account database stored
sequentially by account number, we may want to find
all accounts in a particular branch.
o Example 2: as above, where do we want to find all
accounts with a specified balance or range of balances?
o We can have a secondary index with an index record
for each search-key value; index record points to a
bucket that contains pointers to all the actual records
with that particular search-key value.
 Secondary Index on balance field of account
  Primary and Secondary Indices
 Secondary indices have to be dense.
 Indices offer substantial benefits when searching for records.
 When a file is modified, every index on the file must be updated,
up- dating indices impose overhead on database modification.
 Sequential scan using primary index is efficient, but a sequential
scan using a secondary index is expensive.
 Each record access may fetch a new block from disk
 Bitmap Indices
n Bitmap indices are a special type of index designed for efficient querying
on multiple keys.
n Records in a relation are assumed to be numbered sequentially from, say, 0.
 Given a number n it must be easy to retrieve record n
 Particularly easy if records are of fixed size.

n Applicable on attributes that take on a relatively small number of distinct

values
 E.g. gender, country, state, …
 E.g. income-level (income broken up into a small number of
levels such as 0-9999, 10000-19999, 20000-50000, 50000-
infinity)
n A bitmap is simply an array of bits.
n In its simplest form, a bitmap index on an attribute has a bitmap for each
value of the attribute.
 Bitmap has as many bits as records.
 In a bitmap for value v, the bit for a record is 1 if the record has
the value v for the attribute, and is 0 otherwise.
 n Bitmap indices are useful for queries on multiple attributes  A
 Not particularly useful for single attribute queries lea
f
n Queries are answered using bitmap operations no
 Intersection (and) de
 Union (or) ha
s
 Complementation (not)
bet
n Each operation takes two bitmaps of the same size and applies the we
operation on corresponding bits to get the result bitmap. en
[(n
 Males with income level L1: 10010 AND 10100 = 10000
–
 Can then retrieve required tuples. 1)/
 Counting the number of matching tuples is even faster 2]
n Bitmap indices are generally very small compared with relation size. an
d
 If number of distinct attribute values is 8, bitmap is only 1% of relation size.
n–
n Deletion needs to be handled properly. 1
 Existence bitmap to note if there is a valid record at a record val
location ue
s
 Needed for complementation
n Should keep bitmaps for all values, even null value

B+-Tree Index Files

o B+-tree indices are an alternative to indexed-sequential files.
o Disadvantage of indexed-sequential files: performance degrades
as file grows, since many overflow blocks get created. Periodic
reorganiza- tion of entire file is required.
o Advantage of B+-tree index files: automatically reorganizes
itself with small, local, changes, in the face of insertions and
deletions. Reor- ganization of entire file is not required to maintain
performance.
o Disadvantage of B+-trees: extra insertion and deletion
overhead, space overhead.
o Advantages of B+-trees outweigh disadvantages, and they are
used extensively.
 A B+-tree is a rooted tree satisfying the following properties:
 All paths from root to leaf are of the same length
 Each node that is not a root or a leaf has between [n/2] and n
children.
NOTES
 Edited with the trial version of
Foxit Advanced PDF Editor
To remove this notice, visit:
www.foxitsoftware.com/shopping

CS DATABASE MANAGEMENT SYSTEMS

6302
NOTES  Special cases:
 If the root is not a leaf, it has at least 2 children.
 If the root is a leaf (that is, there are no other nodes in
the tree), it can have values between 0 and (n–1).
 Typical node

o Ki are the search-key values.

o Pi are pointers to children (for non-leaf nodes) or pointers
to records or buckets of records (for leaf nodes).
 The search-keys in a node are ordered.
K1 < K2 < K3 < . . . < Kn–1
Leaf Nodes in B+-Trees
o Properties of a leaf node:
 For i = 1, 2, . . ., n–1, pointer Pi either points to a
file record with search-key value Ki, or to a bucket
of pointers to file records, each record having
search-key value Ki. It needs bucket structure only
if search-key does not form a primary key.
 If Li, Lj are leaf nodes and i < j, Li’s search-key
values are less than Lj’s search-key values,
Pn points to next leaf node in search-key order

Fig 2.13 Leaf nodes

Non-Leaf Nodes in B+-Trees
 Non leaf nodes form a multi-level sparse index on the leaf nodes.
For a non-leaf node with m pointers:
 o All the search-keys in the subtree to which P1 points are
NOTES
less than K1.
o For 2  i  n – 1, all the search-keys in the subtree to which
Pi points have values greater than or equal to Ki–1 and less
than Km–1.

 Example of a B+-tree

B+-tree for account file (n = 3)

B+-tree for account file (n - 5)

o Leaf nodes must have between 2 and 4
values ((n–1)/2 and n –1, with n = 5).
o Non-leaf nodes other than root must have between 3 and
5 children ((n/2 and n with n =5).
o Root must have at least 2 children.
 Observations about B+-trees
o Since the inter-node connections are done by pointers,
“logi- cally” close blocks need not be “physically” close.
o The non-leaf levels of the B+-tree form a hierarchy of
sparse indices.
o The B+-tree contains a relatively small number of levels
(loga- rithmic in the size of the main file), thus searches can
be con- ducted efficiently.
NOTES o Insertions and deletions to the main file can be handled
efficiently, as the index can be restructured in logarithmic time
(as we shall see).
Queries on B+-Trees
 Find all records with a search-key value of k.
o Start with the root node
 Examine the node for the smallest search-key value
> k.
 If such a value exists, assume it is Kj. Then follow
Pi to the child node.
 Otherwise k  Km–1, where there are m pointers
in the node. Then follow Pm to the child node.
o If the node reached by following the pointer above is not a
leaf node, repeat the above procedure on the node, and
follow the corresponding pointer.
o Eventually reach a leaf node. If for some i, key Ki = k
follow pointer Pi to the desired record or bucket.
Otherwise, no record with search-key value k exists.
 In processing a query, a path is traversed in the tree from the
root to some leaf node.
 If there are K search-key values in the file, the path is no longer
than
 logn/2(K).
 A node is generally the same size as a disk block, typically 4
kilo- bytes, and n is typically around 100 (40 bytes per index
entry).
 With 1 million search key values and n = 100, at most
log50(1,000,000) = 4 nodes are accessed in a lookup.
 Contrast this with a balanced binary free with 1 million search
key values — around 20 nodes are accessed in a lookup.
o above difference is significant since every node access may
need a disk I/O, costing around 20 milliseconds!
 Updates on B+-Trees: Insertion
o Find the leaf node in which the search-key value would appear.
o If the search-key value is already there in the leaf node,
record is added to file and if necessary a pointer is inserted
into the bucket.
o If if necessary. Then:
th
e
se
arc
h-
ke
y
val
ue
is
no
t
th
er
e,
th
en
ad
d
th
e
re
co
rd
to
th
e
m
ain
fil
e
an
d
cr
eat
e
bu
ck
et
  If there is room in the leaf node, insert (key-value, B
pointer) pair in the leaf node +-
 Otherwise, split the node (along with the new Tr
ee
(key- value, pointer) entry) as discussed in the next
be
slide. fo
o Splitting a node: re
 take the n(search-key value, pointer) pairs an
(including the one being inserted) in sorted order. d
af
Place the first
te
 n/2  in the original node, and the rest in a new r
node. in
 let the new node be p, and let k be the least key se
value in p. Insert (k,p) in the parent of the node rti
being split. If the parent is full, split it and o
propagate the split fur- ther. n
of
The splitting of nodes proceeds upwards till a node that is not full is found. In the “
worst case the root node may be split increasing the height of the tree by 1. Cl
ea
rv
ie
w
”

Result of splitting node containing Brighton and Downtown on

inserting Clearview
NOTES
NOTES  Updates on B+-Trees: Deletion
 Find the record to be deleted, and remove it from the main file
and from the bucket (if present)
 Remove (search-key value, pointer) from the leaf node if
there is no bucket or if the bucket has become empty
 If the node has too few entries due to the removal, and the
entries in the node and a sibling fit into a single node, then
 Insert all the search-key values in the two nodes into a
single
node (the one on the left), and delete the other node.
 Delete the pair (Ki–1, Pi), where Pi is the pointer to the
deleted node, from its parent, recursively using the above
procedure.
 Otherwise, if the node has too few entries due to the
removal, and the entries in the node and a sibling fit into a
single node, then
• Redistribute the pointers between the node and a
sibling such that both have more than the minimum
number of en- tries.
• Update the corresponding search-key value in the
parent of the node.
 The node deletions may cascade upwards till a node which has
n/2  or more pointers is found. If the root node has only
one pointer after deletion, it is deleted and the sole child
becomes the root.
 Examples of B+-Tree Deletion
Be
for
e
an
d
aft
er
del
eti
ng
“D
ow
nt
ow
n”
 o The removal of the leaf node containing “Downtown” does
NOTES
not result in its parent having too little pointers. So the
cascaded deletions stop with the deleted leaf node’s parent.

Deletion of “Perryridge” from result of previous example

o Node with “Perryridge” becomes underfull (actually empty in

this special case) and it is merged with its sibling.
o As a result “Perryridge” node’s parent became underfull, and
was merged with its sibling (and an entry was deleted from
their parent)
o Root node then had only one child, and was deleted and its
child became the new root node
B+-Tree File Organization
o Index file degradation problem is solved by using B+-Tree
indices. Data file degradation problem is solved by using B+-
Tree File Organization.
o The leaf nodes in a B+-tree file organization store records,
instead of pointers.
o Since records are larger than pointers, the maximum number
of records that can be stored in a leaf node is less than the
number of pointers in a non leaf node.
o Leaf nodes are still required to be half full.
NOTES o Insertion and deletion are handled in the same way as
insertion and deletion of entries in a B+-tree index.

Fig 2.13 Example of B+-tree File Organization

o Good space utilization is important since records use more
space than pointers.
o To improve space utilization, involve more sibling nodes
in redistribution during splits and merges.
 Involving 2 siblings in redistribution (to avoid split /
merge where possible) results in each node having at
least entries.
B-Tree Index Files
 Similar to B+-tree, but B-tree allows search-key values
to appear only once; eliminates redundant storage of
search keys.
 Search keys in nonleaf nodes appear nowhere else in the
B- tree; an additional pointer field for each search key in
a nonleaf node must be included.
 Generalized B-tree leaf node

Fig 2.14 Nonleaf node – pointers Bi are the bucket or file record pointers.
  B-Tree Index File Example
NOTES

Fig 2.15 B-tree (above) and B+-tree (below) on same data

o Advantages of B-Tree indices:

 May use less tree nodes than a corresponding B+-
Tree.
 Sometimes possible to find search-key value
before reaching leaf node.
o Disadvantages of B-Tree indices:
 Only small fraction of all search-key values are
found early.
 Non-leaf nodes are larger, so fan-out is reduced. Thus
B-Trees typically have greater depth than
corresponding B+-Tree.
 Insertion and deletion in B- Trees more complicated
than in B+-Trees
 Implementation is harder than B+-Trees.
o Typically, advantages of B-Trees do not out
weigh disadvantages.

Key words :
Secondary storage , Magnetic Disk , cache , main memory , optical , primary ,
second- ary , tertiary , hashing , indexing , B Tree , B+ Tree , Node , Leaf Node
, Bucket
NOTES Short Questions :
1. Write a short note on Classification of Physical Storage Media.
2. Explain Storage Hierarchy.
3. What is Magnetic Hard Disk Mechanism?
4. What is Performance Measures of Disks?
5. Write a short note on Improvement in Performance via Parallelism.
6. Explain Levels of RAID.
7. Write a short note on Mapping of Objects to Files.
8. Explain Extendable Hash Structure.
9. Define Indexing.
10.Explain B+Tree File Organization.
Answer Vividly :
1. Explain RAID Architecture.
2. Explain Factors in choosing RAID level and Hardware Issues.
3. Briefly describe Various File Operations.
4. Write a note on Organization of Records in Files.
5. Explain Static & Dynamic Hashing.
6. Exlain the basic kinds of indices.
7. Explain B+Tree Index files.
Summary
 Many alternative file organizations exist, each appropriate in some situation.
 If selection queries are frequent, sorting the file or building an index is
impor- tant.
 Hash-based indexes are only good for equality search.
 Sorted files and tree-based indexes best for range search; also good for
equal- ity search. (Files rarely kept sorted in practice; B+ tree index is
better.)
 Index is a collection of data entries plus a way to quickly find entries with
given key values.
 Data entries can be actual data records, <key, rid> pairs, or <key, rid-
list> pairs.
 Choice orthogonal to indexing technique is used to locate data entries
with a given key value.
 It can have several indexes on a given file of data records, each with a
different search key.
  Indexes can be classified as clustered vs. unclustered, primary vs.
NOTES
secondary, and dense vs. sparse. Differences have important consequences
for utility/ performance.
 Understanding the nature of the workload for the application, and the
performance goals, is essential to develop a good design.
 What are the important queries and updates? What attributes/relations
are involved?
 Indexes must be chosen to speed up important queries (and perhaps
some updates!).
 Index maintenance overhead on updates to key fields.
 Choose indexes that can help many queries, if possible.
 Build indexes to support index-only strategies.
 Clustering is an important decision; only one index on a given relation
can be clustered!
 Order of fields in composite index key can be important.
References :
1. 1.Ramez Elamasri and shankant B.Navathe, “ Fundamentals of
Database Systems”, Third Edition , Pearson Education, Delhi,
2002.
2. 2.Abraham Silberschatz, Henry F.Korth and S.Sundarshan ,
“Data- base System concepts “, Fourth Edition , McGrawHill ,
2002.
3. 3.C.J.Date,”An Introduction to Database
Systems”,Seventh Edition,Pearson Education,Delhi, 2002.
 NOTES

RELATIONAL MODEL
Learning Objectives
 To Learn Relational Model Concepts
 To learn about RelationalAlgebra
 To Study SQL , Queries , Views and Constraints
 To learn about Relational Calculus
 To Have idea of Commercial RDBMSs
 To Learn to design Databases with functional dependencies
 To learn about various Normal Forms and Database Tuning

INTRODUCTION
Data Models
o A data model is a collection of concepts for describing data.
o A schema is a description of a particular collection of data, using a
given data model.
o The relational model of data is the most widely used model today.
o Main concept: relation, basically a table with rows and columns.
o Every relation has a schema, which describes the columns, or
fields (that is, the data’s structure).
Relational Database: Definitions
o Relational database: a set of relations
o Relation: made up of 2 parts:
o Instance : a table, with rows and columns.
o Schema : specifies name of relation, plus name and type of
each column.
 E.G. Students(sid: string, name: string, login: string,
age: integer, gpa: real).
o It can think of a relation as a set of rows or tuples that share the same
structure.

CS 6302 78
79 CS 6302
 Importance of the Relational Model
NOTES
 Most widely used model.
 Vendors: IBM, Informix, Microsoft, Oracle, Sybase,etc.
 “Legacy systems” in older models
 E.G., IBM’s IMS
 Recent competitor: XML
 A synthesis emerging: XML & Relational
 BEA; numerous start-ups

Fig 3.1 Example of a Relation

About Relational Model
o Order of tuples not important but Order of attributes not important
(in theory)
o Collection of relation schemas with intension of Relational
database schema
o Corresponding relation instances as extension of Relational
database
o intension vs. extension simulates schema vs. data
o metadata includes schema.
Good Schema
At the logical level…
o Easy to understand
o Helpful for formulating correct
queries At the physical storage level…
o Tuples are stored efficiently.
o Tuples are accessed efficiently.
Design approaches
Top-down
o Start with groupings of attributes achieved from the
conceptual design and mapping.
o Design by analysis is applied.

CS 6302 80
 NOTES Bottom-
up o Consider relationships between attributes.
o Build up relations.
o It is also called design by synthesis.
Informal Measures for Design Semantics of the attributes.
o Design a relation schema so that it is easy to explain its meaning.
o A relation schema should correspond to one semantic object
(entity or relationship).
o Example: The first schema is good due to clear meaning.
Faculty (name, number, office)
Department (dcode, name, phone)
or
Faculty_works (number, name, Salary, rank, phone, email)
Reduce redundant data
o Design has a significant impact on storage requirements.
o The second schema needs more storage due to redundancy.
Faculty and Department
or
Faculty_works
Avoid update anomalies
Relation schemes can suffer from update anomalies
Insertion anomaly
1) Insert new faculty into faculty_works
o We must keep the values for the department
consistent between tuples
2) Insert a new department with no faculty members into faculty_works
o We would have to insert nulls for the faculty info.
o We would have to delete this entry later.
Deletion anomaly
o Delete the last faculty member for a department from the
faculty_works relation.
o If we delete the last faculty member for a department from the
database, all the department information disappears as well.
o This is like deleting the department from the database.
 Modification anomaly
NOTES
o Update the phone number of a department in the
faculty_works relation.
o We would have to search out each faculty member that works in
that department and update the phone information in each of those
tuples.
Reduce null values in tuples
o Avoid attributes in relations whose values may often be null.
o Reduces the problem of “fat” relations
o Saves physical storage space
o Don’t include a “department name” field for each employee.
Avoid spurious tuples
o Design relation schemes so that they can be joined with
equality conditions on attributes that are either primary or
foreign keys.
o If you don’t, spurious or incorrect data will be generated
o Suppose we replace
Section (number, term, slot, cnum, dcode, faculty_num)
with
Section_info (number, cnum, dcode, term, slot)
Faculty_info (faculty_num, name)
then
Section != Section_info * Faculty_info
Relational Design
Simplest approach (not always best):
Convert each Entity Set to a relation and each relationship to a relation.
o Entity Set  Relation
o E.S. attributes become relational attributes.

name manf

Beers

Becomes: Beers(name, manf)

Relational Model
o Table = relation.
o Column headers = attributes.
o Row = tuple
NOTE E.g. keys, other constraints. Example: Beers(name, manf)
o Order of attributes is arbitrary, but in practice we need to assume
S
the order given in the relation schema.
 Relation instance is current set of rows for a relation schema.
 Database schema = collection of relation schemas.
Structure
o Formally, given sets D1, D2, …. Dn a relation r is a subset of
D1 x D2 x … x Dn
Thus a relation is a set of n-tuples (a1, a2, …, an)
where each ai  Di
o Example: if
o customer-name = {Jones, Smith, Curry, Lindsay}
customer-street = {Main, North, Park}
customer-city = {Harrison, Rye,
Pittsfield}
Then r = { (Jones, Main, Harrison),
(Smith, North, Rye),
(Curry, North, Rye),
(Lindsay, Park, Pittsfield)}
is a relation over customer-name x customer-street x
customer- city
Relational Data Model

Relation as table Set theoretic

Domain — set of values
Rows = tuples like a data type
Columns = components Cartesian product (or product)
Names of columns = attributes D1 □D2 □... □ Dn
Set of attribute names = n-tuples (V1,V2,...,Vn)
s.t., V1 □D1, V2 □D2,...,Vn □Dn
schema Relation-subset of cartesian product
REL (A1,A2,...,An) of one or more domains
A1 A2 A3 ... FINITE only; empty set allowed
An Attributes Tuples = members of a relation inst.
C Tuple Arity = number of domains
a a1 a2 a3 an Components = values in a tuple
Domains — corresp. with attributes
r Cardinality = number of tuples
d b1 b2 a3 cn
i Component
n b3 bn
a1 c3
a .
l .
i .
t
y x1 v2 d3 wn
Arity
Fig 3.2 Relation as Table
Attribute Types
NOTES
o Each attribute of a relation has a name
o The set of allowed values for each attribute is called the domain of
the attribute
o Attribute values are (normally) required to be atomic, that is, indivisible
o E.g. multivalued attribute values are not atomic
o E.g. composite attribute values are not atomic
o The special value null is a member of every domain
o The null value causes complications in the definition of many operations
 we shall ignore the effect of null values in our main
presenta- tion and consider their effect later.
Relation Schema
o A1, A2, …, An are attributes
o R = (A1, A2, …, An ) is a relation schema
o E.g. Customer-schema =
(customer-name, customer-street, customer-city)
o r(R) is a relation on the relation schema R
o E.g. customer (Customer-schema)

Relation Instance
 The current values (relation instance) of a relation are specified by a table
 An element t of r is a tuple, represented by a row in a table

attributes
(or columns)
customer- customer- customer-city
name street
Jones Main Harrison
Smith North Rye tuples
Curry North Rye (or rows)
Lindsay Park Pittsfield

customer
Fig 3.3 Relation
instances
Name Address Telephone
Bob 123 Main St 555-1234
Bob 128 Main St 555-1235
Pat 123 Main St 555-1235
Harry 456 Main St 555-2221
Sally 456 Main St 555-2221
NOTE Sally 456 Main St 555-2223
Pat 12 State St 555-1235
S
Unordered Relations
 Order of tuples is irrelevant (tuples may be stored in an arbitrary order)
 E.g. account relation with unordered tuples

Fig 3.4 account relation

RELATIONALALGEBRA
 A basic set of relational model operations constitute the relational algebra.
 It is a Procedural language with Six basic operators.
o select
o project
o union
o set difference
o Cartesian product
o rename
 The operators take two or more relations as inputs and give a new
relation as a result.
Select Operation – Example
A B C
• Relation r D
  1 7
  5 7
  1 3
  2 1
2 0
3

• A=B ^ D > 5 (r)

A B C D

  1 7
  23 10

Fig 3.5 Select operation

 Select Operation
NOTES
 Notation:  p(r)
 p is called the selection
predicate.
 Defined as:
 p(r) = {t | t  r and p(t)}
 Where p is a formula in propositional calculus consisting of terms
connected by :  (and),  (or),  (not)
Each term is one of:
 <attribute> op <attribute> or <constant>
 where op is one of: =, , >, . <. 
 Example of selection:
 branch-name=“Perryridge”(account)
 Notation:  p(r)
 p is called the selection predicate
 Defined as:
 p(r) = {t | t  r and p(t)}
 Where p is a formula in propositional calculus consisting of terms
connected by :  (and),  (or),  (not)
Each term is one of:
 <attribute> op <attribute> or <constant>
 where op is one of: =, , >, . <. 
 Example of selection:
 branch-name=“Perryridge”(account)
Project Operation – Example

 Relation r:

 A,C (r) A C A C

 1  1
 1  1
 1 =  2
 2

Fig 3.6 Project Operation

NOTE Project Operation
S  Notation:
A1, A2, …, Ak (r)
 where A1, A2 are attribute names and r is a relation name.
 The result is defined as the relation of k columns obtained by erasing
the columns that are not listed.
 Duplicate rows are removed from result, since relations are sets.
 E.g. To eliminate the branch-name attribute of account
account-number, balance (account)
Union Operation – Example

 Relations r, s: A B A B

 1  2
 2  3
 1

s
r

r  s: A B

 1
 2
 1
 3

Fig 3.7 Union Operation

Union Operation
 Notation: r  s
 Defined as:
 r  s = {t | t  r or t  s}

 For r  s to be valid.
 r, s must have the same arity (same number of attributes)
 The attribute domains must be compatible (e.g., 2nd column
of r deals with the same type of values as does the
2nd
column of s)
 E.g. to find all customers with either an account or a loan
customer-name (depositor)  customer-name (borrower)
87 CS 6302
Set Difference Operation – Example
NOTES
 Relations r, s: A B A B

 1  2
 2  3
 1

s
r

r – s: A B

 1
 1

Fig 3.8 Set Difference Operation

Set Difference Operation
 Notation r – s
 Defined as:
 r – s = {t | t  r and t  s}
 Set differences must be taken between compatible relations.
o r and s must have the same arity.
o attribute domains of r and s must be compatible.
Cartesian-Product Operation-Example
A B C D
Relations r, s: E
 1  10 a
 2  10 a
 20 b
r  10 b
s
r x s:

A B C D E

 1  10 a
 1  10 a
 1  20 b
 1  10 b
 2  10 a
 2  10 a
 2  20 b

2  10 b

Fig 3.9 Cartesian Product Operation

Cartesian-Product Operation
 Notation r x s
 Defined as:
CS 6302 88
NOTE  r x s = {t q | t  r and q  s}
S  Assume that attributes of r(R) and s(S) are disjoint. (That is,
R  S = ).
 If attributes of r(R) and s(S) are not disjoint, then renaming must be used.
Composition of Operations

 Can build expressions using multiple operations

 Example:
 A=C(r x s)
A B C D
 rxs E
 1  10 a
 1  10 a
 1  20 b
 1  10 b
 2  10 a
 2  10 a
 2  20 b
2 10 b
 
 A=C(r x s) A B C D E

 1  1 a
 2  0 a
 2  2 b
0

Fig 3.10 Composition of Operations

Rename Operation
 Allows us to name, and therefore to refer to, the results of
relational- algebra expressions.
 Allows us to refer to a relation by more than one
name. Example:
 x (E)
returns the expression E under the name
X
If a relational-algebra expression E has arity n, then
x (A1, A2, …, An) (E)
returns the result of expression E under the name X, and with
the attributes renamed to A1, A2, …., An.

Banking Example
branch (branch-name, branch-city, assets)
customer (customer-name, customer-street, customer-only)
account (account-number, branch-name, balance)
loan (loan-number, branch-name, amount)
 depositor (customer-name, account-number)
NOTES
borrower (customer-name, loan-number)

Example Queries
 Find all loans of over $1200
 amount > 1200 (loan)
 Find the loan number for each loan of an amount greater than
$1200
 loan-number (amount > 1200 (loan))

 Find the names of all customers who have a loan, an account, or

both, from the bank
customer-name (borrower)  customer-name (depositor)
 Find the names of all customers who have a loan and an
account at bank.
customer-name (borrower)  customer-name (depositor)

 Find the names of all customers who have a loan at the Perryridge
branch.
 Query 1
customer-name(branch-name = “Perryridge” (
borrower.loan-number = loan.loan-number(borrower x loan)))
 Query 2
customer-name(loan.loan-number = borrower.loan-number
((branch-name = “Perryridge”(loan)) x borrower))
 Find the largest account balance
 Rename account relation as d
 The query is:

balance(account) - account.balance
(account.balance < d.balance (account x d (account)))

Formal Definition of Relational Algebra

 A basic expression in the relational algebra consists of either one of
the following:
o A relation in the database
o A constant relation
 Let E1 and E2 be relational-algebra expressions; the following are
all relational-algebra expressions:
o E1  E2
NOTE o E1 - E2
S o E1 x E2
o p (E1), P is a predicate on attributes in E1
o s(E1), S is a list consisting of some of the attributes in E1
o  x (E1), x is the new name for the result of E1
Additional Operations
Additional operations do not add any power to
the relational algebra, but simplify common
queries.
 Set intersection
 Natural join
 Division
 Assignment
Set-Intersection Operation
 Notation: r  s
 Defined as:
r  s ={ t | t  r and t  s }
 Assume:
o r, s have the same arity
o Attributes of r and s are compatible.
 Note: r  s = r - (r - s)

Example :

 Relation
A r, s: B A B
 1  2
 2  3
 1

 rs rs

A B
 2

Fig 3.11 Set intersection Operation

Natural-Join Operation
 Notation: r s
NOTES

 Let r and s be relations on schemas R and S respectively.

Then, r s is a relation on schema R  S obtained as follows:
 Consider each pair of tuples tr from r and ts from s.
 If tr and ts have the same value on each of the attributes in R  S,
add a tuple t to the result, where
 t has the same value as tr on r
 t has the same value as ts on s
 Example:
R = (A, B, C, D)
S = (E, B, D)
 Result schema = (A, B, C, D, E)
 r s is defined as:
r.A, r.B, r.C, r.D, s.E (r.B = s.B  r.D = s.D (r x s))

Example
:
 Relations r, s:

B D
A B C D

 1  a 1 a 
E
 2  a 3 a 
 4  b 1 a 
 1  a 2 b 
 2  b 3 b 
r s

r s A B C D E

 1  a 
 1  a 
 1  a 
 1  a 
 2  b 

Fig 3.12 Natural Join Operation

NOTE Division Operation
rs
S
 Suited to queries that include the phrase “for all”.
 Let r and s be relations on schemas R and S respectively where
 R = (A1, …, Am, B1, …, Bn)
 S = (B1, …, Bn)
The result of r  s is a relation on schema
R – S = (A1, …, Am)

r  s = { t | t   R-S(r)   u  s ( tu  r ) }

Example 1
:
Relations r, s: A B B

 1 s
 2
 3
 1
 1
 1
 3
4

6
r  s: A  r
1
 2
 


Fig 3.13 Division Operation 1

Example 2
: Relations r, s: A B C D E

 a  a 1
 a  a 1
 a  b 1 D E
 a  a 1
 a  b 3 a 1
 a  a 1 b 1
 a  b 1 s
a  b 1

r

r  s:

A B C

 a 
 a 

Fig 3.14 Division Operation 2

1
2
  Property
NOTES
o Let q – r  s
o Then q is the largest relation satisfying q x s  r
 Definition in terms of the basic algebra
operation Let r(R) and s(S) be relations, and
let S  R
 r  s = R-S (r) –R-S ( (R-S (r) x s) – R-S,S(r))
 To see why

o R-S,S(r) simply reorders attributes of r

o R-S(R-S (r) x s) – R-S,S(r)) gives those tuples t in

R-S (r) such that for some tuple u  s, tu  r.

Assignment Operation
 The assignment operation () provides a convenient way to express
com- plex queries.
o Write query as a sequential program consisting of
 a series of assignments
 followed by an expression whose value is displayed
as a result of the query.
o Assignment must always be made to a temporary relation
variable.
 Example: Write r  s as

 temp1  R-S (r)

temp2  R-S ((temp1 x s) – R-S,S
(r))
result = temp1 – temp2
o The result to the right of the  is assigned to the relation variable
on the left of the .
o May use variable in subsequent expressions.
NOTE Example Queries
S  Find all customers who have an account from at least
the “Downtown” and the Uptown” branches.
Query 1

CN(BN=“Downtown”(depositor account)) 
CN(BN=“Uptown”(depositor account))

where CN denotes customer-name and BN denotes

branch-name.
Query 2

customer-name, branch-name (depositor account)

  temp(branch-name) ({(“Downtown”), (“Uptown”)})
 Find all customers who have an account at all branches located in Brooklyn
city.

customer-name, branch-name (depositor account)

 branch-name (branch-city = “Brooklyn” (branch))

Aggregate Functions and Operations

 Aggregation function takes a collection of values and returns a single

value. As a result.,
 avg: average value
min: minimum value
max: maximum
value sum: sum of
values
count: number of values
 Aggregate operation in relational algebra
 G1, G2, …, Gn g F1( A1), F2( A2),…, Fn( An) (E)
o E is any relational-algebra expression.
o G1, G2 …, Gn is a list of attributes on which group (can be
empty).
o Each Fi is an aggregate function.
o Each Ai is an attribute name.
  Relation r:
NOTES
A B C

  7
  7
  3
  10

g sum(c) (r)
sum-C

27
Fig 3.15 Aggregate Operation

 Result of aggregation does not have a name

 Can use rename operation to give it a name.
 For convenience, we permit renaming as part of aggregate operation
 Relation account grouped by branch-name:

branch-name account-number balance

Perryridge A-102 400
Perryridge A-201 900
Brighton A-217 750
Brighton A-215 750
Redwood A-222 700

branch-name g sum(balance) (account)

branch-name balance
Perryridge 1300
Brighton 1500
Redwood 700
Fig 3.16 Aggregate Operation - group by

Outer Join

 An extension of the join operation that avoids loss of information.

 Computes the join and then adds tuples form one relation that does
not match tuples in the other relation to the result of the join.
NOTES  Uses null values:
o null signifies that the value is unknown or does not exist.
o All comparisons involving null are (roughly speaking) false
by definition.
 We will study precise meaning of comparisons with nulls
later

 Relation loan

loan-number branch-name amount

L-170 Downtown 3000
L-230 Redwood 4000
L-260 Perryridge 1700

 Relation borrower
customer- loan-number
Jones L-170
Smith L-230
Hayes L-155

 Inner Join
loan Borrower

loan-number branch-name amount customer-

L-170 Downtown 3000 Jones
L-230 Redwood 4000 Smith

 Left Outer Join

loan Borrower

loan-number branch-name amount customer-

L-170 Downtown 3000 Jones
L-230 Redwood 4000 Smith
L-260 Perryridge 1700 null
 Right Outer Join

loan borrower
NOTES
loan-number branch-name amount customer-
L-170 Downtown 3000 Jones
L-230 Redwood 4000 Smith
L-155 null null Hayes

 Full Outer Join

Loan borrower
loan-number branch-name amount customer-
L-170 Downtown 3000 Jones
L-230 Redwood 4000 Smith
L-260 Perryridge 1700 null
L-155 null null Hayes

Null Values

 It is possible for tuples to have a null value, denoted by null, for some of
their attributes.
 null signifies an unknown value or that a value does not exist.
 The result of any arithmetic expression involving null is null.
 Aggregate functions simply ignore null values.
o It is an arbitrary decision. It could have returned null as
result instead.
o We follow the semantics of SQL in its handling of null values.
 For duplicate elimination and grouping, null is treated like any other
value, and two nulls are assumed to be the same.
o Alternative: assume each null is different from each other
o Both are arbitrary decisions, so we simply follow SQL.
 Comparisons with null values return the special truth value unknown.
o If false was used instead of unknown, then not (A < 5)
would not be equivalent to A >= 5
 Three-valued logic using the truth value unknown:
o OR: (unknown or true) = true,
(unknown or false) = unknown
(unknown or unknown) =
unknown
o AND: (true and unknown) = unknown,
NOTE (false and unknown) = false,
S (unknown and unknown) =
unknown
o NOT: (not unknown) = unknown
o In SQL “P is unknown” evaluates to true if predicate P evaluates
to
unknown
 Result of select predicate is treated as false if it evaluates to unknown

Modification of the Database

 The content of the database may be modified using the following operations:
o Deletion
o Insertion
o Updating
 All these operations are expressed using the assignment operator.
Deletion
 A delete request is expressed similarly to a query, except instead of
displaying tuples to the user, the selected tuples are removed from
the database.
 It can delete only whole tuples; but it cannot delete values on only
particular attributes.
 A deletion is expressed in relational algebra by:
 rr–E
 where r is a relation and E is a relational algebra query.
Examples

 Delete all account records in the Perryridge branch.

account  account –  branch-name = “Perryridge” (account)
 Delete all loan records with amount in the range of 0 to 50
loan  loan –  amount  0 and amount  50 (loan)

 Delete all accounts at branches located in Needham.

r1   branch-city = “Needham” (account branch)

r2  branch-name, account-number, balance (r1)
r3   customer-name, account-number (r2 depositor)
account  account – r2
depositor  depositor – r3
Insertion
NOTES
 To insert data into a relation, we either:
o specify a tuple to be inserted
o write a query whose result is a set of tuples to be inserted
 in relational algebra, an insertion is expressed by:
 rrE
 where r is a relation and E is a relational algebra expression.
 The insertion of a single tuple is expressed by letting E be a constant
relation containing one tuple.

Examples
 Insert information in the database specifying that Smith has
$1200 in account A-973 at the Perryridge branch.
account  account  {(“Perryridge”, A-973, 1200)}
depositor  depositor  {(“Smith”, A-973)}
 Provide as a gift for all loan customers in the Perryridge
branch, a $200 savings account. Let the loan number serve
as the account number for the new savings account.
r1  (branch-name = “Perryridge” (borrower loan))
account  account  branch-name, account-number,200 (r1)
depositor  depositor  customer-name, loan-number(r1)
Updating

 It is a mechanism to change a value in a tuple without charging all

values in the tuple.
 Use the generalized projection operator to do this task
 r   F1, F2, …, FI, (r)
 Each Fi is either
o the ith attribute of r, if the ith attribute is not updated, or,
o if the attribute is to be updated Fi is an expression, involving
only constants and the attributes of r, which gives the new
value for the attribute.
NOTE Examples
S  Make interest payments by increasing all balances by 5 percent.
account   AN, BN, BAL * 1.05 (account)

where AN, BN and BAL stand for account-number, branch-name

and balance, respectively.
 Pay all accounts with balances over $10,000 6
percent interest and pay all others 5 percent

account   AN, BN, BAL * 1.06 ( BAL  10000 (account))

 AN, BN, BAL * 1.05 (BAL  10000 (account))

STRUCTURED QUERY LANGUAGE (SQL)

Introduction
 Structured Query Language (SQL) is a data sub language that has
constructs for defining and processing a database.

It can be
H Used stand-alone within a DBMS command
H Embedded in triggers and stored procedures
H Used in scripting or programming languages

History of SQL-92
 SQL was developed by IBM in late 1970s.
 SQL-92 was endorsed as a national standard by ANSI in 1992.
 SQL3 incorporates some object-oriented concepts but has not
gained acceptance in industry.
 Data Definition Language (DDL) is used to define database structures.
 Data Manipulation Language (DML) is used to query and update data.
 Data Control Language (DCL) is used to have control on transactions.
Example : Commit, Purge which are administrative level commands.
 SQL statement is terminated with a semicolon.

Create Table
CREATE TABLE statement is used for creating relations
 Each column is described with three parts: column name, data type,
and optional constraints
 Example: NOTES
CREATE TABLE PROJECT (ProjectID Integer Primary Key, Name
Char(25) Unique Not Null, Department VarChar (100) Null, MaxHours
Numeric(6,1) Default 100);

Data Types
 Standard data types
- Character-string for fixed-length character , bit-string , date and time
- VarChar for variable-length character- It requires additional processing
than Char data types
- Numeric ( Integer or Int and smallInt)
- Real numbers of various precision such as float, real and double precision.
- BLOB, CLOB – Binary Large objects to store objects like images , video
etc.

Constraints
 Constraints can be defined within the CREATE TABLE statement, or they can
be added to the table after it is created using the ALTER table statement.
 Five types of constraints:
H PRIMARY KEY may not have null values.
H UNIQUE may have null values.
H NULL/NOT NULL
H FOREIGN KEY
H CHECK
 Example:
CREATE TABLE PROJECT (ProjectID Integer Primary Key, Name
Char(25) Unique Not Null, Department VarChar (100) Null, MaxHours
Numeric(6,1) Default 100);

ALTER Statement

 ALTER statement changes table structure, properties, or constraints after it

has been created.

 Example
ALTER TABLE ASSIGNMENT ADD CONSTRAINT EmployeeFK FOR-
EIGN KEY (EmployeeNum) REFERENCES EMPLOYEE
(EmployeeNumber) ON UPDATE CASCADE ON DELETE NO ACTION;
NOTE DROP Statements
S DROP TABLE statement removes tables and their data from the database.
 A table cannot be dropped if it contains foreign key values needed by other
table. H Use ALTER TABLE DROP CONSTRAINT to remove integrity
constraints in the other table.
Example:
H DROP TABLE CUSTOMER;
H ALTER TABLE ASSIGNMENT DROP CONSTRAINT ProjectFK;

SELECT Statement

 SELECT can be used to obtain values of specific columns,

specific rows, or both.

Basic format:

SELECT (column names or *) FROM (table name(s)) [WHERE

(conditions)];
Example :
SELECT Name, Department FROM EMPLOYEE ;

WHERE Clause Conditions

 Require quotes around values for Char and VarChar columns, but no quotes
for Integer and Numeric columns.
 AND may be used for compound conditions.
 IN and NOT IN indicate ‘match any’ and ‘match all’ sets of values, respectively.
 Wildcards _ and % can be used with LIKE to specify a single or multiple
unknown characters, respectively.
 IS NULL can be used to test for null
values Example: SELECT Statement
SELECT Name, Department, MaxHours FROM
PROJECT;WHERE Name=”XYX”;
Sorting the Results

ORDER BY phrase can be used to sort rows from SELECT statement

SELECT Name, Department FROM EMPLOYEE ORDER BY

Department;
  Two or more columns may be used for sorting purposes.  add_mo
SELECT Name, Department FROM EMPLOYEE ORDER BY nths(dat
e,
Department DESC, Name ASC; months)
INSERT INTO Statement
 The order of the column names must match the order of the values.
 Values for all NOT NULL columns must be provided.
 No value needs to be provided for a surrogate primary key.
 It is possible to use a select statement to provide the values for bulk
inserts from a second table.
 Examples:
– INSERT INTO PROJECT VALUES (1600, ‘Q4 Tax Prep’, ‘Accounting’,
100);
– INSERT INTO PROJECT (Name, ProjectID) VALUES (‘Q1+ Tax Prep’,
1700);

UPDATE Statement
 UPDATE statement is used to modify values of existing data
 Example:
UPDATE EMPLOYEE SET Phone = ‘287-1435’ WHERE Name =
‘James’;
 UPDATE can also be used to modify more than one column value at a time
UPDATE EMPLOYEE SET Phone = ‘285-0091’, Department =
‘Produc- tion’ WHERE EmployeeNumber = 200;

DELETE FROM Statement

 Delete statement eliminates rows from a table
 Example
DELETE FROM PROJECT WHERE Department = ‘Accounting’; ON
DE- LETE CASCADE removes any related referential integrity constraint

BASIC QUERIES USING SINGLE ROW FUNCTIONS

Date Functions

 months_between(date1, date2)
1. select empno, ename, months_between (sysdate, hiredate)/12 from
emp;
2. select empno, ename, round((months_between(sysdate, hiredate)/12),
0) from emp;
3. select empno, ename, trunc((months_between(sysdate, hiredate)/12),
0) from emp;
NOTES
NOTE 1. select ename, add_months (hiredate, 48) from emp;
S 2. select ename, hiredate, add_months (hiredate, 48) from emp;
 last_day(date)
1. select hiredate, last_day(hiredate) from emp;
 next_day(date, day)
1. select hiredate, next_day(hiredate, ‘MONDAY’) from emp;
 Trunc(date, [Foramt])
1. select hiredate, trunc(hiredate, ‘MON’) from emp;
2. select hiredate, trunc(hiredate, ‘YEAR’) from emp;
Character Based Functions
 initcap(char_column)
1. select initcap(ename), ename from emp;
 lower(char_column)
1. select lower(ename) from emp;
 Ltrim(char_column, ‘STRING’)
1. select ltrim(ename, ‘J’) from emp;
 Ltrim(char_column, ‘STRING’)
1. select rtrim(ename, ‘ER’) from emp;
 Translate(char_column, ‘search char,‘replacement char)
1. select ename, translate(ename, ‘J’, ‘CL’) from emp;
 replace(char_column, ‘search string’,‘replacement string’)
1. select ename, replace(ename, ‘J’, ‘CL’) from emp;
 Substr(char_column, start_loc, total_char)
1. select ename, substr(ename, 3, 4) from emp;
Mathematical Functions
 Abs(numerical_column)
1. select abs(-123) from dual;
 ceil(numerical_column)
1. select ceil(123.0452) from dual;
 floor(numerical_column)
1. select floor(12.3625) from dual;
 Power(m,n)
1. select power(2,4) from dual;
 Mod(m,n)
1. select mod(10,2) from dual;
 Round(num_col, size)
1. select round(123.26516, 3) from dual;
  Trunc(num_col,size)
NOTES
1. select trunc(123.26516, 3) from dual;
 sqrt(num_column)
1. select sqrt(100) from dual;
COMPLEX QUERIES USING GROUP FUNCTIONS
Group Functions
There are five built-in functions for SELECT statement:
1. COUNT counts the number of rows in the result.
2. SUM totals the values in a numeric column.
3. AVG calculates an average value.
4. MAX retrieves a maximum value.
5. MIN retrieves a minimum value.
 Result is a single number (relation with a single row and a single column).
 Column names cannot be mixed with built-in functions.
 Built-in functions cannot be used in WHERE clauses.
Example: Built-in Functions
1. Select count (distinct department) from project;
2. Select min (maxhours), max (maxhours), sum (maxhours) from project;
3. Select Avg(sal) from emp;
Built-in Functions and Grouping
 GROUP BY allows a column and a built-in function to be used together.
 GROUP BY sorts the table by the named column and applies the
built-in function to groups of rows having the same value of the
named column.
 WHERE condition must be applied before GROUP BY phrase.
 Example
1. Select department, count (*) from employee where employee_number
< 600 group by department having count (*) > 1
VIEWS
Base relation
A named relation whose tuples are physically stored in the database is
called as Base Relation or Base Table.
Definition
 It is the tailored presentation of the data contained in one or more tables.

 It is the dynamic result of one or more relational operations operating on

the base relations to produce another relation.
NOTE  A view is a virtual relation that does not actually exist in the database
but is produced upon request by a particular user, at the time of request.
S
 A view is defined using the create view statement which has the form
create view v as <query expression
 where <query expression> is any legal relational algebra query
expression. The view name is represented by v.
 Once a view is defined, the view name can be used to refer to the virtual
relation that the view generates.
 View definition is not the same as creating a new relation by evaluating
the query expression.
 Rather, a view definition causes the saving of an expression; the
expression is substituted into queries using the view.
Usage
 Provide a powerful and flexible security mechanism by hiding parts of
the database from certain users.
 The user is not aware of the existence of any attributes or tuples that are
missing from the view.

 Permit users to access data in a way that is customized to their needs, so

that different users can see the same data in different ways, at the same
time.
 Simplify (for the user) complex operations on the base relations.
Advantages
The following are the advantages of views over normal tables.
1. They provide additional level of table security by restricting access to the
data
2. They hide data complexity.
3. They simplify commands by allowing the user to select information from
mul- tiple tables.
4. They isolate applications from changes in data definition.
5. They provide data in different perspectives.
Creation of a view
The following SQL Command is used to create a View in Oracle.
SQL> CREATE [OR REPLACE] VIEW <VIEW-NAME> AS < ANY
VAL
ID
SEL
ECT
STA
TE
ME
NT>
;
 Example:
NOTES
SQL> CREATE VIEW V1 AS SELECT EMPNO, ENAME, JOB FROM
EMP;

SQL> CREATE VIEW EMPLOC AS SELECT EMPNO, ENAME, DEPTNO,

LOC FROM EMP, DEPT WHERE EMP.DEPTNO=DEPT.DNO;

Update of Views
The following SQL Command is used to update a View in Oracle.
SQL> UPDATE VIEW <VIEW-NAME> SET <COLUMNAME= new value >

WHERE (CONDITION);

SQL> UPDATE VIEW EMPLOC SET LOC=’L.A’WHERE LOC =’CHICAGO’;

Dropping a View
The following SQL Command is used to drop a View in Oracle.
SQL> DROP VIEW <VIEW NAME>;
SQL> DROP VIEW VI;

SQL> DROP VIEW EMPLOC;

Disadvantages of Views
 In some cases, it is not desirable for all users to see the entire logical
model (i.e. all the actual relations stored in the database.)
 Consider a person who needs to know a customer’s loan number but has
no need to see the loan amount. This person should see a relation
described, in the relational algebra, by
 customer-name, loan-number (borrower loan)
 Any relation that is not of the conceptual model but is made visible to a
user as a “virtual relation” is called a view.
 Consider the view (named all-customer) consisting of branches and
Examples their customers.
create view all-customer as
 branch-name, customer-name (depositor account)
  branch-name, customer-name (borrower loan)

 We can find all customers of the Perryridge branch by writing:

branch-name
(branch-name = “Perryridge” (all-customer))
NOTE Updates Through View
S  Database modifications expressed as views must be translated to
modifica- tions of the actual relations in the database.
 Consider the person who needs to see all loan data in the loan relation
except amount. The view given to the person, branch-loan, is defined
as:
 create view branch-loan as
o branch-name, loan-number (loan)
 Since we allow a view name to appear wherever a relation name is
allowed, the person may write:
 branch-loan  branch-loan  {(“Perryridge”, L-37)}
 The previous insertion must be represented by an insertion into the
actual relation loan from which the view branch-loan is constructed.
 An insertion into loan requires a value for amount. The insertion can
be dealt with by either.
o rejecting the insertion and returning an error message to the user.
o inserting a tuple (“L-37”, “Perryridge”, null) into the loan relation
 Some updates through views are impossible to translate into
database relation updates.
o create view v as branch-name = “Perryridge” (account))
 v  v  (L-99, Downtown, 23)
 Others cannot be translated uniquely.
o all-customer  all-customer  {(“Perryridge”, “John”)}
 Have to choose loan or account,
and create a new loan/account
number!

Views Defined Using Other Views

 One view may be used in the expression defining another view.
 A view relation v1 is said to depend directly on a view relation v2
if
v2 is used in the expression defining v1.
 A view relation v1 is said to depend on view relation v2 if either
v1 depends directly to v2 or there is a path of dependencies from
v1 to v2
 A view relation v is said to be recursive if it depends on itself.
View Expansion
 A way to define the meaning of views is defined in terms of other views.
 Le
t
vie
w
v1
be
def
ine
d
by
an
ex
pre
ssi
on
e1
tha
t
ma
y
its
elf
co
nta
in
us
es
of
vie
w
rel
ati
on
s.
  View expansion of an expression repeats the following replacement
step:
NOTES
 repeat
Find any view relation vi in e1
Replace the view relation vi by the
expression defining vi
until no more view relations are present in e1
 As long as the view definitions are not recursive, this loop will terminate.

INTEGRITY CONSTRAINTS
Domain Constraints
 Integrity constraints guard against accidental damage to the database,
by ensuring that authorized changes to the database do not result in a
loss of data consistency.
 Domain constraints are the most elementary form of integrity constraint.
 They test values inserted in the database, and test queries to ensure that
the comparisons make sense.
 New domains can be created from existing data types
Referential Integrity
It ensures that a value that appears in one relation for a given set of
attributes also appears for a certain set of attributes in another relation.
Formal Definition
Let r1 (R1) and r2 (R2) be relations with primary keys K1 and K2 respectively.
The subset  of R2 is a foreign key referencing K1 in relation r1, if for every t2
in r2 there must be a tuple t1 in r1 such that t1[K1] = t2[].
Referential integrity constraint is also called subset dependency since its can be
written as
 (r2)  K1 (r1)

Referential Integrity in the E-R Model

 Consider relationship set R between entity sets E1 and E2. The
relational schema for R includes the primary keys K1 of E1 and K2 of
E2.
 Then K1 and K2 form foreign keys on the relational schemas for E1 and
E2 respectively.
 Weak entity sets are also a source of referential integrity constraints.
 For the relation schema for a weak entity set, must include the primary
key attributes of the entity set on which it depends.
CS 6302 110
NOTE Checking Referential Integrity on Database Modification
S The following tests must be made in order to preserve the following referential
integrity constraint:  (r2)  K (r1)

Insert. If a tuple t2 is inserted into r2, the system must ensure that there is a tuple t1
in r1 such that t1[K] = t2[]. That is

t2 []  K (r1)

Delete. If a tuple, t1 is deleted from r1, the system must compute the set of tuples in
r2 that reference t1:

 = t1[K] (r2)

If this set is not empty, the delete command is rejected as an error, or the
tuples that reference t1 must themselves be deleted (cascading deletions are
possible).

Update. There are two cases:

If a tuple t2 is updated in relation r2 and the update modifies values for foreign key
, then a test similar to the insert case is made.
Let t2’ denote the new value of tuple t2. The system must ensure that
t2’[]  K(r1)
 If a tuple t1 is updated in r1, and the update modifies values for the
primary key (K), then a test similar to the delete case is made.
 The system must compute  = t1[K] (r2) using the old value of t1 (the
value before the update is applied).
 If this set is not empty the update may be rejected as an error, or the
update may be cascaded to the tuples in the set, or the tuples in the set
may be deleted.

RELATIONAL ALGEBRA AND CALCULUS

Set Theoretic operations

• Set operators are binary and will only work on two relations or sets of
data.
• Can only be used on union compatible sets
R (A1, A2, …, AN) and S(B1, B2, …, BN) are union compatible
if: degree (R) = degree (S) = N
domain (Ai) = domain (Bi) for all i

111 CS 6302
 Sailors Reserves
NOTES
sid sname rating age sid bid day
22 Dustin 7 45.0 22 101 10/10/96
22 Dustin 7 45.0 58 103 11/12/96
31 Lubber 8 55.5 22 101 10/10/96
31 Lubber 8 55.5 58 103 11/12/96
58 Rusty 10 35.0 22 101 10/10/96
58 Rusty 10 35.0 58 103 11/12/96

Fig 3.17 Sailors and Reserves Table

Union (U)
Assuming R & S are union compatible.
 Union: R U S is the set of tuples in either R or S or both.
 Since it is a set, there are no duplicate
tuples Example :
SELECT S.sname
FROM Sailors S, Boats B, Reserves R
WHERE S.sid = R.sid and R.bid = B.bid and B.color =
‘red’ UNION
SELECT S.sname
FROM Sailors S, Boats B, Reserves R
WHERE S.sid = R.sid and R.bid = B.bid and B.color = ‘green’;

Intersection (n)
Assuming that R & S are union compatible:
 Intersection: R n S is the set of tuples in both R and S
 Note that R n S = S n
R Example :
SELECT S.sname
FROM Sailors S, Boats B, Reserves R
WHERE S.sid = R.sid and R.bid = B.bid and B.color =
‘red’ INTERSECT
SELECT S.sname
FROM Sailors S, Boats B, Reserves R
WHERE S.sid = R.sid and R.bid = B.bid and B.color = ‘green’;
CS 6302 110
CS 6302 112
NOTE Difference (-)
S • Difference: R – S is the set of tuples that appear in R but do not appear in S
• (R – S) ? (S – R)
Example :
SELECT S.sname
FROM Sailors S, Boats B, Reserves R
WHERE S.sid = R.sid and R.bid = B.bid and B.color =
‘red’ MINUS
SELECT S.sname
FROM Sailors S, Boats B, Reserves R
WHERE S.sid = R.sid and R.bid = B.bid and B.color = ‘green’;

Cartesian Product (X)

• Sets do not have to be union compatible (usually not)
• In general, the result of:
R(A1, A2, …, AN) X S(B1, B2, …, BM) is
Q (A1, A2, …, AN, B1, B2, …, BM)
• If R has C tuples and S has D tuples, the result is C*D tuples.
Cartesian Product
• Also called “cross product”
• Note that union, intersection and cross product are commutative and associative
Cartesian product
Relational Operations
• Developed specifically for relational databases
• Used to manipulate data in ways that set operations can’t
Selection Operation
• Used to select a subset of the tuples
• Selection is based on a “select condition”
• The selection condition is basically a filter
• Notation:
<condition>
(<Relation>)
σ
• To process a selection, we:
– look at each tuple
– see if we have a match (based on the condition)
• The degree of the result is the same as the degree of the relation | σ | = |
r(R) |
113 CS 6302
Example:
NOTES
Faculty (fnum, name, office, salary,
•
rank)
σ
salary > (Faculty)
• σ
27000 (Faculty)
rank = associate
• σ fnum = 34567 (Faculty)
• σ (Faculty)
(salary>26000) and (rank=associate)
• σ(salary<=26000) and (rank ! (Faculty)
• σ=associate)
(Faculty)
max(salary)

Project Operation ()

• A select filters out rows
• A project filters out columns
– Reduces data (columns) returned
– Reduces duplicate columns created by cross product (why?)
– Creates a new relation
• Notation:  (Relation)
<attribute list>
• The degree of the result is the number of attributes in the <attribute list>
of the project.
• |  | <= | r(R) |

Example:
Faculty (fnum, name, office, salary, rank)
•  name, office (Faculty)
•  fnum, (Faculty)

salary

Rename ()
• Used to give a name to the resulting relation
• Notation to make relational algebra easier to write and understand
• We can now use the resulting relation in another relational algebra expression.
• Notation:
<New Name>  <Relational Expression>
NOTE Example:
S Faculty (fnum, name, office, salary,
rank) Associates  σrank =
associate
(Faculty) Result   name
(Associates)

Join Operation
• Join is a commonly used sequence of operators.
– Take the Cartesian product of two relations.
– Select only related tuples.
– (Possibly) eliminate duplicate columns.
Example:
• Result  R ? dcode = code
S

Kinds of Joins
join: Ajoin with some condition specified
Equijoin: Ajoin where the only comparison operator used is “=“
Natural join
– It is an equijoin followed by the removal of duplicate (superfluous) column(s)
– A Natural join is denoted by (*).
– Standard definition requires that the columns used to join the tables have the
same name.
Size of a Natural Join
• If R contains tuples and S contains
tuples, then the size of R
S is
n R
nS ? <>

between 0 and nR*nS

• Join selectivity = Expected size of the result
nR * nS
• This value is used by the optimizer to estimate
the cost of the join.
Left Outer Join
H keep all of the tuples from the “left” relation
H join with the right relation
H pad the non-matching tuples with nulls
Right Outer Join
H Same as the left, but keep tuples from the “right” relation
Full Outer Join
NOTES
H Same as left, but keep all tuples from both relations.
TUPLE RELATIONAL CALCULUS
 A nonprocedural query language, where each query is of the form
 {t | P (t) }
 It is the set of all tuples t such that predicate P is true for t.
 t is a tuple variable, t[A] denotes the value of tuple t on attribute
A.
 t  r denotes that tuple t is in relation r.
 P is a formula similar to that of the predicate calculus.
3.9.1 Predicate Calculus Formula
1. Set of attributes and constants
2. Set of comparison operators: (e.g., , , , , , )
3. Set of connectives: and (), or (v)‚ not ()
4. Implication (): x  y, if x if true, then y is true
x  y  x v y
5. Set of quantifiers:
  t  r (Q(t))  ”there exists” a tuple in t in relation r
such that predicate Q(t) is true
 ◻t  r (Q(t))  Q is true “for all” tuples t in relation r
Banking Example
 branch (branch-name, branch-city, assets)
 customer (customer-name, customer-street,
customer- city)
 account (account-number, branch-name, balance)
 loan (loan-number, branch-name, amount)
 depositor (customer-name, account-number)
 borrower (customer-name, loan-number)

Example Queries

 Find the loan-number, branch-name, and amount for loans of over

$1200:
{t | t  loan  t [amount]  1200}
 Find the loan number for each loan of an amount greater than $1200:

{t |  s  loan (t[loan-number] = s[loan-number]  s [amount]  1200)}

Notice that a relation on schema [loan-number] is implicitly defined by the
query
NOTE  Find the names of all customers having a loan, an account, or both at the bank:
S {t | s  borrower( t[customer-name] = s[customer-name])
 u  depositor( t[customer-name] = u[customer-name])

 Find the names of all customers who have a loan and an account
at the bank:

{t | s  borrower( t[customer-name] = s[customer-name])

 u  depositor( t[customer-name] = u[customer-name])
 Find the names of all customers having a loan, an account, or both at
the Perryridge branch:

{ c  |  l ({ c, l   borrower
  b,a( l, b, a   loan  b = “Perryridge”))
  a( c, a   depositor
  b,n( a, b, n   account  b = “Perryridge”))}
 Find the names of all customers who have an account at
all branches located in Brooklyn:

{ c  |  s, n ( c, s, n   customer) 
 x,y,z( x, y, z   branch  y = “Brooklyn”) 
 a,b( x, y, z   account   c,a   depositor)}
 Find the names of all customers having a loan at the Perryridge branch:

{t | s  borrower(t[customer-name] = s[customer-name]
 u  loan(u[branch-name] = “Perryridge”
 u[loan-number] = s[loan-number]))}
 Find the names of all customers who have a loan at the
Perryridge branch, but no account at any branch of the
bank:
{t | s  borrower( t[customer-name] = s[customer-name]
 u  loan(u[branch-name] = “Perryridge”
 u[loan-number] = s[loan-number]))
 not v  depositor (v[customer-name] =
t[customer-name]) }
 Find the names of all customers having a loan from the Perryridge
branch, and the cities they live in:
{t | s  loan(s[branch-name] = “Perryridge”
 u  borrower (u[loan-number] = s[loan-number]
 t [customer-name] = u[customer-name])
  v  customer (u[customer-name] = v[customer-name]
 t[customer-city] = v[customer-city])))}
  Find the names of all customers who have an account at all branches
located in Brooklyn:
NOTES
{t |  c  customer (t[customer.name] = c[customer-name]) 
 s  branch(s [branch-city] = “Brooklyn” 
 u  account ( s [branch-name] = u [branch-name]
  s  depositor ( t [customer-name] = s [customer-name]
 s [account-number] = u[account-number] )) )}

Safety of Expressions

 It is possible to write tuple calculus expressions that generate infinite

rela- tions.
 For example, {t |  t  r} results in an infinite relation if the domain of any
attribute of relation r is infinite.
 To guard against the problem, we restrict the set of allowable expressions
to safe expressions.
 An expression {t | P(t)} in the tuple relational calculus is safe if
every component of t appears in one of the relations, tuples, or
constants that appear in P.
o NOTE: This is more than just a syntax condition.
 E.g. { t | t[A]=5  true } is not safe — it defines an
infinite set with attribute values that do not appear in any
relation or tuples or constants in P.

DOMAIN RELATIONAL CALCULUS

 A nonprocedural query language equivalent in power to the tuple
relational calculus
 Each query is an expression of the form:
 {  x1, x2, …, xn  | P(x1, x2, …, xn)}
o x1, x2, …, xn represent domain variables.
o P represents a formula similar to that of the predicate calculus.
 Find the loan-number, branch-name, and amount for loans of over
$1200
{ l, b, a  |  l, b, a   loan  a > 1200}
 Find the names of all customers who have a loan of over $1200

{ c  |  l, b, a ( c, l   borrower   l, b, a   loan  a > 1200)}

 Find the names of all customers who have a loan from the
Perryridge branch and the loan amount:

{ c, a  |  l ( c, l   borrower  b( l, b, a   loan 

b = “Perryridge”))}
or { c, a  |  l ( c, l   borrower   l, “Perryridge”, a   loan)}
NOTE  Find the names of all customers having a loan, an account, or both at the
S Perryridge branch:
{ c  |  l ({ c, l   borrower
  b,a( l, b, a   loan  b = “Perryridge”))
  a( c, a   depositor
  b,n( a, b, n   account  b = “Perryridge”))}
 Find the names of all customers who have an account at all
branches located in Brooklyn:

{ c  |  s, n ( c, s, n   customer) 
 x,y,z( x, y, z   branch  y = “Brooklyn”) 
 a,b( x, y, z   account   c,a   depositor)}

Safety of Expressions
{  x1, x2, …, xn  | P(x1, x2, …,
xn)} is safe if all of the following hold:
1. All values that appear in tuples of the expression are values from dom(P)
(that is, the values appear either in P or in a tuple of a relation mentioned
in P).
2. For every “there exists” subformula of the form  x (P1(x)), the
subformula is true if and only if there is a value of x in dom(P1) such
that P1(x) is true.
3. For every “for all” subformula of the form x (P1 (x)), the
subformula is true if and only if P1(x) is true for all values x from
dom (P1).
RELATIONAL DATABASE DESIGN
Functional Dependency is a particular relationship between two attributes.
It is a relation as for ever valid instance of one attribute , that value of
attribute uniquely determines the value of other attribute. For any relation
R , attribute A is functionally dependent on A. A B.
Functional Dependencies Definition

o Functional dependency (FD) is a constraint between two sets

of attributes from the database.
o A functional dependency is a property of the semantics
or meaning of the attributes.
In every relation R(A1, A2, …, An) there is a FD called
t
h
e

P
K

(
P
r
i
m
a
r
y

K
e
y
)

-
>

A
1
,

A
2
,

…
,

A
n
 Formally the FD is defined as follows
NOTES
o If X and Y are two sets of attributes, that are subsets of T.
For any two tuples t1 and t2 in r , if t1[X]=t2[X], we must
also have t1[Y]=t2[Y].
Notation:
o If the values of Y are determined by the values of X, then
it is denoted by X -> Y
o Given the value of one attribute, we can determine the value
of another attribute
X f.d. Y or X -> y

Example: Consider the following,

Student Number -> Address, Faculty Number -> Department, Depart-
ment Code -> Head of Dept
Goal — Devise a Theory for the Following
 Decide whether a particular relation R is in “good” form.
 In the case that a relation R is not in “good” form, decompose it into a
set of relations {R1, R2, ..., Rn} such that
o each relation is in good form.
o the decomposition is a lossless-join decomposition.
 Our theory is based on:
o functional dependencies
o multivalued dependencies
FUNCTIONAL DEPENDENCIES
o Constraints on the set of legal relations.
o Require that the value for a certain set of attributes
determines uniquely the value for another set of attributes.
o A functional dependency is a generalization of the notion of a key.
o The functional dependency
α
holds on R if and only if for any legal relations r(R), whenever any two
tuples and of r agree on the attributes , they also agree on the
t1 t2
attributes . That is,
 t1[] = t2 []  t1[ ] = t2 [ ]
CS 6302 120
NOTE o Example: Consider r(A,B) with the following instance of r.
S
1 4
1 5
3 7
o On this instance, A  B does NOT hold, but B  A does hold.

 K is a superkey for relation schema R if and only if K  R.

 K is a candidate key for R if and only if
o K  R, and
o for no   K,   R
 Functional dependencies allow us to express constraints that cannot
be expressed using superkeys. Consider the schema:
Loan-info-schema = (customer-name, loan-
number, branch-name, amount).
We expect this set of functional dependencies to hold:
loan-number 
amount loan-number 
branch-name
but would not expect the following to hold:
loan-number  customer-name
Use of Functional Dependencies
 We use functional dependencies to:
o test relations to see if they are legal under a given set of functional
dependencies.
o If a relation r is legal under a set F of functional dependencies,
we say that r satisfies F.
o specify constraints on the set of legal relations
o We say that F holds on R if all legal relations on R satisfy the
set of functional dependencies F.
 Note: A specific instance of a relation schema may satisfy a
functional dependency even if the functional dependency does not hold
on all legal in- stances.
o For example, a specific instance of Loan-schema may, by
chance, satisfy

121 CS 6302
 loan-number  customer-name.
NOTES
 A functional dependency is trivial if it is satisfied by all instances of a
relation.
o E.g.
o customer-name, loan-number  customer-name
o customer-name  customer-name
o In general,    is trivial if   
Closure of a Set of Functional Dependencies
 Given a set F set of functional dependencies, there are certain other
func- tional dependencies that are logically implied by F.
o E.g. If A  B and B  C, then we can infer that A  C
 The set of all functional dependencies logically implied by F is the
closure of
F.
 We denote the closure of F by F+.
 We can find all of F+ by applying Armstrong’s Axioms:
o if   , then    (reflexivity)
o if   , then      (augmentation)
o if   , and   , then    (transitivity)
 These rules are
o sound (generate only functional dependencies that actually hold)
and
o complete (generate all functional dependencies that hold).
Example
 R = (A, B, C, G, H,
I) F = { A  B
A 
C CG 
H CG 
I
B  H}

NORMALIZATION – NORMAL FORMS

o Introduced by Codd in 1972 as a way to “certify” that a relation
has a certain level of normalization by using a series of tests.
o It is a top-down approach, so it is considered relational design by analysis.
o It is based on the rule: one fact – one place
o It is the process of ensuring a schema design that is free of redundancy
Uses of Normalization
o Minimize redundancy
o Minimize update anomalies
CS 6302 122
NOTE o We use normal form tests to determine the level of normalization
S for the scheme.
Pitfalls in Relational Database Design
 Relational database design requires that we find a “good” collection
of relation schemas. A bad design may lead to
o Repetition of Information.
o Inability to represent certain information.
 Design Goals:
o Avoid redundant data.
o Ensure that relationships among attributes are represented.
o Facilitate the checking of updates for violation of database
integrity constraints.
 Example
o Consider the relation schema:
Lending-schema = (branch-name, branch-city,
assets, customer-name, loan-number,
amount)

Fig 3.18 Lending Schema

Redundancy:
 Data for branch-name, branch-city, assets are repeated for each
loan that a branch makes.
o Wastes space
o Complicates updating, introducing possibility of inconsistency of
assets value
o Null values
o Cannot store information about a branch if no loans exist
o Can use null values, but they are difficult to handle.

123 CS 6302
CS 6302 124
Decomposition
NOTES
 Decompose the relation schema Lending-schema into:
Branch-schema = (branch-name, branch-
city,assets) Loan-info-schema = (customer-name,
loan-number,
branch-name, amount)
 All attributes of an original schema (R) must appear in the
decomposition (R1, R2):
R = R1  R 2
 Lossless-join decomposition.
For all possible relations r on schema R
r = R1 (r) R2(r)
Example of Non Lossless-Join Decomposition
 Decomposition of R = (A, B)
R2 = (A) R2 = (B)

A B A B
 1  1
 2  2
 1
A(r) B(r)
r
A B
A (r) B (r)
 1
 2
 1
 2

Fig 3.19 Loss less join Operation

Normalization Using Functional Dependencies

 When we decompose a relation schema R with a set of functional
dependen- cies F into R1, R2,.., Rn we want
 Lossless-join decomposition: Otherwise decomposition would result in
information loss.
 No redundancy: The relations Ri preferably should be either in Boyce-
Codd Normal Form or in Third Normal Form.
 Dependency preservation: Let Fi be the set of dependencies F+ that include
only attributes in Ri.

125 CS 6302
NOTES o Preferably the decomposition should be dependency preserving,
that is, (F  F  …  F )+ = F+
1 2 n
 Otherwise, checking updates for violation of functional dependencies
may require computing joins, which is expensive.
Example
 R = (A, B, C)
F = {A  B, B  C)
 Can be decomposed in two different ways
o R1 = (A, B), R2 = (B, C)
 Lossless-join decomposition:
o R1  R2 = {B} and B  BC
 Dependency preserving
o R1 = (A, B), R2 = (A, C)
 Lossless-join decomposition:
o R1  R2 = {A} and A  AB
 Not dependency preserving
(cannot check B  C without computing R2)
1
R
Testing for Dependency Preservation
 To check if a dependency  is preserved in a decomposition of R
into R1, R2, …, Rn we apply the following simplified test (with attribute
closure done w.r.t. F)
 result = 
while (changes to result) do
for each Ri in the decomposition
t = (result  Ri)+  Ri
result = result  t
 If result contains all attributes in , then the functional dependency
   is preserved.
 We apply the test on all dependencies in F to check if a
decomposition is dependency preserving.
 This procedure takes polynomial time, instead of the exponential time
re- quired to compute F+ and (F1  F2  …  Fn)+ .
 TYPES OF NORMAL FORMS
NOTES
First Normal Form (1NF)

o Now part of the formal definition of a relation

o Attributes may only have atomic values (i.e. single values).
o Disallows “relations within relations” or “relations as attributes of tuples”
1NF

2NF

3NF

Boyce-Codd NF

4NF
5NF

Fig 3.20 Normal Forms

Second Normal Form (2NF)

o A relation is in 2NF if all of its non-key attributes are fully
dependent on the key.
o This is known as full functional dependency.
o When in 2NF, the removal of any attribute will break
the dependency
R (fnum, dcode, fname, frank, dname)
R1(fnum, fname, frank)
R2(dcode, dname)
Third Normal Form (3NF)
o A relation is in 3NF if it is in 2NF and has no
transitive dependencies.
o A transitive dependency is when X->Y and Y->Z implies X-
>Z Faculty (number, fname, frank, dname, doffice,
dphone) R1 (number, fname, frank, dname)
R2 (dname, doffice, dphone)
R (snum, cnum, dcode, s_term, s_slot, fnum,
c_title, c_description, f_rank, f_name, d_name,
d_phones)
The following is the 3NF of above schema,
1. snum, cnum, dcode -> s_term, s_slot, fnum, c_title, c_description,
NOTE f_rank, f_name, d_name, d_phones
S 2. dcode -> d_name
3. cnum, dcode -> c_title, c_description
4. cnum, dcode, snum -> s_term, s_slot, fnum, f_rank, f_name
5. fnum -> f_rank, f_name
Boyce Codd Normal Form (BCNF)
o It is based on FD that takes into account all candidate keys in a
relation.
o A relation is said to be in BCNF if and only if every determinant
is a candidate key.
o A determinant is an attribute or a group of attributes on which
some other attribute is fully functionally determinant.
o To test whether a relation is in BCNF, we identify all the
determinants and make sure that they are candidate keys.

FIRST NORMAL FORM

 Domain is atomic if its elements are considered to be indivisible units.
o Examples of non-atomic domains:
o Set of names, composite attributes
o Identification numbers like CS101 that can be broken up into parts
 A relational schema R is in first normal form if the domains of all
attributes of R are atomic.
 Non-atomic values complicate storage and encourage redundant
(repeated) storage of data.
o E.g. Set of accounts stored with each customer, and set of
owners stored with each account
o We assume all relations are in first normal form (revisit this
in Chapter 9 on Object Relational Databases).

 Atomicity is actually a property of how the elements of the domain are

used.
o E.g. Strings would normally be considered indivisible
o Suppose that students are given roll numbers which are strings of
the form CS0012 or EE1127
o If the first two characters are extracted to find the department,
the domain of roll numbers is not atomic.
o Doing so is a bad idea: leads to encoding of information
in application program rather than in the database.
 BOYCE-CODD NORMAL FORM c
    is trivial (i.e.,   ) o
  is a superkey for R m
Example p
 R = (A, B, u
C) F = {A  t
B e
B  C}
Key = {A} F
 R is not in BCNF +

 Decomposition R1 = (A, B), R = (B, C) ;

o R1 and R2 in BCNF while (not
o Lossless-join decomposition done) do
o Dependency preserving
Testing for BCNF
 To check if a non-trivial dependency   causes a violation
of BCNF
1. compute + (the attribute closure of ), and
2. verify whether it includes all attributes of R, that is it is a superkey
of
R.
 Simplified test: To check if a relation schema R is in BCNF, it
suffices to check only the dependencies in the given set F for
violation of BCNF, rather than checking all dependencies in F+.
 If none of the dependencies in F causes a violation of BCNF, then
none
of the dependencies in F+ will cause a violation of BCNF either.
 However, using only F is incorrect when testing a relation in
a decomposition of R
o E.g. Consider R (A, B, C, D), with F = { A B, B C}
o Decompose R into R1(A,B) and R2(A,C,D)
o Neither of the dependencies in F contain only attributes from
(A,C,D) so we might be mislead into thinking R2 satisfies
BCNF.
o In fact, dependency A  C in F+ shows R
2 is not in BCNF.
BCNF Decomposition Algorithm
result := {R};
done := false;
NOTES
NOTES if (there is a schema Ri in result that is not in BCNF)
then begin
let    be a nontrivial functional
dependency that holds on Ri
such that   R iis not in F+,
and    = ;
result := (result – Ri )  (R – )  (,  );
end
else done := true;
Note: each Ri is in BCNF, and decomposition is lossless-join.
Example of BCNF Decomposition
 R = (branch-name, branch-city, assets,
customer-name, loan-number, amount)
F = {branch-name  assets branch-
city loan-number  amount branch-
name} Key = {loan-number, customer-
name}
 Decomposition
R1 = (branch-name, branch-city, assets)
R2 = (branch-name, customer-name, loan-number, amount)
R3 = (branch-name, loan-number, amount)
R4 = (customer-name, loan-number)
 Final decomposition
R1, R3, R4
Testing Decomposition for BCNF
 To check if a relation Ri in a decomposition of R is in BCNF,
o Either test Ri for BCNF with respect to the restriction of F to
Ri (that is, all FDs in F+ that contain only attributes from
i
R)
o or use the original set of dependencies F that hold on R, but with
the following test:
o for every set of attributes   Ri, check that + (the attribute
closure of ) either includes no attribute of Ri- , or includes all
attributes of Ri.
o If the condition is violated by some    in F, the dependency
  (+ -  )  Ri
can be shown to hold on Ri, and Ri violates BCNF.
o We use above dependency to decompose Ri
 BCNF and Dependency Preservation
NOTES
 R = (J, K, L)
F = {JK 
L
L  K}
Two candidate keys = JK and JL
 R is not in BCNF
 Any decomposition of R will fail to preserve
JK  L

THIRD NORMAL FORM: MOTIVATION

 There are some situations where
o BCNF is not dependency preserving, and
o efficient checking for FD violation on updates is important.
 Solution: define a weaker normal form, called Third Normal Form.
o Allows some redundancy (with resultant problems; we will
see examples later)
o But FDs can be checked on individual relations without
computing a join.
o There is always a lossless-join, dependency-preserving
decomposi- tion into 3NF.
Third Normal Form
 A relation schema R is in third normal form (3NF) if for all:
   in F+
at least one of the following holds:
o    is trivial (i.e.,   )
o  is a superkey for R
o Each attribute A in  –  is contained in a candidate key
for R. (NOTE: each attribute may be in a different candidate key)
 If a relation is in BCNF it is in 3NF (since in BCNF one of the first
two conditions above must hold).
 Third condition is a minimal relaxation of BCNF to ensure
dependency preservation (will see why later).
 Example
R = (J, K, L)
F = {JK  L, L  K}
 Two candidate keys: JK and JL
131 CS 6302
NOTES  R is in 3NF
JK  L
 JK is a superkey
L  K K is contained in a candidate key
 BCNF decomposition has (JL) and (LK)
 Testing for JK  L requires a join
o There is some redundancy in this schema
o Equivalent to example in book:
Banker-schema = (branch-name, customer-name, banker-name)
banker-name  branch name
branch name customer-name  banker-name

Testing for 3NF

 Optimization: Need to check only FDs in F, need not check all FDs in F+.
 Use attribute closure to check for each dependency   , if  is a superkey.
 If  is not a superkey, we have to verify if each attribute in  is contained
in a candidate key of R.
o This test is rather more expensive, since it involve finding candidate keys.
o Testing for 3NF has been shown to be NP-hard.
o Interestingly, decomposition into third normal form (described shortly) can
be done in polynomial time.
3NF Decomposition Algorithm
Let Fc be a canonical cover for F;
i := 0;
for each functional dependency    in cF do
if none of the schemas Rj , 1  j  i contains  
then begin
i := i + 1;
Ri :=  
end
if none of the schemas R , 1  j  i contains a candidate key for R
j
then begin
i := i + 1;
Ri := any candidate key for R;
end
return (R1, R2, ...,
Ri)

CS 6302 130
 Above algorithm ensures:
NOTES
o Each relation schema Ri is in 3NF
o Decomposition is dependency preserving and lossless-
join.
o Proof of correctness is at end of this file (click here)
Example
 Relation schema:
Banker-info-schema = (branch-name, customer-
name, banker-name, office-number)
 The functional dependencies for this relation schema
are: banker-name  branch-name office-
number customer-name branch-name 
banker-name
 The key is:
{customer-name, branch-name}

Applying 3NF to Banker-info-schema

 The for loop in the algorithm causes us to include the following
schemas in our decomposition:
Banker-office-schema = (banker-name, branch-name,
office-number)
Banker-schema = (customer-name, branch-name,
banker-name)

 Since Banker-schema contains a candidate key for

Banker-info-schema, we are done with the decomposition process.
Comparison of BCNF and 3NF
 It is always possible to decompose a relation into relations in 3NF; and
o The decomposition is lossless.
o The dependencies are preserved.
 It is always possible to decompose a relation into relations in BCNF; and
o The decomposition is lossless.
o It may not be possible to preserve dependencies.
 Example of problems due to redundancy in 3NF
R = (J, K, L)
F = {JK  L, L  K}
CS 6302 132
NOTE J L K
S j1 l1 k1

j2 l1 k1

j3 l1 k1

null l2 k2

Fig 3.21 Redundancy in 3NF

DESIGN GOALS OF 4NF

 Goal for a relational database design is:
o BCNF.
o Lossless join.
o Dependency preservation.
 If we cannot achieve this, we accept
o Lack of dependency preservation or
o Redundancy due to use of 3NF
 Interestingly, SQL does not provide a direct way of specifying
functional dependencies other than superkeys.
It can specify FDs using assertions, but they are expensive to test.
 Even if we had a dependency preserving decomposition, using SQL we
would not be able to efficiently test a functional dependency whose left hand
side is not a key.
Testing for FDs Across Relations
 If decomposition is not dependency preserving, we can have an extra
materialized view for each dependency   in Fc that is not preserved in
the decomposition.
 The materialized view is defined as a projection on   of the join of the
relations in the decomposition.
 Many newer database systems support materialized views and database
system maintains the view when the relations are updated.
 No extra coding effort for programmer.
 The functional dependency    is expressed by declaring  as a
candidate key on the materialized view.
 Checking for candidate key is cheaper than checking   

133 CS 6302
CS 6302 134
 BUT: tuples
 Space overhead: for storing the materialized view need to
be
 Time overhead: Need to keep materialized view up to date inserted.
when relations are updated (datab
 Database system may not support key declarations as
on materialized views. e,
Multivalued Dependencies Sa
 There are database schemas in BCNF that do not seem to be ra,
sufficiently normalized. D
 Consider a database B
o classes(course, teacher, book) C
such that (c,t,b)  classes means that t is qualified to teach c, and on
b
ce
is a required textbook for c
pt
 The database is supposed to list for each course the set of teachers any
s)
one of which can be the course’s instructor, and the set of books, all of
(d
which are required for the course (no matter who teaches it).
at
course ab
as
teacher book e,
database Avi DB Concepts Sa
database Avi Ullman
Hank DB Concepts ra,
database
database Hank Ullman Ul
database Sudarshan DB Concepts lm
database Sudarshan Ullman
an
operating systems Avi OS Concepts
Avi Shaw )
operating systems
operating systems Jim OS Concepts  Therefor
operating systems Jim Shaw e, it is
better to
decomp
ose
classes
into:
Fig 3.22 Multi valued Dependencies
 There are no non-trivial functional dependencies and therefore the
relation is in BCNF.
 Insertion anomalies – i.e. if Sara is a new teacher who can teach database,
two
135 CS 6302
NOTES

CS 6302 136
NOTES course teacher
database Avi
database Hank
database Sudarshan
operating systems Avi
operating systems Jim

teaches

course book
database DB Concepts
database Ullman
operating systems OS Concepts
operating systems Shaw
text
Fig 3.23 Decomposed Tables

We shall see that these two relations are in Fourth Normal Form (4NF).

Multivalued Dependencies (MVDs)

Let R be a relation schema and let   R and   R. The multivalued depen-

dency
  
holds on R if in any legal relation r(R), for all pairs for tuples t1 and t2 in r

137 CS 6302
 such that t1 [] = t2 [], there exist tuples 3t and t in r such that:
t1[] = t2 [] = t3 [] t4 []
t3[] = t1 []
t3[R – ] = t2[R – ]
t4 [] = t2[]
t4[R – ] = t1[R – ]
Let us see Tabular representation of   

CS 6302 138
NOTES
 Fig 3.24 Tabular representation of   
Example
 Let R be a relation schema with a set of attributes that are partitioned
into 3 nonempty subsets.
Y, Z, W
 We say that Y  Z (Y multidetermines Z)
if and only if for all possible relations r(R)
< y , z , w >  r and < y , z , > r
w
1 1 1 2 2 2
then
< y , z , w >  r and < y , z , w > r
1 1 2 2 2 1
 Note that since the behavior of Z and W are identical it follows that Y  Z if Y
 W

 In our example:
course  teacher
course  book
 The above formal definition is supposed to formalize the notion that given a
particular value of Y (course) it has associated with a set of values of Z
(teacher) and a set of values of W (book), and these two sets are in some
sense indepen- dent of each other.
Note:
o If Y  Z then Y  Z
o Indeed we have (in above notation) Z1 = Z2
The claim follows.

Use of Multivalued Dependencies

We use multivalued dependencies in two ways:
1. To test relations to determine whether they are legal under a given set
of functional and multivalued dependencies
2. To specify constraints on the set of legal relations. We shall thus concern
NOTE ourselves only with relations that satisfy a given set of functional and multivalued
S dependencies.
If a relation r fails to satisfy a given multivalued dependency, we can construct a
relations r that does satisfy the multivalued dependency by adding tuples to r.

Theory of MVDs
 From the definition of multivalued dependency, we can derive the following rule:
o If   , then   
That is, every functional dependency is also a multivalued dependency.
 The closure D+ of D is the set of all functional and multivalued
dependencies logically implied by D.
o We can compute D+ from D, using the formal definitions of functional
depen- dencies and multivalued dependencies.
o We can manage with such reasoning for very simple multivalued
dependen- cies, which seem to be most common in practice.
o For complex dependencies, it is better to reason about sets of
dependencies using a system of inference rules .
Fourth Normal Form
 A relation schema R is in 4NF with respect to a set D of functional and
multivalued dependencies if for all multivalued dependencies in D+ of the
form 
 , where   R and   R, at least one of the following hold:
o    is trivial (i.e.,    or    = R)
o  is a superkey for schema R
 If a relation is in 4NF it is in BCNF.
Restriction of Multivalued Dependencies
 The restriction of D to R is the set D consisting of
i i
o All functional dependencies in D+ that include only attributes ofi R
o All multivalued dependencies of the form
  (  R i )
where   Ri and    is in D+
4NF Decomposition Algorithm
result: = {R};
done := false;
compute D+;
Let D denote the restriction of D+ to R
i i
 while (not done)
NOTES
if (there is a schema Ri in result that is not in 4NF)
then begin
let    be a nontrivial multivalued dependency that holds
on R i such that   Ri is not in Di , and ;
result := (result - R )  i
(R i
- )  ( , );
end
else done:= true;
Note: each Ri is in 4NF, and decomposition is lossless-join
Example
 R =(A, B, C, G, H, I)
F ={ A  B
B  HI
CG  H }
 R is not in 4NF since A  B and A is not a superkey for R.
 Decomposition
a) R1 = (A, B) (R1 is in 4NF)
b) R2 = (A, C, G, H, I) (R2 is not in 4NF)
c) R3 = (C, G, H) (R3 is in 4NF)
d) R4 = (A, C, G, I) (R4 is not in 4NF)
 Since A  B and B  HI, A  HI, A  I
e) R5 = (A, I) (R5 is in 4NF)
f)R6 = (A, C, G) (R6 is in 4NF)

FURTHER NORMAL FORMS

o Join dependencies generalize multivalued dependencies.
 It leads to project-join normal form (PJNF) (also called
fifth normal form)
o A class of even more general constraints, leads to a normal form called
domain-key normal form.
o Problem with these generalized constraints are hard to reason with,
and no set of sound and complete set of inference rules exists.
o Hence it is rarely used.
Overall Database Design Process
 We have assumed schema R is given
o R could have been generated when converting E-R diagram to a
set of tables.
o R could have been a single relation containing all attributes that
are of interest (called universal relation).
NOTE o Normalization breaks R into smaller relations.
S o R could have been the result of some ad hoc design of
relations, which we then test/convert to normal form.
ER Model and Normalization
 When an E-R diagram is carefully designed, identifying all entities
correctly, the tables generated from the E-R diagram should not need
further normal- ization.
 However, in a real (imperfect) design there can be FDs from non-
key attributes of an entity to other attributes of the entity.
o E.g. employee entity with attributes department-number and
department-address, and an FD department-number 
depart- ment-address
 Good design would have made department an entity.
 FDs from non-key attributes of a relationship set possible, but rare —
most relationships are binary.
Universal Relation Approach
 Dangling tuples – Tuples that “disappear” in computing a join.
o Let r1 (R1), r2 (R2), …., rn (Rn) be a set of relations.
o A tuple r of the relation ri is a dangling tuple if r is not in the relation:
 Ri (r1 r2 … rn)
o The relation r1 r2 … rn is called a universal relation since it involves
all the attributes in the “universe” defined by
R1  R2  …  Rn
o If dangling tuples are allowed in the database, instead of decomposing
a universal relation, we may prefer to synthesize a collection of normal
form schemas from a given set of attributes.
 Dangling tuples may occur in practical database applications.
 They represent incomplete information.
 E.g. may want to break up information about loans into:
(branch-name, loan-number)
(loan-number, amount)
(loan-number, customer-name)
 Universal relation would require null values, and have dangling tuples.
 A particular decomposition defines a restricted form of incomplete
information that is acceptable in our database.
o Above decomposition requires at least one of customer-name,branch-
name or amount in order to enter a loan number without using null
values.
 o Rules out storing of customer-name, amount without an appropriate
NOTES
loan- number (since it is a key, it can’t be null either!).
 Universal relation requires unique attribute names unique role assumption.
o e.g. customer-name, branch-name
 Reuse of attribute names is natural in SQL since relation names can be
prefixed to disambiguate names.
Denormalization for Performance
 May want to use non-normalized schema for performance
 E.g. displaying customer-name along with account-number and balance
requires join of account with depositor.
 Alternative 1: Use denormalized relation containing attributes of account as
well as depositor with all above attributes.
o faster lookup
o Extra space and extra execution time for updates
o extra coding work for programmer and possibility of error in extra code
 Alternative 2: use a materialized view defined
as account depositor
o Benefits and drawbacks same as above, except no extra coding work for
programmer and avoids possible errors
Other Design Issues
 Some aspects of database design are not caught by normalization.
 Examples of bad database design (to be avoided):
Instead of earnings(company-id, year, amount), use
o earnings-2000, earnings-2001, earnings-2002, etc., all on the
schema (company-id, earnings).
o Above are in BCNF, but make querying across years difficult and
needs new table each year
o company-year(company-id, earnings-2000, earnings-2001, earnings-
2002)
o Also in BCNF, but also makes querying across years difficult
and requires new attribute each year.
o It is an example of a crosstab, where values for one attribute
become column names
o It is used in spreadsheets, and in data analysis tools
141 CS 6302
NOTE A COMPLETE EXAMPLE OFA BANKING DOMAIN
S

Fig 3.25 Sample lending Relation

Fig 3.26 Sample Relation r Fig 3.27 The customer Relation

Fig 3.28 The loan Relation Fig 3.29 The branch Relation

CS 6302 140
 NOTES

Fig 3.30 The Relation branch-customer

Fig 3.31 The Relation customer-loan

Fig 3.32 The Relation branch-customer customer-loan

141 CS 6302
CS 6302 142
NOTE
S

Fig 3.33 An Instance of Banker-schema

Fig 3.34 Tabular Representation of   

Fig 3.35 Relation bc: An Example of Reduncy in a BCNF Relation

Fig 3.36 An Illegal bc Relation

Fig 3.37 Decomposition of loan-info

143 CS 6302
CS 6302 144
Key words:
NOTES
Data model, schema, Relational Model ,Instance, Top down, Bottom up, anamoly,
null value, tuple, attribute, select , project, union, set difference, cartesian-
product, composition, rename, intersection, join, division, assignment, aggregate,
insertion, update, deletion, sql, group, view, integrity constraints, normalization,
dependency.

Short Answers :
1. Define Data model.
2. Define Schema.
3. Define Relational model.
4. Write a short on the following.
a. Two design approches
b. Informal Measures for Design
c. Modifications of the Database
d. BASIC QUERIES USING SINGLE ROW FUNCTIONS
e. COMPLEX QUERIES USING GROUP FUNCTIONS
f. INTEGRITY CONSTRAINTS
g. Functional Dependencies
h. Multivalued Dependencies
Answer in Detail :
1. Explain Basic Structure of Relational Model.
2. Explain RelationalAlgebra Operations.
3. Explain Formal Definition of RelationalAlgebra.
4. Explain Aggregate Functions and Operations.
5. Explain STRUCTURED QUERY LANGUAGE.
6. Explain VIEWS & View Operations.
7. Explain RELATIONAL ALGEBRAAND CALCULUS Operations.
8. Explain Tuple Relational Calculus.
9. Explain Domain Relational Calculus.
10. Explain Relational Database Design.
11. Explain Normalization – Types of Normal forms.
Summary :
 Relation is a tabular representation of data.
 Simple and intuitive, currently the most widely used.
 Integrity constraints can be specified by the DBA, based on
application semantics. DBMS checks for violations.
 Two important ICs: primary and foreign keys
 In addition, we always have domain constraints.
145 CS 6302
CS 6302 146
NOTE  Powerful and natural query languages exist.
 Rules to translate ER to relational model
S
o The relational model has rigorously defined query languages that
are simple and powerful.
o Relational algebra is more operational; useful as internal
representa- tion for query evaluation plans.
o Several ways of expressing a given query; a query optimizer
should choose the most efficient version.
o Relational calculus is non-operational, and users define queries
in terms of what they want, not in terms of how to compute it.
 (Declarativeness.)
o Algebra and safe calculus have same expressive power, leading
to the notion of relational completeness.
o SQL was an important factor in the early acceptance of the
relational model; more natural than earlier, procedural query
languages.
o Relationally complete; in fact, significantly more expressive
power than relational algebra.
o Even queries that can be expressed in RA can often be
expressed more naturally in SQL.
o Many alternative ways to write a query; optimizer should look
for most efficient evaluation plan.
 In practice, users need to be aware of how queries are optimized
and evaluated for best results.
o NULL for unknown field values brings many complications.
o SQL allows specification of rich integrity constraints.
o Triggers respond to changes in the database.
References :
1. Ramez Elamasri and shankant B.Navathe, “ Fundamentals
of Database Systems”, Third Edition , Pearson Education,
Delhi, 2002.
2. Abraham Silberschatz, Henry F.Korth and S.Sundarshan ,
“Database System concepts “, Fourth Edition ,
McGrawHill, 2002.
3. C.J.Date,”An Introduction to Database
Systems”,Seventh Edition,Pearson Education,Delhi,
2002.

147 CS 6302
 NOTES

QUERY AND TRANSACTION

PROCESSING
Learning Objectives
 To Learn about Query Execution Operations
 To learn to use hermistics in Query Operations
 To Study how to estimate cost of a query
 To learn about semantic query optimizer
 To learn about Transaction Processing
 To Learn about properties of Transactions
 To learn about serializability and Transaction support in SQL

INTRODUCTION
Basic Principles of Query Execution
o Many DB operations require reading tuples, tuple vs. previous tuples,
or tuples vs. tuples in another table.
o Techniques generally used for implementing operations:
 Iteration: for/while loop comparing with all tuples on disk
 Index: if comparison of attribute that’s indexed, look up matches
in index & return those
 Sort/merge: iteration against presorted data (interesting orders)
 Hash: build hash table of the tuple list, probe the hash table
 Must be able to support larger-than-memory data
NOTE Query Processing
S Basic Steps in Query Processing
n Parsing and translation
 Translate the query into its internal form. It
is then translated into relational algebra.
 Parser checks syntax and verifies relations.

Fig 4.1 Query Processing

n Evaluation
 The query-execution engine takes a
query-evaluation plan, executes that plan, and returns the answers to the query.
n Optimizer
The selection of optimal execution plan

Optimization
n A relational algebra expression may have many equivalent expressions
 E.g.,  ( (account)) is equivalent to

balance 2500 balance
 (  (account))
balance balance 2500
n Each relational algebra operation can be evaluated using one of several different
algorithms
 Correspondingly, a relational-algebra expression can be evaluated in
many ways.
n Annotated expression specifying detailed evaluation strategy is called an
evaluation-plan.
 E.g., can use an index on balance to find accounts with balance < 2500,
 or can perform complete relation scan and discard accounts with balance 
2500
n Query Optimization: Amongst all equivalent evaluation plans choose the

one with lowest cost.

  Cost is estimated using statistical information from
NOTES
the database catalog
è e.g. number of tuples in each relation, size of tuples, etc.
Measures of Query Cost
n Cost is generally measured as total elapsed time for answering query.

 Many factors contribute to time cost.

è disk accesses, CPU, or even network communication
n Typically disk access is the predominant cost, and is also relatively easy to

estimate. It is measured by taking into account

 Number of seeks * average-seek-cost
 Number of blocks read * average-block-read-cost
 Number of blocks written * average-block-write-cost
è Cost to write a block is greater than cost to read a block
– data is read back after being written to ensure that the write is successful.
n Real systems take CPU cost into account, differentiate between sequential and
random I/O, and take buffer size into account.

Selection Operation
n File scan – search algorithms that locate and retrieve records that fulfill a

selection condition.
n Algorithm A1 (linear search). Scan each file block and test all records to see

whether they satisfy the selection condition.

 Cost estimate (number of disk blocks scanned) = br
è br denotes number of blocks containing records from relation
r.
 If selection is on a key attribute, cost = (br /2)
è stop on finding record
 Linear search can be applied regardless of
è selection condition or
è ordering of records in the file, or
è availability of indices
n A2 (binary search). Applicable if selection is an equality comparison on the

attribute on which file is ordered.

 Assume that the blocks of a relation are stored contiguously.
 Cost estimate (number of disk blocks to be scanned):
NOTE è log2 (b ) — cost of locating the first tuple by a binary
S search on the blocks
è Plus number of blocks containing records that
satisfy selection condition
Selections Using Indices
n Index scan – search algorithms that use an index

 selection condition must be on search-key of index.

n A3 (primary index on candidate key, equality). Retrieve a single record that

satisfies the corresponding equality condition.

 Cost = HTi + 1
n A4 (primary index on nonkey, equality) Retrieve multiple records.
 Records will be on consecutive blocks.
 Cost = HTi + number of blocks containing retrieved records
n A5 (equality on search-key of secondary index).
 Retrieve a single record if the search-key is a candidate key.
è Cost = HTi + 1
 Retrieve multiple records if search-key is not a candidate key.
è Cost = HTi + number of records retrieved
– Can be very expensive!
è Each record may be on a different block.
– one block access for each retrieved record

Selections Involving Comparisons

n Can implement selections of the form   (r) or   (r) by using
A V AV
 a linear file scan or binary search,
 or by using indices in the following ways:
n A6 (primary index, comparison). (Relation is sorted on A)
è For A V (r) use index to find first tuple  v and scan relation
sequentially from there
è For A V (r) just scan relation sequentially till first tuple > v;
do not use index
n A7 (secondary index, comparison).
è For A V (r) use index to find first index entry  v and scan
index sequentially from there, to find pointers to records.
è For AV (r) just scan leaf pages of index finding pointers
to records, till first entry > v
 In either case, retrieve records that are pointed to.
è
NOTES
– Requires an I/O for each record.
– Linear file scan may be cheaper if many records are to be fetched!
Implementation of Complex Selections
n Conjunction:    . . .  (r)
1 2 n
n A8 (conjunctive selection using one index).
 Select a combination of  i and algorithms A1 through A7 that results in the
least cost fori (r).
 Test other conditions on tuple after fetching it into memory buffer.
n A9 (conjunctive selection using multiple-key index).

 Use appropriate composite (multiple-key) index if available.

n A10 (conjunctive selection by intersection of identifiers).
 Requires indices with record pointers.
 Use corresponding index for each condition, and take intersection of all
the obtained sets of record pointers.
 Then fetch records from file.
 If some conditions do not have appropriate indices, apply test in memory.
Basic Query Operators
o One-pass operators:
o Scan
o Select
o Project
o Multi-pass operators:
o Join
o Various implementations
 Handling of larger-than-memory sources
 Semi-join
 Aggregation, union,
etc. The examples are provided as follows
1-Pass Operators: Scanning a Table
o Sequential scan: read through blocks of table
o Index scan: retrieve tuples using “data entries” in index
order May require 1 seek per tuple! When?
o Cost in page reads – b(T) blocks, r(T) tuples
o b(T) pages for sequential scan
151 CS 6302
NOTE o Up to r(T) for index scan if unclustered index
S o Requires memory for one block
1-Pass Operators:
Select (?)
 Typically done while scanning a file
 If unsorted & no index, check against predicate:
Read tuple
While tuple exists
If tuple meets predicate
Return tuple
Read next tuple
 Sorted data: can stop after particular value encountered
 Indexed data: apply predicate to index, if possible
 If predicate is:
 conjunction: may use indexes and/or scanning loop above (may need to
sort/hash to compute intersection)
 disjunction: may use union of index results, or scanning loop
Project (?)
o Simple scanning method often used if no index:
Read tuple
While tuple exists
Output specified
attributes Read tuple
o Duplicate elimination may be necessary.
o Partition output into separate files by bucket, do duplicate removal on those.
o If have many duplicates, sorting may be better.
o If attributes belong to an index, don’t need to retrieve tuples!
Multi-pass Operators:
Join (?) – Nested-Loops Join
o Requires two nested loops:
o For each tuple in outer relation
For each tuple in inner, compare
If match on join attribute,
output
o Results have order of outer relation
o Can use indices

CS 6302 150
151 CS 6302
 o Very simple to implement, supports any joins predicates
NOTES
o Supports any join predicates
o Cost: # comparisons = t(R) t(S)
# disk accesses = b(R) + t(R) b(S)
Index Nested-Loops Join
For each tuple in outer relation
For each match in inner’s index
Retrieve inner tuple + output joined tuple
Cost: b(R) + t(R) * cost of matching in S
 For each R tuple, costs of probing index are about:
o 1.2 for hash index, 2-4 for B+-tree and:
o Clustered index: 1 I/O on average
o Unclustered index: Up to 1 I/O per S tuple
Two-Pass Algorithms
It is different from one pass algorithm since it executes in two phases.
Sort-based
o Need to do a multiway sort first (or have an index)
o Approximately linear in practice, 2 b(T) for table T
Hash-based
Store one relation in a hash table
(Sort-)Merge Join
o Requires data sorted by join attributes
Merge and join sorted files, reading sequentially a block at a
time
 Maintain two file pointers
o While tuple at R < tuple at S, advance R (and vice versa)
o While tuples match, output all possible pairings
 Preserves sorted order of “outer” relation
 Very efficient for presorted data
 Can be “hybridized” with NL Join for range joins
 May require a sort before (adds cost + delay)
 Cost: b(R) + b(S) plus sort costs, if necessary
In practice, approximately linear, 3 (b(R) +
b(S))
Hash-Based Joins
 Allows partial pipelining of operations with equality comparisons
 Sort-based operations block, but allow range and inequality comparisons
CS 6302 150
CS 6302 152
NOTE  Hash joins usually done with static number of hash buckets
S  Generally have fairly long chains at each bucket
 What happens when memory is too small?
Hash Join
 Read entire inner relation into hash table (join attributes as key)
 For each tuple from outer, look up in hash table & join
 Very efficient, very good for databases
 Not fully pipelined
 Supports equijoins only
 Delay-sensitive
Other Operators
 Duplicate removal very similar to grouping
 All attributes must match
 No aggregate
 Union, difference, intersection:
o Read table R, build hash/search tree
o Read table S, add/discard tuples as required
 Cost: b(R) + b(S)
OVERVIEW OF COST ESTIMATION & QUERY OPTIMIZATION
A query plan: algebraic tree of operators, with choice of algorithm for each op
Two main issues in optimization:
 For a given query, which possible plans are considered?
o Algorithm to search plan space for cheapest (estimated) plan
 How is the cost of a plan estimated?
o Ideally: Want to find best plan
o Practically: Avoid worst plans!
The System-R Optimizer: Establishing the
Basic Model
 Most widely used model; works well for < 10 joins
 Cost estimation: Approximate art at best
o Statistics, maintained in system catalogs, used to estimate cost
of operations and result sizes
o Considers combination of CPU and I/O costs
 Plan Space: Too large, must be pruned
o Only the space of left-deep plans is considered.

153 CS 6302
CS 6302 154
 o Left-deep plans allow output of each operator to be pipelined into
NOTES
the next operator without storing it in a temporary relation.
o Cartesian products are avoided
Schema for Examples
o Reserves:
 Each tuple is 40 bytes long, 100 tuples per page, 1000 pages.
o Sailors:
 Each tuple is 50 bytes long, 80 tuples per page, 500 pages.
Query Blocks: Units of Optimization
 An SQL query is parsed into a collection of query blocks, and
these are optimized one block at a time.
 Nested blocks are usually treated as calls to a subroutine,
made once per outer tuple.
 RelationalAlgebra Equivalences
 Allow us to choose different join orders and to ‘push’ selections
and projections ahead of joins.
Selections:
 More Equivalences
 A projection commutes with a selection that only uses attributes retained
by the projection.
Selection between attributes of the two arguments of a cross-
product converts cross-product to a join.
 A selection on ONLY attributes of R commutes with R ? S:
(R ? S)  (R) ? S
 If a projection follows a join R ? S, we can “push” it by retaining
only attributes of R (and S) that are needed for the join or are
kept by the projection
Enumeration of Alternative Plans
 There are two main cases:
o Single-relation plans
o Multiple-relation plans
 For queries over a single relation, queries consist of a combination of
selects, projects, and aggregate ops:
o Each available access path (file scan / index) is considered, and the
one with the least estimated cost is chosen.

155 CS 6302
CS 6302 156
 NOTE  The different operations are essentially carried out together (e.g., if an index is
S used for a selection, projection is done for each retrieved tuple, and the
resulting tuples are pipelined into the aggregate computation).
Cost Estimation
For each plan considered, must estimate cost:
 Must estimate cost of each operation in plan tree.
o Depends on input cardinalities.
 Must also estimate size of result for each operation in tree!
o Use information about the input relations.
 For selections and joins, assume independence of predicates.
 Assume can calculate a “reduction factor” (RF) for each selection predicate.
Estimates for Single-Relation Plans
 Index I on primary key matches selection:
o Cost is Height(I)+1 for a B+ tree, about 1.2 for hash index.
 Clustered index I matching one or more selects:
o (NPages(I)+NPages(R)) * product of RF’s of matching selects.
 Non-clustered index I matching one or more selects:
o (NPages(I)+NTuples(R)) * product of RF’s of matching selects.
 Sequential scan of file:
o NPages(R).
Example
 Given an index I on rating: (RF= 1/NKeys(I)
o (1/NKeys(I)) * NTuples(R) = (1/10) * 40000 tuples retrieved
 Clustered index: (1/NKeys(I)) * (NPages(I)+NPages(R)) = (1/10)
* (50+500) pages are retrieved

Unclustered index: (1/NKeys(I)) * (NPages(I)+NTuples(R)) = (1/10)
* (50+40000) pages are retrieved
 Given an index on sid:
o Would have to retrieve all tuples/pages. With a clustered index,
the cost is 50+500, with unclustered index, 50+40000
 A simple sequential scan:
o We retrieve all file pages (500)
SEMANTIC QUERY OPTIMIZATION
 Remember a relational query does not specify how the system should
compute the result, but it should also compute a meaningful result, that is
why

157 CS 6302
 we call it semantic query optimizer.
NOTES
 Legacy systems did and therefore provided better performance -
Relational systems use query optimizer.
 Modern optimizors are relatively sophisticated there for may provide
better performance than if user was to define strategies
 Optimizer has two tasks:
o Logical Optimization : generates a sequence of relational
algebra operations which will solve the query
o Physical Optimization : determines an efficient means of carrying
out each operation.
Optimization
 Automatic query optimisation should have:
o cardinality of each domain
o cardinality of each table
o number of values in each column
o number of times each different value occurs in each
column etc
 The optimizer:
o rate assessment of the efficiency of any strategy and choose the
most efficient
o be able to try many different strategies
o has had lots of research.
o can use query to see query execution plan in some databases.
System Tables - System Catalogue eg db2
o 20-30 tables of systems information on
o base tables, views, applications, users, authorization,
application plans etc
 systables: has one row for each base table in the
database, base table name, creator number of
columns etc.
 syscolumns: one row for each column of each relation in the
database column name, base table, data type etc.
 sysindexes: one row for every index in the database
index name, base table name, creator name etc.
 can be accessed by SQL statements during development of an application.
 e.g Select * from syscolumns where tablename = ‘Employee’;
NOTE Simple optimization Plan
S
supplier supplies parts
n m
Fig 4.2 Supplier Parts Relationship

supplier ( S# integer not null, Sname char (30), Saddress char(60),.......)

cardinality = 100
supplies-parts ( S# integer, P# integer)
cardinality = 10,000
part ( P# integer not null, Pname char(20) - not needed )
Select Distinct S.Sname
from S, SP
where S.S# = SP.S#
and SP.P# = ‘p2’.
50 parts are for p2 therefore the output will be only 50 tuples
READ tuple from disk to memory 50
microseconds RESTRICT (select function per tuple) 20
microseconds
JOIN (per tuple) 30
microseconds PROJECT (per tuple ) 10 microseconds
Answer:
Unoptimised Optimised
read S & SP read SP only
join S to SP restrict for ‘p2’
restrict for ‘p2’ read S only
project S.Sname join S with SP’
project S.Sname
Total time
 SQL Optimization
o Query optimization, access path selection, compilation
 Access path selection
o based on statistics accumulated
o cost of an access path is the weighted sum of calculated IO +
CPU cost
 IO cost : function of the estimated number of pages read for a query
 CPU cost: function of the estimated number of calls to the DB2 component
 that retrieves and filters the data
NOTES
o least expensive path is selected
o if sort, the cost of hashing an index to satisfy the sort order is
compared with the least cost of retrieving the table plus the cost
of sorting the rows - choose cheapest
 for a query involving joins, the order of joins significantly affects
the performance of the query.
Optimisation Process 4
stages
o cast query into some internal representation
o build a query tree
– convert to canonical form
o reduce query tree no matter how it was written

choose candidate low-level procedures

see if there are indexes etc available
look at statistics available
decide on the actual procedure in the DBMS which suits
this processing
o generate query plans and choose cheapest
 many plans may be produced....
 evaluation is expensive in its own right
 may take non-DBMS traffic into account
Tips
:
o apply selections as early as possible
o use nested selection in preference to join where possible
o (only use join when attributes from > 1 table are required in result)
o perform projections as late as possible
o retain duplicate tuples unless they need to be removed (remove in last result)
• Physical optimization
o if one of the relations can be held in memory - it is read only once from
disk then the larger is pulled in tuple by tuple and the complete smaller
relation’s tuples are tested for match (using Nested Loop Join)
o More difficult if both relations large,
• if one relation has an index, each tuple from the non indexed relation
NOTE is retrieved and compared with each tuple in the indexed relation (which
S uses the indexes to access the tuples directly + is efficient if index fits into
memory)
Database Performance
 In DB2 the aim is to provide maximum concurrency with minimal
o number of instructions executed by CPU
o number of I/O operations
o time to perform I/O operation
o failure due to resource overcommitment
 To achieve this = 4 aspects
o SQL Optimization
o optimized internal design of DB2
o facility to the user for application design
o tuning facilities
 DBMS Tuning
 With the help of Database Tuning, Performance of a system is determined
by a number of factors such as use ability, portability and timing.
 Tuning is Measurement by direct observation or by programmed
processes of activity at various points in a computer system to find out
where bottle- necks and delays are taking place.
 DB2 generates statistics which can be used for tuning.
 User then uses SQL to repartitioning base tables, creating new indexes etc.
 Run time tuning : e.g. buffer space allocation is also possible

 Focuses on:
o Utilisation of resources
o Responsiveness to enquires
o general productivity
 Interest:
o Computer Management : summary reports
o Database Manager/Administrator : timing reports -> capacity planning
o Research / System Designers: very detailed timing reports
The Performance life cycle
NOTES

performance
DATABASE report
SYSTEM
usuesrer
perfcohrma
nagnecse performance
request measurement

result
interpretation
system
tuning
environmental
change user
changes

Fig 4.3 Performance life cycle

o The performance of the database is measured based upon the system
tuning based upon the environmental changes and user changes. It is in
turn de- pended upon the interpretations of previous performance.
o physical database design => greatest scope
o change & Performance: corporate, user requirements, new release
Real World Example –Banking
Here is an example of a Banking domain explaining the Bank Database and
its frequent performance evaluation at various levels, to decide upon the capacity
planning.

IBM Kit
database
response time
extraction file
Database tool
System SAS

database
buffer
file

log tapes daily

performance
reports
monthly
capacity performance
planning reports SAS

Fig 4.4 Banking domain

CS 6302 160
NOTE Metrics
S o Responsiveness:
o time (user input -> user output)
o Response time, I/O time
o influences: I/O calls, Q length, buffer, paging rates, locality of access
o Utilisation
o ratio length of time used by particular system component for interval
o device time/ total process time
o influence: program size, paging rules, overflow techniques,
unbalanced scheduling
o Productivity
o information processed over time
o influences : time slice quantum, switching time, scheduling and
multi- processing, contention
o Cost
o hardware prices, development time, depreciation over time
o only used in benchmarking
System Components
 Basic components which effect performance
 CPU
o In processor bound applications, the throughput of a CPU is
very important, especially so in batch processing.
o Multi-processor systems try to distribute workload over many
processors, but there still may be contention for other
resources.
 Disk Subsystem
o A low performance disk subsystem can waste CPU - i.e... under
utilized.
o VLDB demand large and fast disk systems.
o Methods of organizing disk allocation at the physical level has a
great impact on performance as disk organization at the logical
level
 Terminal Subsystems
 Workstations and users influences tend to be overlooked in
performance studies.
 especially important if processing utilizes workstations
 TPNS for IBM
161 CS 6302
  Operating Systems

CS 6302 160
 o OS can provide some services to DBMS e.g. locking which
NOTES
effect performance.
o OS can actually impair performance, some DBMS choose to
ignore some sub-components of OS.
 DBMS Application Software
o most relational systems comply to a minimal SQL standard
but performance of each system differs greatly because of
different application types.
o remember!
o measurement and analysis of overall system performance is
difficult because of many inteorperating components.
o so standard benchmarks were developed to test most of these
components in a particular type of environment.
TRANSACTION PROCESSING & PROPERTIES
Transaction Concept
 A transaction is a unit of program execution that accesses and
possibly updates various data items.
 A transaction must see a consistent database.
 During transaction execution the database may be inconsistent.
 When the transaction is committed, the database must be consistent.
 Two main issues to deal with:
o Failures of various kinds such as hardware failures and
system crashes
o Concurrent execution of multiple transactions

ACID Properties
To preserve integrity of data, the database system must ensure:
Atomicity. Either all operations of the transaction or none are properly reflected in
the database.
Consistency. Execution of a transaction in isolation preserves the consistency of
the database.
Isolation. Although multiple transactions may execute concurrently, each
transaction must be unaware of other concurrently executing transactions.
Intermediate transaction results must be hidden from other concurrently executed
transactions. That is, for every pair of transactions Ti and Tj, it appears to Ti that
either Tj finished execution before Ti
started, or Tj started execution after Ti finished.
CS 6302 162
NOTE Durability. After a transaction completes successfully, the changes it has made to
S the database persist, even if there are system failures.
Example of Fund Transfer
Transaction to transfer $50 from account Ato account B:
1. read( A)
2. A:= A- 50
3. write( A)
4. read(B)
5. B:= B+
50
6. write( B)
Consistency requirement — the sum of A and B is unchanged by the execution of
the transaction.
Atomicity requirement — if the transaction fails after step 3 and before step 6,
the system should ensure that its updates are not reflected in the database, or
else an inconsistency will result.
Durability requirement — once the user has been notified that the transaction
has completed (i.e. the transfer of the $50 has taken place), the updates to the
database by the transaction must persist despite failures.
Isolation requirement — if between steps 3 and 6, another transaction is
allowed to access the partially updated database, it will see an inconsistent
database (the sum A+ B will be less than it should be).

Example of Supplier Number

• Transaction : logical unit of work
• e.g change supplier number for Sx to Sy;
• The following transaction code must be written:

TransX: Proc Options (main);

/* declarations omitted */
EXEC SQL WHEREVER SQL-ERROR ROLLBACK;

GET LIST (SX, SY);

EXEC SQL UPDATE
S SET S# =: SY
WHERE S# = SX;

163 CS 6302
CS 6302 164
 EXEC SQL UPDATE NOTES
SP SET S# =: SY
WHERE S# = SX;
EXEC SQL COMMIT
RETURN ();
Note:
single unit of work here =
•2 updates to the DBMS
•to 2 separate databases
Therefore it is a sequence of several operations which transforms data from
one consistent state to another.
•either updates or none of them must be performed e.g. banking.
•A.C.I.D
•Atomic, Consistent, Isolated, Durable
ACID properties
•Atomicity - ‘all or nothing’, transaction is an indivisible unit that is
performed either in its entirety or not at all.
•Consistency - a transaction must transform the database from one consistent
state to another.
•Isolation - transactions execute independently of one another. Partial effects
of incomplete transactions should not be visible to other transactions.
•Durability - effects of a successfully completed (committed) transaction are
stored in DB and not lost despite failure.
Transaction State
 Active, the initial state; the transaction stays in this state while it is executing
 Partially committed, after the final statement has been executed.
 Failed, after the discovery that normal execution can no longer proceed.
 Aborted, after the transaction has been rolled back and the database
restored to its state prior to the start of the transaction.
Two options after it has been aborted
 Kill the transaction
 Restart the transaction, only if no internal logical error
 Committed, after successful completion.
Transaction Control/concurrency control
•vital for maintaining the CONSISTENCY of database
•allowing RECOVERY after failure
•allowing multiple users to access database CONCURRENTLY
•database must not be left inconsistent because of failure mid-tx.
•other processes should have consistent view of database.
•completed tx. should be ‘logged’ by DBMS and rerun if failure.

165 CS 6302
NOTE Transaction Boundaries
S • SQL identifies start of transaction as BEGIN.
• end of successful transaction as COMMIT.
• or ROLLBACK to abort unsuccessful transaction
• Oracle allows SAVEPOINTS at start of sub-transaction in
nested transactions.
Implementation
• Simplest form - single user
– keep uncommitted updates in memory
– Write to disk immediately a COMMIT is issued.
– Logged tx. re-executed when system is recovering.
– WAL write ahead log
• before view of the data is written to stable log
• carry out operation on data (in memory first)
– Commit Precedence
• after view of update written in log
memory
WAL(A) LOG(A)

1st
begin
A=A+1 commit
time
Read(A) 2nd
Write(A)

disk

Fig 4.5 Example over First time

memory
WAL(A) LOG(A)

1st
begin A=A+1 commit
tim Read(A) 2nd
Write(A)

disk

Fig 4.6 Example over Second time

Why the fuss?
NOTES
Multi-user complications
consistent view of DB to all other processes while transaction in
progress.
1. Lost Update Problem
• Balance of a bank account holds a Bal of £100
• Transaction 1 is withdrawing £10 from a bank account
• Transaction 2 is depositing £100 pounds
• Time Tx1 Tx2 Bal
1 Begin_transaction 100
2 Begin_transaction Read (bal) 100
3 Read (bal) Bal=bal +100 100
4 Bal = bal – 10 Write (bal) 200
5 Write (bal) Commit 90
6 commit 90

Solution : £100 is lost, so we must prevent T1 from reading the

value of Bal after T2 is finished
2. Uncommitted dependency problem
Time Tx3 Tx4 Bal
1 Begin_transaction 100
2 Read (bal) 100
3 Bal=bal +100 100
4 Begin_transaction Write (bal) 200
5 Read (bal) …. 200
6 Bal = bal –10 Rollback 100
7 Write (bal) 190
8 commit 190
Solution :
Transaction 4 needed to abort at time t6.The problem is where
transaction is allowed to see intermediate results of another
uncommitted transaction.
The optimized answer is - £90

Implementation of Atomicity and Durability

The recovery-management component of a database system implements
the support for atomicity and durability.
The shadow-database scheme:
 Assume that only one transaction is active at a time.
 A pointer called db pointer always points to the current consistent copy of
the database.
NOTE  All updates are made on a shadow copy of the database, and db pointer
S is made to point to the updated shadow copy only after the transaction
reaches partial commit and all updated pages have been flushed to disk.
 In case transaction fails, old consistent copy pointed to by db pointer
can be used, and the shadow copy can be deleted.

Fig 4.7 Shadow Database

Importance of Shadow Database Scheme
1. Assumes that disks do not fail.
2. Useful for text editors, but extremely inefficient for large databases:
executing a single transaction requires copying the entire database.

CONCURRENT EXECUTIONS
Definition: They are multiple transactions that are allowed to run concur- rently in
the system.
Advantages of concurrent executions
o Increased processor and disk utilization, leading to better
transaction throughput: one transaction can be using the CPU while
another is read- ing from or writing to the disk
o Reduced average response time for transactions: short transactions
need not wait behind long ones
Concurrency control schemes:
They are mechanisms to control the interaction among the concurrent
transactions in order to prevent them from destroying the consistency of the
database.
Schedules
Schedules are sequences that indicate the chronological order in which
instructions of concurrent transactions are executed.
  A schedule for a set of transactions must consist of all instructions of
NOTES
those transactions.
 Must preserve the order in which the instructions appear in each
individual transaction
Example Schedules
Let T1 transfer $50 from A to B, and
T2 transfer 10% of the Balance from A to
B. The following is a Serial Schedule in
which T1 is followed by T2.
Let T1 and T2 be the transactions
defined previously. The schedule 2 is not a
serial
schedule, but it is equivalent to Schedule 1. Figure 4.8 Schedule -1

Figure 4.9 Schedule - 2 Figure 4.10 Schedule 3

In both Schedules 1 and 3, the sum A+ B is preserved.

The Schedule in Fig-3 does not preserve the value of the sum A+ B.
Recoverability
Need to address the effect of transaction failures on concurrently running transactions.
 Recoverable schedule — if a transaction Tj reads a data items previously written
by a transaction Ti, the commit operation of Ti appears before the commit
opera- tion of Tj.
 The following schedule (Schedule 11) is not recoverable
 If T9 commits immediately after the read
 If T8 should abort, T9 would have read
(and possibly shown to the user) an inconsistent
database state. Hence database must ensure Figure 4.11 Recoverability

that schedules are recoverable.

 NOTE Cascading rollback:
S Definition: If a single transaction failure leads to a series of transaction rollbacks, it
is called as cascaded rollback.
Consider the following schedule where none of the transactions has yet committed.
 If T10 fails, T11 and T12
must also be rolled back.
 Can lead to the undoing of
a significant amount of
work
Cascade less schedules: Figure 4.12 Cascade less
The cascading rollbacks cannot occur; for each pair of transactions Ti and Tj such
that Tj reads a data item previously written by Ti, the commit operation of Ti
appears before the read operation of Tj.
 Every cascade less schedule is also recoverable.
 It is desirable to restrict the schedules to those that are cascade less.
SERIALIZABILITY
Basic Assumption:
 Each transaction preserves database consistency.
 Thus serial execution of a set of transactions preserves database consistency
 A (possibly concurrent) schedule is serializable if it is equivalent to a serial schedule.
Different forms of schedule equivalence give rise to the notions of:
1. Conflict Serializability
2. View Serializability
Conflict Serializability
Instructions Ii and Ij, of transactions Ti and Tj respectively, conflict if and
only if there exists some item Q accessed by both Ii and Ij, and at least one of
these instruc- tions wrote Q.

1. Ii = read( Q), Ij = read( Q). Ii and Ij don’t conflict.

2. Ii = read( Q), Ij = write( Q). They conflict.
3. Ii = write( Q), Ij = read( Q). They conflict.
4. Ii = write( Q), Ij = write( Q). They conflict.

Intuitively, a conflict between Ii and Ij forces a temporal order between

them. If Ii and Ij are consecutive in a schedule and they do not conflict, their
results would remain the same even if they had been interchanged in the
schedule.
 If a schedule S can be transformed into a schedule S0 by a series of swaps of
non- conflicting instructions, we say that S and S0 are conflict equivalent.
 We say that a schedule Sis conflict serializable if it is conflict equivalent to a
NOTES
serial schedule.
Example of a schedule that is not conflict serializable:
We are unable to swap instructions in the
above schedule to obtain either the serial schedule <
T3, T4 >, or the serial schedule < T4, T3 >.
Schedule 3 below can be transformed into Schedule 1, a serial schedule where
T2 follows T1, by a series of swaps of non-conflicting instructions. Therefore
Schedule 3 is conflict serializable.
View Serializability
Let Sand S0 be two schedules with the same set of transactions. Sand S0
are view equivalent if the following three conditions are met:
1. For each data item Q, if transaction Ti reads the initial value of Q in
schedule S, then transaction Ti must, in schedule S0, also read the initial
value of Q.
2. For each data item Q, if transaction Ti executes read( Q) in schedule S,
and that value was produced by transaction Tj (if any), then transaction Ti
must in schedule S0 also read the value of Q that was produced by
transaction Tj.
3. For each data item Q, the transaction (if any) that performs the final
write
(Q) operation in schedule S must perform the final write (Q) operation in
sched- ule S0.
 As can be seen, view equivalence is also based purely on reads and
writes alone.
 A schedule Sis view serializable if it is view equivalent to a serial schedule.
 Every conflict serializable schedule is also view serializable.
The following schedule is view-serializable but not conflict
serializable.

Figure 4.14 View - serializable

Every view serializable schedule, which is not conflict serializable, has blind writes.
Precedence graph: It is a directed graph where the vertices are the transactions
(names). We draw an arc from Ti to Tj if the two transactions conflict, and Ti
accessed
171 CS 6302
NOTE the data item on which the conflict arose earlier. We may label the arc by the item
S that was accessed.
Example :

Figure 4.15 Precedence Graph

Example Schedule (Schedule A)

Figure 4.16 Example Schedule (A)

Precedence Graph for Schedule A

Figure 4.17 Precedence Graph (A)

Test for Conflict Serializability

A schedule is conflict serializable if and only if its precedence graph is
acyclic. Cycle-detection algorithms exist which take order n2 time, where n is the
number of vertices in the graph.

CS 6302 170
171 CS 6302
 If precedence graph is acyclic, the serializability order can be obtained
NOTES
by a topological sorting of the graph. This is a linear order consistent with the
partial order of the graph.
For example, a Serializability order for Schedule Awould be T5
T1T3T2T4.

Test for View Serializability

The precedence graph test for conflict serializability must be modified to
apply to a test for view serializability.
Construct a labeled precedence graph. Look for an acyclic graph, which
is derived from the labeled precedence graph by choosing one edge from every
pair of edges with the same non-zero label. Schedule is view serializable if and
only if such an acyclic graph can be found.
The problem of looking for such an acyclic graph falls in the class of
NP- complete problems. Thus existence of an efficient algorithm is unlikely.
However practical algorithms that just check some sufficient conditions
for view serializability can still be used.

Summary :
 Many DB operations require reading tuples, tuple vs. previous tuples, or tuples
vs. tuples in another table.
 Basic Steps in Query Processing are Parsing , translation , Evaluation and
Optimizer
 Basic Query Operators are One-pass operators and Multi-pass operators.
 Aquery planis algebraic tree ofoperators, with choice ofalgorithmfor each
operator.
 With the help of Database Tuning, Performance of a system is determined
by a number of factors such as use ability, portability and timing.
 ACID Properties are Atomicity,Consistency,Isolation and Durability.

Questions :
1. Explain the properties of transaction.
2. Define ACID.
3. Differentiate One pass and Two pass Operators.
4. Define Database Tuning.
5. Explain Transaction failures.
6. Explain Serializability, Query processing, Optimization , Recoverability.

CS 6302 170
CS 6302 172
NOTE References :
S 1. Ramez Elamasri and shankant B.Navathe, “ Fundamentals of
Database Systems”, Third Edition , Pearson Education, Delhi, 2002.
2. Abraham Silberschatz, Henry F.Korth and S.Sundarshan , “Database
System concepts “, Fourth Edition , McGrawHill , 2002.
3. C.J.Date,”An Introduction to Database Systems”,Seventh
Edition,Pearson Education,Delhi, 2002.

173 CS 6302
 NOTES

CONCURRENCY, RECOVERY AND

SECURITY
Learning Objectives
 To Learn various Locking Techniques
 To learn about Time Stamp Ordering
 To Study various Validation Techniques
 To learn about Granularity of Data Items
 To Study about Recovery Concepts
 To Learn about Shadow paging and Log Based Recovery
 To learn about Data Security Issues such as Access Control ,
Statistical Database security

LOCKING TECHNIQUES

Lock-Based Protocols

A lock is a mechanism to control concurrent access to a data item Data items can
be locked in two modes:

1. Exclusive (X) mode. Data item can be both read as well as written. X-
lock is requested using lock-X instruction.

2. Shared (S) mode. Data item can only be read. S-lock is requested using
lock-S instruction.

Lock requests are made to concurrency-control

manager. Transaction can proceed only after request is

granted.
NOTE
Lock-compatibility matrix
S

Fig 5.1 Lock – Compatibility Matrix

A transaction maybe granted a lock on an itemif the requested lock is
compatible with locks already held on the item by other transactions.
The matrix allows any number of transactions to hold shared locks on an
item, but if any transaction holds an exclusive on the item no other transaction
may hold any lock on the item.
If a lock cannot be granted, the requesting transaction is made to wait till
all incompatible locks held by other transactions have been released. The lock is
then granted.
Example of a transaction performing locking:
T2: lock-S(
A); read(
A);
unlock( A)
; lock-S(
B);
read( B);
unlock( B);
display( A+ B).
Locking as above is not sufficient to guarantee serializability.
If A and B get updated in-between the read of A and B, the displayed sum would
be wrong.
A locking protocol is a set of rules followed by all transactions while requesting
and releasing locks. Locking protocols restrict the set of possible schedules.
Pitfalls of Lock-Based Protocols

Consider the partial schedule

Fig 5.2 Partial Schedule

 Neither T3 nor T4 can make progress — executing lock-S( B) causes T4 o
to wait for T3 to release its lock on B, while executing lock-X( A) causes T3 to b
wait for T4 to release its lock on A. Such a situation is called a deadlock. t
a
To handle a deadlock one of T3 or T4 must be rolled back and its locks
i
released. The potential for deadlock exists in most locking protocols. n
Deadlocks are a necessary evil. e
d
Starvation
It is also possible if concurrency controlmanager is badly designed. For i
example: f
A transaction maybe waiting for an X-lock on an item, while a t
sequence of other transactions request and are granted an S-lock w
on the same item. o
-
The same transaction is repeatedly rolled back due to deadlocks. p
Concurrency control manager can be designed to prevent h
a
starvation.
s
The Two-Phase Locking Protocol e
Introduction l
o
This is a protocol, which ensures conflict-serializable schedules. c
Phase 1: Growing Phase k
i
– Transaction may obtain locks.
n
– Transaction may not release locks.
g
Phase 2: Shrinking Phase
i
– Transaction may release locks.
s
– Transaction may not obtain locks. u
– The protocol assures serializability. It can be proved that the s
transactions can be serialized in the order of their lock points. e
d
– Lock points: It is the point where a transaction acquired its final
.
lock.
– Two-phase locking does not ensure freedom from deadlocks.
– Cascading rollback is possible under two-phase locking.
– There can be conflict serializable schedules that cannot be
NOTES
NOTE Given a transaction Ti that does not follow two-phase locking, we can find
a
S
transaction Tj that uses two-phase locking, and a schedule for Ti and Tj that is not
conflict serializable.
Lock Conversions
Two-phase locking with lock conversions:
First Phase:
_ can acquire a lock-S on item
_ can acquire a lock-X on item
_ can convert a lock-S to a lock-X (upgrade)
Second Phase:
_ can release a lock-S
_ can release a lock-X
_ can convert a lock-X to a lock-S (downgrade)
This protocol assures serializability. But still relies on the programmer to insert the
various locking instructions.
Automatic Acquisition of Locks
Atransaction Ti issues the standard read/write instruction, without explicit locking
calls.
The operation read( D) is processed as:
if Ti has a lock on D
then
read( D)
else
begin
if necessary wait until no
other transaction has a lock-
X on D grant Ti a lock-S on
D;
read( D)
end;
write( D) is processed
as: if Ti has a lock-X on
D then
write( D)
else
b
e
g
i
n
 if necessary wait until no other trans. has any lock on D, – o
if Ti has a lock-S on D t
then h
upgrade lock on Dto lock- e
X else r
grant Ti a lock-X on p
D write( D) r
o
end;All locks are released after commit or abort
c
Graph-Based Protocols e
s
 It is an alternative to two-phase locking. s
 Impose a partial ordering on the set D = f d1, d2 , ..., dhg of all data e
items. s
 If di! dj, then any transaction accessing both di and dj must access di m
before a
accessing dj. y
 Implies that the set D may now be viewed as a directed acyclic graph, r
called a database graph. e
 Tree-protocol is a simple kind of graph protocol. a
d
Tree Protocol b
Only exclusive locks are allowed. u
t
 The first lock by Ti may be on any data item. Subsequently, Ti can lock a n
data item Q, only if Ti currently locks the parent of Q. o
 Data items may be unlocked at any time. t
 A data item that has been locked and unlocked by Ti cannot u
subsequently be re-locked by Ti. p
 The tree protocol ensures conflict serializabilityas well as freedomfrom d
deadlock. a
 Unlocking may occur earlier in the tree-locking protocol than in the two- t
phase locking protocol. e
 Shorter waiting times, and increase in concurrency it
 Protocol is deadlock-free.
 However, in the tree-locking protocol, a transaction may have to lock
data items that it does not access.
 Increased locking overhead, and additional waiting time
 Potential decrease in concurrency
 Schedules not possible under two-phase locking are possible under tree
protocol, and vice versa.

Locks

 Exclusive

– intended to write or modify database

NOTES
NOTE – Oracle sets Xlock before INSERT/DELETE/UPDATE
S – explicitly set for complex transactions to avoid Deadlock

 Shared

– read lock, set by many transactions simultaneously

– Xlock can’t be issued while Slock on
– Oracle automatically sets Slock
– explicitly set for complex transactions to avoid Deadlock

 Granularity

 Physical level: block or page

 Logical level: attribute, tuple, table, database
Probability performance process
of conflict overhead granularity management

lowest highest fine complex

highest lowest thick easy

Lock escalation
 promotion/escalation of locks occur when a % of individual rows locked
in a table reaches a threshold
 becomes table lock = reduce number of locks to manage.
 Oracle does not use escalation.
 maintains separate locks for tuples and tables.
 also a ‘row exclusive’ lock on T itself to prevent another transaction
locking table T.
SELECT *each row which matches the condition in table T receives a Xlock
FROM T
WHERE condition
FOR UPDATE also a ‘row exclusive’ lock on T itself to prevent
another transaction locking table T.
 this statement explicitly identifies a set of rows which will be used to
update another table in the database, and whose values must remain
unchanged throughout the rest of the transaction.
 DeadLock
Dead lock occurs when a process gets locked due to lack of resource ,
NOTES
which is under the control of other process. And in turn also holds a resource
which is needed by that process.
As clearly depicted by Fig. 5.4, Such a state occurs in DBMS also when
more than one transactions occur on the same resource in case of multi user
environment.

pA
time
pB
• Deadlock happens in DBMS also due to
blockage of resources by few processes.
read R
process
R
read S
process
• DBMS maintains 'Wait For' graphs
read R
showing processes waiting and resources
read S
wait on S S wait on R
held.
• if cycle detected, DBMS selects

Fig 5.4 Deadlock

wait for

• The other transaction S pA

can not proceed due to its
resource unavailability. held held
by by
• rolled back transaction pB
re-executes. R
wait for
Fig 5.5 Deadlock Detection

TIME STAMP ORDERING

 concurrency protocol in which the fundamental goal is to order

transactions globally in such a way that older transactions,
transactions with smaller timestamps, get priority in the event of
conflict.
 lock + 2pl => serializability of schedules
 alternative uses transaction timestamps
 no deadlock as no locks!
 no waiting, conflicting tx rolled back and restarted
 Timestamp: unique identifier created by the DBMS that indicates the
relative starting time of a transaction
 logical counter or system clock
 if a tx attempts to R/W data, then it only allowed to proceed if the
LAST UPDATE ON THAT DATA was carried out by an older tx.
 otherwise that tx. is restarted with new timestamp.
 Data also has timestamps, read and write.
CS 6302 180
NOTES Three problems: check for next year
time T12 T13 T14
read where ts(T) < write-ts(X) : trying
to update a data item after it has t1 begin_tx
been updated with a transaction t2 read(x)
which is younger than the current t3 x=x+10
tx. Is it too late? Abort+restart t4 write(x) begin_tx
t5 read(y)
write where ts(T) < read_ts(X) : later t6 Y=Y+20 begin_tx
tx is already using current value t7 read(y)
of item and erroneous to update t8
it now. younger tx already read
write(y)
t9 y=y+30
old value/written new one.
Abort+restart
t10 write(y)
t11 z=100
write where ts(T) < write_ts(X) : a t12 write(z)
later tx has updated the data, and t13 z=50 abort COMMIT
this tx is based in obsolete data t14 write(z) begin_tx
value. t15 COMMIT read(y)
Ignore write. - Ignore t16 y=y+20
obsolete write rule. t17 write(y)
t18 COMMIT
t8: 2nd rule violated: T14 read(y)
T13 restarted at t14
t14: write ignored as T14 wrote at t12

Fig 5.6 Time Stamp Ordering

5.3 OPTIMISTIC VALIDATION TECHNIQUES

• Oracle physically updates data using LRU algorithm
• state of database at failure time is complex,
– results of committed tx. stored on disc
– results of committed tx. stored in memory buffers
– results of UNcommitted tx. stored on disc
– results of UNcommitted tx. stored in memory buffers

• Fu rthermore:
– most recent data
– no.+how far back depends on disk space
– number of rollback segments for uncommitted
tx.
– Log of committed transactions
• Failures can leave database in a corrupted or inconsistent state -weakening
Consistency and Durability features of transaction processing.
• Oracle level of failures:
– statement: causes relevant tx. to be rolled back - database returns to
previous state releasing resources
181 CS 6302
 – process: abnormal disconnection form session, rolled back...etc.
– instance: crash in DBMS software, operating system, or hardware -
NOTES
DBA
issues a SHUTDOWNABORT command which triggers off instance
re-
cover procedures.
– media failure: disk head crash, worst case = destroyed database and
log
files. Previous backup version must be restored from another disk.
Data
Replication.
• Consequently:
– SHUTDOWN trashes the memory buffers - recovery used disk data
and:
– ROLLS FORWARD: re-apply committed tx. on log which holds
before/after images of updated rec.
– ROLLBACK uncommitted tx. already written to database using
rollback segments.
5.3.3. Media Recovery
• disk failure - recover from backup tape using Oracle EXPORT/IMPORT
• necessary to backup log files and control files to tape
• DBA decide to keep complete log archives which are costly in terms of:
• time
• space
• administrative overheads
 – OR
• cost of manual re-entry

• Log archiving provides the power to do on-line backups without shutting

down the system, and full automatic recovery. It is obviously a necessary
option for operationally critical database applications.

GRANULARITY OF DATA ITEMS

Storage Structure
 Volatile storage:
o does not survive system crashes
o examples: main memory, cache memory
 Nonvolatile storage:
o survives system crashes
o examples: disk, tape, flash memory,
non-volatile (battery backed up) RAM
 Stable storage:
o a mythical form of storage that survives all failures
o approximated by maintaining multiple copies on
distinct nonvolatile media
CS 6302 182
NOTE Stable-Storage Implementation
 Maintain multiple copies of each block on separate disks
S o copies can be at remote sites to protect against disasters such as
fire or flooding.
 Failure during data transfer can still result in inconsistent copies:
Block transfer can result in
o Successful completion
o Partial failure: destination block has incorrect information
o Total failure: destination block was never updated
 Protecting storage media from failure during data transfer (one solution):
o Execute output operation as follows (assuming two copies of
each block):
 Write the information onto the first physical block.
 When the first write successfully completes, write the same information
onto the second physical block.
 The output is completed only after the second write successfully completes.
 Protecting storage media from failure during data transfer (cont.):
 Copies of a block may differ due to failure during output operation.
To recover from failure:
 First find inconsistent blocks:
o Expensive solution: Compare the two copies of every disk block.
o Better solution: Record in-progress disk writes on non-volatile
storage (Non-volatile RAM or special area of disk).

 Use this information during recovery to find blocks that may be

inconsistent, and only compare copies of these.
 Used in hardware RAID systems
 If either copy of an inconsistent block is detected to have an error
(bad checksum), overwrite it by the other copy. If both have no
error, but are different, overwrite the second block by the first block.
Data Access
 Physical blocks are those blocks residing on the disk.
 Buffer blocks are the blocks residing temporarily in main memory.
 Block movements between disk and main memory are initiated through
the following two operations:
o input(B) transfers the physical block B to main memory.
o output(B) transfers the buffer block B to the disk, and replaces
the appropriate physical block there.
 Each transaction Ti has its private work-area in which local copies of all
data items accessed and updated by it are kept.

183 CS 6302
CS 6302 184
 o Ti’s local copy of a data item X is called xi.
NOTES
 We assume, for simplicity, that each data item fits in, and is stored inside, a
single block.
 Transaction transfers data items between system buffer blocks and its
private work-area using the following operations :
o read(X) assigns the value of data item X to the local variable xi.
o write(X) assigns the value of local variable xi to data item {X} in
the buffer block.
o both these commands may necessitate the issue of an input(BX)
instruc- tion before the assignment, if the block BX in which X resides
is not already in memory.
 Transactions
o Perform read(X) while accessing X for the first time;
o All subsequent accesses are to the local copy.
o After last access, transaction executes write(X).
 output(BX ) need not immediately follow write(X). System can perform the
output operation when it deems fit.

5.4.4. Example of Data Access

buffer
Buffer Block A input(A)
x
Buffer Block B Y A
output(B) B
read(X)
write(Y)

x2 disk
x1
y1
work area work area
of T1 of T2
memory

Fig 5.7 Data Access

185 CS 6302
NOTE FAILURE & RECOVERY CONCEPTS
S Types of Failures
 Computer failure
 Hardware or software error (or dumb user error)

 Main memory may be lost

 Transaction failure
 Transaction abort
 Forced by local transaction error

 Forced by system (E.g. concurrency control)

 Disk failure
 Disk copy of DB lost

 Physical problem or catastrophe

 Earthquake, fire, flood, open window, etc.

Another type of Failure Classification

 Transaction failure :
 Logical errors: transaction cannot complete due to some
internal error condition
 System errors: the database system must terminate an
active transaction due to an error condition (E.g., deadlock)
 System crash: a power failure or other hardware or software failure
causes the system to crash.
 Fail-stop assumption: non-volatile storage contents are assumed
that they are not corrupted by system crash
 Database systems have numerous integrity checks to prevent
corruption of disk data.
 Disk failure: a head crash or similar disk failure destroys all or part
of disk storage
 Destruction is assumed to be detectable: disk drives use checksums
to detect failures
Recovery
 Recovery fromfailure means to restore the database to the most recent
consistent state just before the failure.
 The DBMS makes use of the system logs for this purpose.
Log Records
NOTES
 log records contain a unique transaction id
 need to log writes but not reads
 records old and new values LOG
 supports undo and redo
[begin_trans, T1]
UNDO: Its effects of incomplete transactions
[write_item, T1, X, 50, 40]
are rolled back [write_item, T1, Y, 20, 30]
REDO: It ensure effects of [commit, T1]
committed transactions are reflected [begin_trans, T2]
[write_item, T2, X, 40, 20]
in state of the DB
[commit, T2]
Recovery from Catastrophic Failure
Fig 5.8 LOG Records
 Extensive damage to database (such as disk
crash)
o Recovery method is used to restore from back-up (hopefully you were
smart enough to have one)
o Use logs to redo operations of the committed transactions up to the
time of the failure.
Recovery from Non-Catastrophic Failures
 Database has become inconsistent. Use the logs to:
o Reverse any changes that caused the inconsistency by undoing operations.
o It may also be necessary to redo some operations to restore consistency.
Buffer Pools (DBMS Caching)
Buffer Pool Directory
 Keeps track of which pages are in the buffer pool
o <disk page address, buffer location>
 No changes made to the page?
o Dirty bit = 0
 Changes made to the page?
o Dirty bit = 1
 Pin-unpin (in db2 called “fix/unfix”) indicates whether or not a page can
be written back to disk yet.
o Fixed or pinned means it cannot be written to disk at this time.
NOTE Strategies for flushing
S  A modified buffer is “flushed” to disk
 2 strategies:
o In-Place Updating – writes the buffer back to the same original disk
location thus overwriting
o Shadowing – writes the buffer at a different disk location
o Old value is maintained in the old location. This is called the
“before image”.The new value in the new location is called the
“after image”.
o In this case, logging is not necessary.
Incremental Log with Immediate Updates
 All updates are applied “immediately” to DB.
 Log keeps track of changes to DB state.
 Commit just requires writing a commit record to log.
Write Ahead Logging
 In-place updating requires a log to record the old value for a data object
 Log entry must be flushed to disk BEFORE the old value is overwritten.
o The log is a sequential, append only file with the last n log blocks stored
in the buffer pools. So, when it is changed (in memory), these changes
have to be written to disk.
o Recall that the logs we have seen record:
o <transid, objectid, OLDVALUE, NEWVALUE>

RECOVERY TECHNIQUES
 The DBMS recovery subsystem maintains several lists including:
o Active transactions (those that have started but not committed yet)
o Committed transactions since last checkpoint
o Aborted transactions since last checkpoint
Checkpoints
 A [checkpoint] is another entry in the log
o Indicates a point at which the system writes out to disk all DBMS buffers
that have been modified.
  Any transactions, which have committed before a checkpoint will not have
NOTES
to have their WRITE operations redone in the event of a crash since we
can be guaranteed that all updates were done during the checkpoint
operation.
 Consists of:
 Suspend all executing transactions
 Force-write all modified buffers to disk
 Write a [checkpoint] record to log
 Force the log to disk
 Resume executing transactions
Transaction Rollback
 If data items have been changed by a failed transaction, the old values must be
restored.
 If T is rolled back, if any other transaction has read a value modified by T,
then that transaction must also be rolled back…. cascading effect.
Two Main Techniques
 Deferred Update
 No physical updates to db until after a transaction commits.
 During the commit, log records are made then changes are made permanent
on disk.
 What if a Transaction fails?
o No UNDO required
o REDO may be necessary if the changes have not yet been made
permanent before the failure.
 Immediate Update
 Physical updates to db may happen before a transaction commits.
 All changes are written to the permanent log (on disk) before changes are
made to the DB.
 What if a Transaction fails?
o After changes are made but before commit – need to UNDO the
changes
o REDO may be necessary if the changes have not yet been made
permanent before the failure.
NOTE Recovery based on Deferred Update
S  Deferred update –
 Changes are made in memory and after T commits, the changes are made
permanent on disk.
 Changes are recorded in buffers and in log file during T’s execution.
 At the commit point, the log is force-written to disk and updates are made
in database.
 No need to ever UNDO operations because changes are never made permanent
 REDO is needed if the transaction fails after the commit but before the changes
are made on disk.
 Hence the name NO UNDO/REDO algorithm
 2 lists maintained:
 Commit list: committed transactions since last checkpoint
 Active list: active transactions
 REDO all write operations from commit list
 in order that they were written to the log
 Active transactions are cancelled & must be resubmitted.
Recovery based on Immediate Update
 Immediate update:
 Updates to disk can happen at any time
 But, updates must still first be recorded in the system logs (on disk) before
changes are made to the database.
 Need to provide facilities to UNDO operations which have affected the db
 2 flavors of this algorithm
 UNDO/NO_REDO recovery algorithm
o if the recovery technique ensures that all updates are made to the
database on disk before T commits, we do not need to REDO any
committed transactions
o UNDO/REDO recovery algorithm
o Transaction is allowed to commit before all its changes are written to
the database
o (Note that the log files would be complete at the commit point)
 UNDO/REDO Algorithm
NOTES
 2 lists maintained:
 Commit list: committed transactions since last checkpoint
 Active list: active transactions
 UNDO all write operations of active transactions.
 Undone in the reverse of the order in which they were written to the log
 REDO all write operations of the committed
transactions
 In the order in which they were written to the log.
Recovery and Atomicity
 Modifying the database without ensuring that the transaction will commit may
leave the database in an inconsistent state.
 Consider transaction Ti that transfers $50 from account A to account B; goal
is either to perform all database modifications made by Ti or none at all.
 Several output operations may be required for i
T (to output A and B). A
failure may occur after one of these modifications has been made but before
all of them are made.
 To ensure atomicity despite failures, we first output information describing
the modifications to stable storage without modifying the database itself.
 We study two approaches:
o log-based recovery, and
o shadow-paging
 We assume (initially) that transactions run serially, that is, one after the other.
SHADOW PAGING
 Shadow paging is an alternative to log-based recovery; this scheme is
useful if transactions execute serially.
 Idea: maintain two page tables during the lifetime of a transaction –the
current page table, and the shadow page table
 Store the shadow page table in nonvolatile storage, such that state of the
database prior to transaction execution may be recovered.
o Shadow page table is never modified during execution.
 To start with, both the page tables are identical. Only current page table is
used for data item accesses during execution of the transaction.
CS 6302 190
NOTE  Whenever any page is about to be written for the first time
S o A copy of this page is made onto an unused page.
o The current page table is then made to point to the copy
o The update is performed on the copy

Sample Page Table Example of Shadow Paging

 To commit a transaction :
1. Flush all modified pages in main memory to disk
2. Output current page table to disk
3. Make the current page table the new shadow page table, as follows:
o keep a pointer to the shadow page table at a fixed (known)
location on disk.
o to make the current page table the new shadow page table,
simply update the pointer to point to current page table on disk
 Once pointer to shadow page table has been written, transaction is
commit- ted.
 No recovery is needed after a crash — new transactions can start
right away, using the shadow page table.
 Pages not pointed to from current/shadow page table should be
freed (garbage collected).
 Advantages of shadow-paging over log-based schemes
o no overhead of writing log records
o recovery is trivial

191 CS 6302
CS 6302 190
  Disadvantages :
NOTES
o Copying the entire page table is very expensive
n Can be reduced by using a page table structured like a B+-tree
n No need to copy entire tree, only need to copy paths in the
tree that lead to updated leaf nodes
o Commit overhead is high even with above extension
n Need to flush every updated page, and page table
o Data gets fragmented (related pages get separated on disk)
o After every transaction completion, the database pages containing
old versions of modified data need to be garbage collected
o Hard to extend algorithm to allow transactions to run concurrently
 Easier to extend log based schemes
Recovery With Concurrent Transactions
 We modify the log-based recovery schemes to allow multiple transactions
to execute concurrently.
 All transactions share a single disk buffer and a single log

 A buffer block can have data items updated by one or

more transactions
 We assume concurrency control using strict two-phase locking;
 i.e.
the updates of uncommitted transactions should not be visible
to other transactions
 Otherwise how to perform undo if T1 updates A, then T2

updates A and commits, and finally T1 has to abort?

 Logging is done as described earlier.
 Log records of different transactions may be interspersed in
the log.
n The checkpointing technique and actions taken on
recovery have to be changed
 since several transactions may be active when a checkpoint
is performed.
n Checkpoints are performed as before, except that the
check point log record is now of the form
< checkpoint L>
where L is the list of transactions active at the time of the
checkpoint

191 CS 6302
CS 6302 192
NOTE  We assume no updates are in progress while the checkpoint is
S carried out (will relax this later).
 When the system recovers from a crash, it first does the following:
 Initialize undo-list and redo-list to empty
 Scan the log backwards from the end, stopping when the first <checkpoint
L> record is found.
For each record found during the backward scan:
 if the record is <Ti commit>, add T to redo-list
 if the record is <T i start>, then if T iis not in redo-list, add T to undo-list
 For every T iin L, if Ti is not in redo-list, add T to undo-list
 At this point undo-list consists of incomplete transactions which must be
undone, and redo-list consists of finished transactions that must be redone.
 Recovery now continues as follows:
 Scan log backwards from most recent record, stopping when
<Ti start> records have been encountered for every Ti in undo-list.
 During the scan, perform undo for each log record that belongs to a
transaction in undo-list.
 Locate the most recent <checkpoint L> record.
 Scan log forwards from the <checkpoint L> record till the end of the log.
 During the scan, perform redo for each log record that belongs to a
transaction on redo-list
Example of Recovery
 Go over the steps of the recovery algorithm on the following log:
<T0 start>
<T0, A, 0, 10>
<T0 commit>
<T1 start>
<T1, B, 0, 10>
<T2 start> /* Scan in Step 4 stops here */
<T2, C, 0, 10>
<T2, C, 10, 20>
<checkpoint {T1, T2}>
<T3 start>
<T3, A, 10, 20>
<T3, D, 0, 10>
<T3 commit>

193 CS 6302
CS 6302 194
Log Record Buffering
NOTES
 Log record buffering: log records are buffered in main memory, instead of
being output directly to stable storage.
 Log records are output to stable storage when a block of
log records in the buffer is full, or a log force operation is
executed.
 Log force is performed to commit a transaction by forcing all its log
records (including the commit record) to stable storage.
 Several log records can thus be output using a single output operation,
reducing the I/O cost.
 The rules below must be followed if log records are buffered:
 Log records are output to stable storage in the order in
which they are created.
 Transaction Ti enters the commit state only when the log record
<Ti commit> has been output to stable storage.
 Before a block of data in main memory is output to the
database, all log records pertaining to data in that block must
have been
output to stable storage.
 This rule is called the write-ahead logging or WAL rule
 Strictly speaking WAL only requires undo information to be output

5.7.5. Database Buffering

 Database maintains an in-memory buffer of data blocks

 When a new block is needed, if buffer is full an existing
block needs to be removed from buffer.
 If the block chosen for removal has been updated, it must
be output to disk.
 As a result of the write-ahead logging rule, if a block with uncommitted
updates is output to disk, log records with undo information for the updates
are output to the log on stable storage first.
 No updates should be in progress on a block when it is output to disk. Can
be ensured as follows.
 Before writing a data item, transaction acquires exclusive lock
on block containing the data item
 Lock can be released once the write is completed.
 Such locks held for short duration are called latches.
195 CS 6302
CS 6302 196
NOTE  Before a block is output to disk, the system acquires an
S exclusive latch on the block.
 Ensures that no update can be in progress on the block
 Database buffer can be implemented either
 in an area of real main-memory reserved for the database, or
 in virtual memory
 Implementing buffer in reserved main-memory has drawbacks:
 Memory is partitioned before-hand between database buffer
and applications, limiting flexibility.
 Needs may change, and although operating system knows best
how memory should be divided up at any time, it cannot change the
partitioning of memory.
 Database buffers are generally implemented in virtual memory in spite of some
drawbacks:
 When operating system needs to evict a page that has been
modified, to make space for another page, the page is written to
swap space on disk.
 When database decides to write buffer page to disk, buffer page
may be in swap space, and may have to be read from swap space
on disk and output to the database on disk, resulting in extra I/O!
 Known as dual paging problem.
 Ideally when swapping out a database buffer page, operating
system should pass control to database, which in turn outputs
page to
database instead of to swap space (making sure to output log
records first)
 Dual paging can thus be avoided, but common operating systems do not
support such functionality.

LOG-BASED RECOVERY

 A log is kept on stable storage.

o The log is a sequence of log records, and maintains a record of
update activities on the database.
 When transaction T i starts, it registers itself by writing a
<Ti start>log record
 Before T executes write(X), a log record <T , X, V , V > is written, where V
is
i i 1 2 1
the value of X before the write, and V2 is the value to be written to X.
o Log record notes that Ti has performed a write on data item Xj Xj had value
V1
197 CS 6302
 before the write, and will have value V2 after the write.

CS 6302 198
  When T finishes it last statement, the log record <T commit> is written.  Below we
i
 We assume that log records are written directly to stable storage (that is, show the
log as it
they are not buffered) appears at
 Two approaches using logs three
instances of
o Deferred database modification time.
o Immediate database modification

Deferred Database Modification

 The deferred database modification scheme records all modifications to

the log, but defers all the writes to after partial commit.
 Assume that transactions execute serially.
 Transaction starts by writing <Ti start> record to log.
 A write(X) operation results in a log record <T
i
, X, V> being written, where
V
is the new value for X
o Note: old value is not needed for this scheme
 The write is not performed on X at this time, but is deferred.
 When Ti partially commits, <T commit> is written to the log
 Finally, the log records are read and used to execute the previously deferred
writes.
 During recovery after a crash, a transaction needs to be redone if and
only if both <Ti start> and<Ti commit> are there in the log.
 Redoing a transaction Ti ( redoT ) sets the value of all data items updated by
the
transaction to the new values.
 Crashes can occur while
o the transaction is executing the original updates, or
o while recovery action is being taken
 example transactions T
0
and T1 (T0 executes before T ):
T0: read (A) T1 : read (C)
A: - A - 50 C:- C- 100
Write (A) write (C)
read (B)
B:- B + 50
write (B)
NOTES
NOTE
S

Fig 5.11 Log entries at various instances

 If log on stable storage at time of crash is as in case:
(a) No redo actions need to be taken
(b) redo(T0) must be performed since <T0 commit> is present
(c) redo(T0) must be performed followed by redo(T1) since
<T0 commit> and <Ti commit> are present
Immediate Database Modification
 The immediate database modification scheme allows database updates of
an uncommitted transaction to be made as the writes are issued.
o since undoing may be needed, update logs must have both old value
and new value
 Update log record must be written before database item is written
o We assume that the log record is output directly to stable storage
o Can be extended to postpone log record output, so long as prior to
execution of an output(B) operation for a data block B, all log
records corresponding to items B must be flushed to stable storage
 Output of updated blocks can take place at any time before or after
transaction commit
 Order in which blocks are output can be different from the order in which
they are written.
Immediate Database Modification Example
Log Write Output
<T0 start>
<T0, A, 1000, 950>
To, B, 2000, 2050
A = 950
B = 2050
<T0 commit>
<T1 start>
<T1, C, 700, 600>
C = 600
BB, BC
<T1 commit>
BA
Fig 5.12 Immediate Database Modification
 Note: B denotes block containing X.
X NOTES
 Recovery procedure has two operations instead of one:
o undo(Ti) restores the value of all data items updated by Ti to their old
values, going backwards from the last log record for Ti
o redo(Ti) sets the value of all data items updated by Ti to the new
values, going forward from the first log record for Ti
o Both operations must be idempotent
o That is, even if the operation is executed multiple times the effect is
the same as if it is executed once
n Needed since operations may get re-executed during recovery
 When recovering after failure:
o Transaction Ti needs to be undone if the log contains the record
<Ti start>, but does not contain the record <Ti commit>.
o Transaction Ti needs to be redone if the log contains both the record
<Ti
start> and the record <Ti commit>.
 Undo operations are performed first, then redo operations.
Immediate DB Modification Recovery Example

Below we show the log as it appears at three instances of time.

Fig 5.13 Immediate DB Modification Recovery

Recovery actions in each case above are:
(a) undo (T0): B is restored to 2000 and A to 1000.
(b) undo (T1) and redo (T0): C is restored to 700, and then A and B
are set to 950 and 2050 respectively.
(c) redo (T0) and redo (T1): A and B are set to 950 and
2050 respectively. Then C is set to 600

Checkpoints

 Problems in recovery procedure as discussed earlier :

 searching the entire log is time-consuming
 we might unnecessarily redo transactions which have already
NOTE  output their updates to the database.
S  Streamline recovery procedure by periodically performing checkpointing
 Output all log records currently residing in main memory onto stable storage.
 Output all modified buffer blocks to the disk.
 Write a log record < checkpoint> onto stable storage.
 During recovery we need to consider only the most recent transaction T i
that started before the checkpoint, and transactions that started after Ti.
 Scan backwards from end of log to find the most recent <checkpoint> record
 Continue scanning backwards till a record <Ti start> is found.
 Need only consider the part of log following above start record. Earlier part
of log can be ignored during recovery, and can be erased whenever desired.
 For all transactions (starting from Ti or later) with no <T commit>, execute
undo(Ti). (Done only in case of immediate modification.)
 Scanning forward in the log, for all transactions starting from Ti or later
with a <Ti commit>, execute redo(Ti).
Example of Checkpoints

Tc Tf

T1

T2

T3

T4

checkpoint system failure

Fig 5.14 Check

point
 T1 can be ignored (updates already output to disk due to checkpoint)
 T2 and T3 redone.
 T4 undone

DATABASE SECURITY ISSUES

Introduction to DBA

• database administrators
 •use specialized tools for archiving , backup, etc
NOTES
•start/stop DBMS or taking DB offline (restructuring tables)
•establish user groups/passwords/access privileges
•backup and recovery
•secruity and integrity
•import/export (see interoperability)
•monitoring tuning for performance
Security
The advantage of having shared access to data is in fact a disadvantage also
•Consequences: loss of competitiveness, legal action from individual
•Restrictions
– Unauthorized users seeing data
– Corruption due to deliberate incorrect updates
– Corruption due to accidental incorrect updates
•Reading ability allocated to those who have a right to know
•Writing capabilities restricted for the casual user - who may accidentally
corrupt data due to lack of understanding
•Authorization is restricted to the chosen few to avoid deliberate corruption

Secure off-site storage standby hardware system

building equipment room peripheral equipment

non-computer-based non-computer-based
controls computer-based

data hardware
DBMS communication link user
O.S.

controls
controls

external communication link

Fig 5.15 Security Issues

Computer Based Countermeasures

•Authorisation
– determine user is who they claim to be
– Privileges
NOTES • Passwords
– low storage overhead
– many passwords and users forget them - write them down!!
– user time high - type in many passwords
– held in file and encrypted.
• Lists
– initial password entry to system
– user name checked against control list
– the access control list has very limited access, superuser
– if many users and applications and data then list can be large
• READ, WRITE, MODIFY access controls
• Restrictions at many levels of granularity
– Database Level: ‘add a new DB’
– Record Level: ‘delete a new record’
– Data Level: ‘delete an attribute’
• Remember there are overheads with security mechanisms
• views
– subschema
– dynamic result of one or more relational operations operating
on base relations to produce another relations
– virtual relation - doesn’t exist but is produce at runtime
• back-up
– periodic copy of database and log file (programs) onto
offline storage
– stored in secure location
• Journaling
– keeping log file of all changes made to database to enable
recovery in the event of failure
• Check pointing
– synchronization point where all buffers in the DBMS are
force- written to secondary storage
• integrity (see later)
• encryption
– data encoding by special algorithm that render data
unreadable without decryption key
 – degradation in performance
NOTES
– good for communication
•associated procedures
– specify procedures for authorization and backup/recovery
– audit : auditor observe manual and computer procedures
– installation/upgrade procedures
– contingency plan
– escrow agreement.
Non Computer Counter Measures
•establishment of security policy and contingency plan
•personnel controls
•secure positing of equipment, data and software
•escrow agreements (3rd party holds source code)
•maintenance agreements
•physical access controls
•building controls
•emergency arrangements.
ACCESS CONTROL
Privacy in Oracle
•user gets a password and user name
•Privileges :
– Connect : users can read and update tables (can’t create)
– Resource: create tables, grant privileges and control
auditing
– DBA: any table in complete DB
•user owns tables they create
•they grant other users privileges:
– Select : retrieval
– Insert : new rows
– Update : existing rows
– Delete : rows
– Alter : column def.
– Index : on tables
•owner can offer GRANT to other users as well
•this can be revoked
•Users can get audits of:
– list of successful/unsuccessful attempts to access tables
 NOTE – selective audit E.g. update only
S – control level of detail reported
• DBA has this and logon, logoff oracle, grants/revolts privilege
• Audit is stored in the Data Dictionary.
Integrity
The integrity of a database concerns
– consistency
– correctness
– validity
– accuracy

• that it should reflect real world

• i.e. it reflects the rules of the organization it models.
• rules = integrity rules or integrity constraints
examples
• accidental corruption of the DB,
• invalid PATIENT #
• non serializable schedules
• recovery manager not restoring DB correctly after failure
concepts
• trying to reflect in the DB rules which govern
organization E.g.:
INPATENT(patent#, name, DOB, address, sex, gp)
LABREQ (patent#, test-type, date, reqdr)
• E.g. rules
– lab. test can’t be requested for non-existent PATIENT (insert
to labreq) (referential)
– every PATIENT must have unique patent number (relation)
– PATIENT numbers are integers in the range 1-99999. (domain)
• real-world rules = integrity constraints
Integrity constraints
• Implicit Constraints - relation, domain, referential - integral part of
Relational model
– Relational constraints - define relation and attributes supported by
all RDBMS
 – Domain constraints - underlying domains on which attributes
defined
NOTES
– Referential constraints - attribute from one table referencing another
•Integrity subsystem: conceptually responsible for enforcing constraints, violation
+ action
– monitor updates to the databases, multi user system this can get
expensive
•integrity constraints shared by all applications, stored in systemcatalogue
defined by data definition
Relation constraints
•how we define relations in SQL • using CREATE
we give relation name and define attributes
E.g.
CREATE TABLE INPATIENT (PATIENT #, INTEGER, NOT NULL,
name, VARCHAR (20), NOT NULL,
. .....
gpn VARCHAR (20),
PRIMARY KEY PATIENT#);
Domain constraints • Domains are very
im portant to the relational model.
•through attributes defined over common domains that relationships between
tuples belonging to different relations can be defined.
•This also ensures consistency in typing.
E.g
CREATE TABLE (PATIENT# DOMAIN (PATIENT #) not NULL,
name DOMAIN (name) not
NULL, sex DOMAIN (sex),
PRIMARY KEY PATIENT #);
CREATE DOMAIN PATIENT# INTEGER PATIENT# > 0 PATIENT#
<10000;
CREATE DOMAIN sex CHAR (1) in (‘M’, ‘F’);
Referential integrity • refers
to
foreign keys • consequences for up
dates and deletes • 3 possibilities
for
– RESTRICTED: disallow the update/deletion of primary keys as long as
NOTE – CASCADES: update/deletion of the primary key has a cascading
S effect on all tuples whose foreign key references that primary key,
(and they too are deleted).
– NULLIFIES: update/deletion of the primary key results in the
referencing foreign keys being set to null.
FOREIGN KEY gpn REFERENCES gpname OF TABLE GPLIST
NULLS ALLOWED DELETION NULLIFIES UPDATE CASCADES;
Explicit (semantic) constraints
• defined by SQL but not widely implemented
• E.g.. can’t overdraw if you have a poor credit rating
ASSERT OVERDRAFT_CONSTRAINT ON customer,
account: account.balence >0
AND account.acc# = CUSTOMER.cust#
AND CUSTOMER.credit_rating = ‘poor’;
• Triggers
DEFINE TRIGGER reord-constraint ON STOCK
noinstock < reordlevel
ACTION ORDER_PROC (part#);
and Dynamic Constraints
• State or Transaction constraints
• static refer to legal DB states
• dynamic refer to legitimate transactions of the DB form one state to another
ASSERT payrise-constraint ON UPDATE OF
employee: NEW salary > OLD salary;
Security and Integrity Summary
• Security can protect the database against unauthorized users
• Integrity can protect it against authorized users
– Domain integrity rules: maintain correctness of attribute values in
relations
– Intra-relational Integrity: correctness of relationships among atts. in
same rel.
– Referential integrity: concerned with maintaining correctness and
consistency of relationships between relations.
Recovery and Concurrency
NOTES
•closely bound to the idea of Transaction Processing
•important in large multi-user centralized environment

Authorization in SQL

• File systems identify certain access privileges on files, E.g., read,

write, execute.
• In partial analogy, SQL identifies six access privileges on relations, of
which the most important are:
1. SELECT = the right to query the relation.
2. INSERT = the right to insert tuples into the relation – may refer to
one attribute, in which case the privilege is to specify only one
column of the inserted tuple.
3. DELETE = the right to delete tuples from the relation.
4. UPDATE = the right to update tuples of the relation – may refer to
one attribute.
Granting Privileges

• You have all possible privileges to the relations you create.

• You may grant privileges to any user if you have those privileges “with
grant option.”
u You have this option to your own relations.
Example
1. Here, Sally can query Sells and can change prices, but cannot pass on
this power:
GRANT SELECT ON Sells,
UPDATE(price) ON Sells
TO sally;
2. Here, Sally can also pass these privileges to whom she
chooses: GRANT SELECT ON Sells,
UPDATE(price) ON Sells
TO sally
WITH GRANT OPTION;
Revoking Privileges
• Your privileges can be revoked.
• Syntax is like granting, but REVOKE ... FROM instead of GRANT ... TO.
• Determining whether or not you have a privilege is tricky, involving
“grant diagrams” as in text. However, the basic principles are:
NOTE a) If you have been given a privilege by several different people,
S then all of them have to revoke in order for you to lose the
privilege.
b) Revocation is transitive.
Example :
If A granted P to B, who then granted P to C, and then A revokes P from B, it is
as if
B also revoked P from C.

Summary :
 A lock is a mechanism to control concurrent access to a data item
 THE TWO-PHASE LOCKING PROTOCOL ensures conflict-
serializable schedules. So in order to avoid conflicts between the
various schedules
 Failure Classifications are Transaction failure, System crash and Disk failure
 DBMS recovery subsystem are Active transactions , Committed
transac- tions since last checkpoint and Aborted transactions since last
checkpoint
 Recovery and Atomicity are log-based recovery, and shadow-paging
 Log record buffering is to buffer log records in main memory,
instead of being output directly to stable storage.
 Database maintains an in-memory buffer of data blocks
 The deferred database modification scheme records all modifications to
the log, but defers all the writes to after partial commit.
 The immediate database modification scheme allows database updates of
an uncommitted transaction to be made as the writes are issued
 Streamline recovery procedure by periodically performing check pointing
 Integrity constraints are Implicit Constraints , Relational constraints
,Domain constraints and Referential constraints

Questions :
1. Define Lock.
2. Define Two phase locking protocol.
3. Define Check point.
4. Explain System failures.
5. Explain Integrity Constraints.
6. Explain Database Modifications.
7. Expl ain Various Recovery procedures.
Answer Vividly :
NOTES
1. Explain Deadlock.
2. Explain DB Recovery System.
3. Explain DB security.
4. Explain Concurrency in DB system.

References :

1. Ramez Elamasri and shankant B.Navathe, “ Fundamentals of

Database Systems”, Third Edition , Pearson Education, Delhi, 2002.
2. Abraham Silberschatz, Henry F.Korth and S.Sundarshan ,
“Database System concepts “, Fourth Edition , McGrawHill ,
2002.
3. C.J.Date,”An Introduction to Database
Systems”,Seventh Edition,Pearson Education,Delhi,
2002.
 Edited with the trial version of
Foxit Advanced PDF Editor
To remove this notice, visit:
www.foxitsoftware.com/shopping

CS DATABASE MANAGEMENT SYSTEMS

6302
NOTE
NOTES
 Edited with the trial version of
Foxit Advanced PDF Editor
To remove this notice, visit:
www.foxitsoftware.com/shopping

CS DATABASE MANAGEMENT SYSTEMS

6302
NOTE
NOTES
 Edited with the trial version of
Foxit Advanced PDF Editor
To remove this notice, visit:
www.foxitsoftware.com/shopping

CS DATABASE MANAGEMENT SYSTEMS

6302

NOTE
NOTES