Chapter 2

Query Processing and Optimization

Chapter Outline

1. Introduction to Query Processing

2. Translating SQL Queries into Relational Algebra

3. Using Heuristics in Query Optimization

4. Using Selectivity and Cost Estimates in Query Optimization

5. Semantic Query Optimization

1. Introduction to Query Processing (1)

• Query Processing : the process that transforms a high-level query (SQL) into
an equivalent, correct and efficient execution plan expressed in low- level
language (such as physical query plan)

• Query Processing: the activities involved in Decomposition (parsing,

validating), optimizing, code generation, and executing a query.
1. Introduction to Query Processing (2)

• Query processing can be divided into four main phases:

– decomposition (consisting of parsing and validation),

– optimization,

– code generation, and

– execution,
Phases of query processing.
Introduction to Query Processing (3)

• Query decomposition is the first phase of query processing. The aims of query
decomposition are to transform a high-level query into a relational algebra
query,

• A relational algebra query: is a procedural query language, which takes

instances of relations as input and yields instances of relations as output.

• It uses operators (unary or binary) to perform queries.

Introduction to Query Processing - Query optimization (1)

• Query optimization:

– The process of choosing a suitable execution strategy for processing a query.

• An internal representation (query tree or query graph) of the query is created

after scanning, parsing, and validating.

• the aim of query optimization is to choose the one that minimizes resource usage.

• Generally, we try to reduce the total execution time of the query, which is the
sum of the execution times of all individual operations that make up the query
Introduction to Query Processing - Query optimization (2)

• Dynamic versus Static optimization: There are two choices for when the first three phases
of query processing can be carried out.

– One option is to dynamically carry out decomposition and optimization every time the
query is run.

– The alternative option is static query optimization, where the query is parsed,
validated, and optimized once.

• However, a hybrid approach could be used to overcome this disadvantage, where the
query is re-optimized if the system detects that the database statistics have changed
significantly since the query was last compiled.
Introduction to Query Processing - Query optimization (3)
Relational algebra query (1)

• A relational algebra query: is a procedural query language, which takes instances

of relations as input and yields instances of relations as output.
• It uses operators (unary or binary) to perform queries. They accept relations as
their input and yield relations as their output.
• The fundamental operations of relational algebra are as follows −

• Select • Set different

• Project • Cartesian product and
• Union • Rename.
Relational algebra query (2)
• Select Operation (σ): It selects tuples that satisfy the given predicate from a relation.

– Notation − σp(r)

– Where σ stands for selection predicate and r stands for relation. p is prepositional logic formula.

• Example: σsubject = "database"(Books)

• Project Operation (∏): It projects column(s) that satisfy a given predicate.
– Notation − ∏A1, A2, An (r)

– Where A1, A2 , An are attribute names of relation r.

– Duplicate rows are automatically eliminated, as relation is a set.

• Example: ∏subject, author (Books)

Relational algebra query (3)
• Union Operation (∪): It performs binary union between two given relations and is defined
as − r ∪ s = { t | t ∈ r or t ∈ s}

– Notation − r U s

– Where r and s are either database relations or relation result set (temporary relation).

– Example: ∏ author (Books) ∪ ∏ author (Articles)

• Set Difference (−): The result of set difference operation is tuples, which are present in
one relation but are not in the second relation.
– Notation − r −s
– Finds all the tuples that are present in r but not in s.
– Example: ∏ author (Books) − ∏ author (Articles)
Relational algebra query (4)
• Cartesian Product (Χ): Combines information of two different relations into one.
– Notation − r Χ s
– Where r and s are relations and their output will be defined as −
• r Χ s = { q t | q ∈ r and t ∈ s}
– Example: σauthor = ‘Jonathan’(Books Χ Articles)

• Rename Operation (ρ)(rho): The rename operation allows us to rename the output
relation.
– Notation − ρ x (E)

– Where the result of expression E is saved with name of x.

• Additional operations are: Set intersection, Assignment and Natural join
Aggregate Functions and Grouping

• A type of request that cannot be expressed in the basic relational algebra is to specify
mathematical aggregate functions on collections of values from the database

• Examples of such functions include retrieving the average or total salary of all employees
or the total number of employee tuples.

• Common functions applied to collections of numeric values include SUM, AVERAGE,

MAXIMUM, and MINIMUM. The COUNT function is used for counting tuples or values.

• Use of the Functional operator ℱ

– ℱMAX Salary (Employee) retrieves the maximum salary value from the Employee
relation
Algorithms for Executing Query Operations
• Translating SQL Queries into Relation Algebra

• Algorithms for External Sorting

• Algorithms for SELECT and JOIN Operations

• Algorithms for PROJECT and SET Operations

• Implementing Aggregate Operations and OUTER JOINS

• Combining Operations Using Pipelining

• Using Heuristics in Query Optimization

• Using Selectivity and Cost Estimates in Query Optimization

2. Translating SQL Queries into Relational Algebra (1)

• Query block:

– The basic unit that can be translated into the algebraic operators and
optimized.

• A query block contains a single SELECT-FROM-WHERE expression, as well as

GROUP BY and HAVING clause if these are part of the block.

• Nested queries within a query are identified as separate query blocks.

• Aggregate operators in SQL must be included in the extended algebra.

2. Translating SQL Queries into Relational Algebra (2)

SELECT LNAME, FNAME

FROM EMPLOYEE
WHERE SALARY > ( SELECT MAX (SALARY)
FROM EMPLOYEE
WHERE DNO = 5);

SELECT LNAME, FNAME SELECT MAX (SALARY)

FROM EMPLOYEE FROM EMPLOYEE
WHERE SALARY > C WHERE DNO = 5

πLNAME, FNAME (σSALARY>C(EMPLOYEE)) ℱMAX SALARY (σDNO=5 (EMPLOYEE))

3. Using Heuristics in Query Optimization(1)
• There are two main techniques for implementing query optimization.

– Heuristic rules for re-ordering the operations in a query.

– Systematically estimating the cost of different execution strategies and choosing the
lowest cost estimate.

• The heuristical approach to query optimization, uses transformation rules to convert one
relational algebra expression into an equivalent form that is known to be more efficient.
3. Using Heuristics in Query Optimization(2)
Process for heuristics optimization
1. The parser of a high-level query generates an initial internal representation;
2. Apply heuristics rules to optimize the internal representation.
3. A query execution plan is generated to execute groups of operations based on the
access paths available on the files involved in the query.

• The main heuristic is to apply first the operations that reduce the size of intermediate
results.
– E.g., Apply SELECT and PROJECT operations before applying the JOIN or other binary
operations.
Using Heuristics in Query Optimization (3)
• Query tree:
– A tree data structure that corresponds to a relational algebra expression. It represents the
input relations of the query as leaf nodes of the tree, and represents the relational algebra
operations as internal nodes.

• An execution of the query tree consists of executing an internal node operation whenever
its operands are available and then replacing that internal node by the relation that
results from executing the operation.
• Query graph:
– A graph data structure that corresponds to a relational calculus expression. It does not
indicate an order on which operations to perform first. There is only a single graph
corresponding to each query.
Using Heuristics in Query Optimization (4)
Example:
– For every project located in ‘Stafford’, retrieve the project number, the
controlling department number and the department manager’s last name,
address and birthdate.
• Relation algebra:
PNUMBER, DNUM, LNAME, ADDRESS, BDATE (((PLOCATION=‘STAFFORD’(PROJECT))
DNUM=DNUMBER (DEPARTMENT)) MGRSSN=SSN (EMPLOYEE))

SELECT P.NUMBER,P.DNUM,E.LNAME,
E.ADDRESS, E.BDATE
FROM PROJECT AS P,DEPARTMENT AS D, EMPLOYEE AS E
WHERE P.DNUM=D.DNUMBER AND D.MGRSSN=E.SSN AND
P.PLOCATION=‘STAFFORD’;
Using Heuristics in Query Optimization (5)
Using Heuristics in Query Optimization (6)
• Heuristic Optimization of Query Trees:
– The same query could correspond to many different relational algebra expressions — and
hence many different query trees.

– The task of heuristic optimization of query trees is to find a final query tree that is efficient to
execute.

Example:
Q: SELECT LNAME

FROM EMPLOYEE, WORKS_ON, PROJECT

WHERE PNAME = ‘AQUARIUS’ AND PNMUBER=PNO AND ESSN=SSN AND

BDATE > ‘1957-12-31’;
Using Heuristics in Query Optimization (7)

Slide 15- 24
Using Heuristics in Query Optimization (8)
Using Heuristics in Query Optimization (9)

General Transformation Rules for Relational Algebra Operations:

1. Cascade of : A conjunctive selection condition can be broken up into a cascade (sequence) of individual 
operations:

–  c1 AND c2 AND ... AND cn(R) = c1 (c2 (...(cn(R))...) )

2. Commutativity of : The  operation is commutative:

– c1 (c2(R)) = c2 (c1(R))

3. Cascade of : In a cascade (sequence) of  operations, all but the last one can be ignored:

– List1 (List2 (...(Listn(R))...) ) = List1(R)

4. Commuting  with : If the selection condition c involves only the attributes A1, ..., An in the projection list, the
two operations can be commuted:

– A1, A2, ..., An (c (R)) = c (A1, A2, ..., An (R))

Using Heuristics in Query Optimization (10)

General Transformation Rules for Relational Algebra Operations (contd.):

5. Commutativity of ( and x ): The operation is commutative as is the x operation:
– R C S=S C R; R x S = S x R

6. Commuting  with (or x ): If all the attributes in the selection condition c involve only the
attributes of one of the relations being joined—say, R—the two operations can be commuted as
follows:

– c ( R S ) = (c (R)) S
• Alternatively, if the selection condition c can be written as (c1 and c2), where condition c1 involves
only the attributes of R and condition c2 involves only the attributes of S, the operations commute
as follows:

– c ( R S ) = (c1 (R)) (c2 (S))

Using Heuristics in Query Optimization (11)
General Transformation Rules for Relational Algebra Operations (contd.):

7. Commuting  with (or x): Suppose that the projection list is L = {A1, ..., An, B1, ..., Bm},
where A1, ..., An are attributes of R and B1, ..., Bm are attributes of S. If the join condition c
involves only attributes in L, the two operations can be commuted as follows:

– L ( R C S ) = (A1, ..., An (R)) C ( B1, ..., Bm (S))

• If the join condition C contains additional attributes not in L, these must be added to the
projection list, and a final  operation is needed.
Using Heuristics in Query Optimization (12)

General Transformation Rules for Relational Algebra Operations (contd.):

8. Commutativity of set operations: The set operations υ and ∩ are commutative but “–” is not.

E1  E2 = E2  E1
E1  E2 = E2  E1

9. Associativity of , x, υ, and ∩ : These four operations are individually associative; that is, if q
stands for any one of these four operations (throughout the expression), we have

– (RqS)qT = Rq(SqT)

10. Commuting  with set operations: The  operation commutes with υ , ∩ , and –. If q stands for
any one of these three operations, we have

– c ( R q S ) = (c (R)) q (c (S))

Using Heuristics in Query Optimization (13)

General Transformation Rules for Relational Algebra Operations (contd.):

(11). The p operation commutes with υ.

L ( R υ S ) = (L (R)) υ (L (S))

(12). Converting a (s, x) sequence into

: If the condition c of a  that follows a x Corresponds to a join condition, convert the (,
x) sequence into a as follows:

(C (R x S)) = (R C S)
Using Heuristics in Query Optimization (14)
Outline of a Heuristic Algebraic Optimization Algorithm:

1. Using rule 1, break up any select operations with conjunctive conditions into a cascade of select operations.

2. Using rules 2, 4, 6, and 10 concerning the commutativity of select with other operations, move each select
operation as far down the query tree as is permitted by the attributes involved in the select condition.

3. Using rule 9 concerning associativity of binary operations, rearrange the leaf nodes of the tree so that the leaf
node relations with the most restrictive select operations are executed first in the query tree representation.

4. Using Rule 12, combine a Cartesian product operation with a subsequent select operation in the tree into a join
operation.

5. Using rules 3, 4, 7, and 11 concerning the cascading of project and the commuting of project with other
operations, break down and move lists of projection attributes down the tree as far as possible by creating new
project operations as needed.

6. Identify subtrees that represent groups of operations that can be executed by a single algorithm.
Using Heuristics in Query Optimization (15)
Summary of Heuristics for Algebraic Optimization:
1. The main heuristic is to apply first the operations that reduce the size of intermediate results.

2. Perform select operations as early as possible to reduce the number of tuples and perform
project operations as early as possible to reduce the number of attributes. (This is done by
moving select and project operations as far down the tree as possible.)

3. The select and join operations that are most restrictive should be executed before other
similar operations. (This is done by reordering the leaf nodes of the tree among themselves
and adjusting the rest of the tree appropriately.)
Using Heuristics in Query Optimization (16)

Query Execution Plans

– An execution plan for a relational algebra query consists of a combination

of the relational algebra query tree and information about the access
methods to be used for each relation as well as the methods to be used
in computing the relational operators stored in the tree.
4. Using Selectivity and Cost Estimates in Query Optimization (1)

• Cost-based query optimization:

– Estimate and compare the costs of executing a query using different
execution strategies and choose the strategy with the lowest cost estimate.

– Issues
• Cost function

• Number of execution strategies to be considered

Using Selectivity and Cost Estimates in Query Optimization (2)

Cost Components for Query Execution

1. Access cost to secondary storage

2. Storage cost

3. Computation cost

4. Memory usage cost

5. Communication cost
Note: Different database systems may focus on different cost components.
Using Selectivity and Cost Estimates in Query Optimization (3)
• Catalog Information Used in Cost Functions
– Information about the size of a file
• number of records (tuples) (r),
• record size (R),
• number of blocks (b)
• blocking factor (bfr)

– Information about indexes and indexing attributes of a file

• Number of levels (x) of each multilevel index
• Number of first-level index blocks (bI1)
• Number of distinct values (d) of an attribute
• Selectivity (sl) of an attribute
• Selection cardinality (s) of an attribute. (s = sl * r)
5. Semantic Query Optimization
• Semantic Query Optimization:
– Uses constraints specified on the database schema in order to modify one query into another query
that is more efficient to execute.
• Consider the following SQL query,
SELECT E.LNAME, M.LNAME
FROM EMPLOYEE E M
WHERE E.SUPERSSN=M.SSN AND E.SALARY>M.SALARY
• Explanation:
– Suppose that we had a constraint on the database schema that stated that no employee can earn more
than his or her direct supervisor. If the semantic query optimizer checks for the existence of this
constraint, it need not execute the query at all because it knows that the result of the query will be
empty. Techniques known as theorem proving can be used for this purpose.
Group Assignment

• Select n(number of students in your group) SQL queries

• Translate them into a relational algebra expressions

• Build query trees for each relational algebra expression created

• NB: marks depend on how complex your queries are