Chapter 6: Query Decomposition and Data Localization

Query Decomposition Data Localization

Acknowledgements: I am indebted to Arturas Mazeika for providing me his slides of this course.

DDB 2008/09

J. Gamper

Page 1

Query Decomposition

Query decomposition: Mapping of calculus query (SQL) to algebra operations (select, project, join, rename)

Both input and output queries refer to global relations, without knowledge of the distribution of data.

The output query is semantically correct and

good in the sense that redundant work is avoided.

Query decomposistion consists of 4 steps:

1. Normalization: Transform query to a normalized form 2. Analysis: Detect and reject incorrect queries; possible only for a subset of relational calculus 3. Elimination of redundancy: Eliminate redundant predicates 4. Rewriting: Transform query to RA and optimize query
DDB 2008/09 J. Gamper Page 2

Query Decomposition Normalization

Normalization: Transform the query to a normalized form to facilitate further processing.
Consists mainly of two steps. 1. Lexical and syntactic analysis Check validity (similar to compilers) Check for attributes and relations Type checking on the qualication 2. Put into normal form With SQL, the query qualication (WHERE clause) is the most difcult part as it might be an arbitrary complex predicate preceeded by quantiers (, ) Conjunctive normal form

(p11 p12 p1n ) (pm1 pm2 pmn )

Disjunctive normal form

(p11 p12 p1n ) (pm1 pm2 pmn )

In the disjunctive normal form, the query can be processed as independent conjunctive subqueries linked by unions (corresponding to the disjunction)
DDB 2008/09 J. Gamper Page 3

Query Decomposition Normalization . . .

Example: Consider the following query: Find the names of employees who have been
working on project P1 for 12 or 24 months?

The query in SQL: SELECT ENAME FROM EMP, ASG WHERE EMP.ENO = ASG.ENO AND ASG.PNO = P1 AND DUR = 12 OR DUR = 24 The qualication in conjunctive normal form:
EM P.EN O = ASG.EN O ASG.P N O = P 1 (DU R = 12 DU R = 24)

The qualication in disjunctive normal form:

(EM P.EN O = ASG.EN O ASG.P N O = P 1 DU R = 12) (EM P.EN O = ASG.EN O ASG.P N O = P 1 DU R = 24)
DDB 2008/09 J. Gamper Page 4

Query Decomposition Analysis

Analysis: Identify and reject type incorrect or semantically incorrect queries Type incorrect
Checks whether the attributes and relation names of a query are dened in the global schema Checks whether the operations on attributes do not conict with the types of the attributes, e.g., a comparison > operation with an attribute of type string

Semantically incorrect
Checks whether the components contribute in any way to the generation of the result Only a subset of relational calculus queries can be tested for correctness, i.e., those that do not contain disjunction and negation Typical data structures used to detect the semantically incorrect queries are: Connection graph (query graph) Join graph

DDB 2008/09

J. Gamper

Page 5

Query Decomposition Analysis . . .

Example: Consider a query:
SELECT FROM WHERE AND AND AND AND

ENAME,RESP EMP, ASG, PROJ EMP.ENO = ASG.ENO ASG.PNO = PROJ.PNO PNAME = "CAD/CAM" DUR 36 TITLE = "Programmer"

Query/connection graph
Nodes represent operand or result relation Edge represents a join if both connected nodes represent an operand relation, otherwise it is a projection

Join graph
a subgraph of the query graph that considers only the joins

Since the query graph is connected, the query is semantically correct

DDB 2008/09 J. Gamper Page 6

Query Decomposition Analysis . . .

Example: Consider the following query and its query graph:

SELECT FROM WHERE AND AND AND

ENAME,RESP EMP, ASG, PROJ EMP.ENO = ASG.ENO PNAME = "CAD/CAM" DUR 36 TITLE = "Programmer"

Since the graph is not connected, the query is semantically incorrect. 3 possible solutions:
Reject the query Assume an implicit Cartesian Product between ASG and PROJ Infer from the schema the missing join predicate ASG.PNO = PROJ.PNO

DDB 2008/09

J. Gamper

Page 7

Query Decomposition Elimination of Redundancy

Elimination of redundancy: Simplify the query by eliminate redundancies, e.g.,
redundant predicates Redundancies are often due to semantic integrity constraints expressed in the query language e.g., queries on views are expanded into queries on relations that satiesfy certain integrity and security constraints

Transformation rules are used, e.g., p p p p p p p true p p f alse p p f alse f alse p true true p p f alse p p true p1 (p1 p2 ) p1 p1 (p1 p2 ) p1
DDB 2008/09 J. Gamper Page 8

Query Decomposition Elimination of Redundancy . . .

Example: Consider the following query: SELECT TITLE FROM EMP WHERE EMP.ENAME = "J. Doe" OR (NOT(EMP.TITLE = "Programmer") AND ( EMP.TITLE = "Elect. Eng." OR EMP.TITLE = "Programmer" ) AND NOT(EMP.TITLE = "Elect. Eng.")) Let p1 be ENAME = J. Doe, p2 be TITLE = Programmer and p3 be TITLE = Elect.
Eng.

Then the qualication can be written as p1 (p2 (p2 p3 ) p3 ) and then be transformed into p1 Simplied query:
SELECT FROM WHERE

TITLE EMP EMP.ENAME = "J. Doe"

J. Gamper Page 9

DDB 2008/09

Query Decomposition Rewriting

Rewriting: Convert relational calculus query to relational algebra query and nd an
efcient expression.

Example: Find the names of employees other

than J. Doe who worked on the CAD/CAM project for either 1 or 2 years.

SELECT FROM WHERE AND AND AND AND

ENAME EMP, ASG, PROJ EMP.ENO = ASG.ENO ASG.PNO = PROJ.PNO ENAME = "J. Doe" PNAME = "CAD/CAM" (DUR = 12 OR DUR = 24)

A query tree represents the RA-expression

Relations are leaves (FROM clause) Result attributes are root (SELECT clause) Intermediate leaves should give a result from the leaves to the root

DDB 2008/09

J. Gamper

Page 10

Query Decomposition Rewriting . . .

By applying transformation rules, many different trees/expressions may be found that
are equivalent to the original tree/expression, but might be more efcient.

In the following we assume relations R(A1 , . . . , An ), S (B1 , . . . , Bn ), and T which is union-compatible to R. Commutativity of binary operations RS =SR R S=S R RS =SR Associativity of binary operations (R S ) T = R (S T ) (R S) T =R (S T) Idempotence of unary operations A (A (R)) = A (R) p1(A1) (p2(A2) (R)) = p1(A1)p2(A2) (R)
DDB 2008/09 J. Gamper Page 11

Query Decomposition Rewriting . . .

Commuting selection with binary operations p(A) (R S ) p(A) (R) S

p(A1 ) (R p(A2 ,B2 ) S ) p(A1 ) (R) p(A2 ,B2 ) S p(A) (R T ) p(A) (R) p(A) (T ) (A belongs to R and T )

Commuting projection with binary operations (assume C = A B , A A, B B ) C (R S ) A (R) B (S ) C (R p(A ,B ) S ) A (R) p(A ,B ) B (S )

C (R S ) C (R) C (S )

DDB 2008/09

J. Gamper

Page 12

Query Decomposition Rewriting . . .

Example: Two equivalent query trees for the previous example
Recall the schemas: EMP(ENO, ENAME, TITLE) PROJ(PNO, PNAME, BUDGET) ASG(ENO, PNO, RESP, DUR)

DDB 2008/09

J. Gamper

Page 13

Query Decomposition Rewriting . . .

Example (contd.): Another equivalent query tree, which allows a more efcient query
evaluation, since the most selective operations are applied rst.

DDB 2008/09

J. Gamper

Page 14

Data Localization

Data localization
Input: Algebraic query on global conceptual schema Purpose: Apply data distribution information to the algebra operations and determine which fragments are involved Substitute global query with queries on fragments Optimize the global query

DDB 2008/09

J. Gamper

Page 15

Data Localization . . .
Example:
Assume EMP is horizontally fragmented into EMP1, EMP2, EMP3 as follows:

EM P 1 = EN OE 3 (EM P ) EM P 2 = E 3<EN OE 6 (EM P ) EM P 3 = EN O>E 6 (EM P )

ASG fragmented into ASG1 and ASG2 as follows:

ASG1 = EN OE 3 (ASG) ASG2 = EN O>E 3 (ASG) Simple approach: Replace in all queries EMP by (EMP1EMP2 EMP3) ASG by (ASG1ASG2)
Result is also called generic query

In general, the generic query is inefcient since important restructurings and

simplications can be done.
DDB 2008/09 J. Gamper Page 16

Data Localization . . .
Example (contd.): Parallelsim in the evaluation is often possible
Depending on the horizontal fragmentation, the fragments can be joined in parallel followed by the union of the intermediate results.

DDB 2008/09

J. Gamper

Page 17

Data Localization . . .
Example (contd.): Unnecessary work can be eliminated e.g., EM P3 ASG1 gives an empty result EM P 3 = EN O>E 6 (EM P ) ASG1 = EN OE 3 (ASG)

DDB 2008/09

J. Gamper

Page 18

Data Localizations Issues

Various more advanced reduction techniques are possible to generate simpler and
optimized queries.

Reduction of horizontal fragmentation (HF)

Reduction with selection Reduction with join

Reduction of vertical fragmentation (VF)

Find empty relations

DDB 2008/09

J. Gamper

Page 19

Data Localizations Issues Reduction of HF

Reduction with selection for HF Consider relation R with horizontal fragmentation F = {R1 , R2 , . . . , Rk }, where Ri = pi (R) Rule1: Selections on fragments, pj (Ri ), that have a qualication contradicting the
qualication of the fragmentation generate empty relations, i.e.,

pj (Ri ) = x R(pi (x) pj (x) = f alse)

Can be applied if fragmentation predicate is inconsistent with the query selection predicate.

Example: Consider the query: SELECT * FROM EMP WHERE ENO=E5

After commuting the selection with the union operation, it is easy to detect that the selection predicate contradicts the predicates of EMP1 and EMP3 .
DDB 2008/09 J. Gamper Page 20

Data Localizations Issues Reduction for HF . . .

Reduction with join for HF
Joins on horizontally fragmented relations can be simplied when the joined relations are fragmented according to the join attributes. Distribute join over union

(R 1 R 2 ) S (R1 S ) (R 2 S)
Rule 2: Useless joins of fragments, Ri = pi (R) and Rj = pj (R), can be determined when the qualications of the joined fragments are contradicting, i.e.,

Ri Rj = x Ri , y Rj (pi (x) pj (y ) = f alse) pi ,p1 Ri R1

J. Gamper

Ri R1 R2 R3
DDB 2008/09

pi ,p2 Ri R2

pi ,p3 Ri R3
Page 21

Data Localizations Issues Reduction for HF . . .

Example: Consider the following query and fragmentation:
Query: SELECT * FROM EMP, ASG WHERE EMP.ENO=ASG.ENO Horizontal fragmentation: EM P 1 = EN OE 3 (EM P ) EM P 2 = E 3<EN OE 6 (EM P ) EM P 3 = EN O>E 6 (EM P )

ASG1 = EN OE 3 (ASG) ASG2 = EN O>E 3 (ASG)

Generic query

The query reduced by distributing joins over unions and applying rule 2 can be implemented as a union of three partial joins that can be done in parallel.

DDB 2008/09

J. Gamper

Page 22

Data Localizations Issues Reduction for HF . . .

Reduction with join for derived HF
The horizontal fragmentation of one relation is derived from the horizontal fragmentation of another relation by using semijoins.

If the fragmentation is not on the same predicate as the join (as in the previous
example), derived horizontal fragmentation can be applied in order to make efcient join processing possible.

Example: Assume the following query and fragmentation of the EMP relation:
Query: SELECT * FROM EMP, ASG WHERE EMP.ENO=ASG.ENO Fragmentation (not on the join attribute): EMP1 = TITLE=Prgrammer (EMP)

EMP2 = TITLE=Prgrammer (EMP)

To achieve efcient joins ASG can be fragmented as follows: ASG1= ASG<EN O EMP1 ASG2= ASG<EN O EMP2 The fragmentation of ASG is derived from the fragmentation of EMP Queries on derived fragments can be reduced, e.g., ASG1
DDB 2008/09 J. Gamper

EM P2 =
Page 23

Data Localizations Issues Reduction for VF

Reduction for Vertical Fragmentation

Recall, VF distributes a relation based on projection, and the reconstruction operator is the join. Similar to HF, it is possible to identify useless intermediate relations, i.e., fragments that do not contribute to the result. Assume a relation R(A) with A Ri = A (R), where A i A.
i

= {A1 , . . . , An }, which is vertically fragmented as

Rule 3: D,K (Ri ) is useless if the set of projection attributes D is not in A i and K is the key attribute. Note that the result is not empty, but it is useless, as it contains only the key attribute.

DDB 2008/09

J. Gamper

Page 24

Data Localizations Issues Reduction for VF . . .

Example: Consider the following query and vertical fragmentation:
Query: SELECT ENAME FROM EMP Fragmentation:

EM P 1 = EN O,EN AM E (EM P ) EM P 2 = EN O,T IT LE (EM P )

Generic query

Reduced query
By commuting the projection with the join (i.e., projecting on ENO, ENAME), we can see that the projection on EMP2 is useless because ENAME is not in EMP2 .

DDB 2008/09

J. Gamper

Page 25

Conclusion

Query decomposition and data localization maps calculus query into algebra operations
and applies data distribution information to the algebra operations.

Query decomposition consists of normalization, analysis, elimination of redundancy, and

rewriting.

Data localization reduces horizontal fragmentation with join and selection, and vertical
fragmentation with joins, and aims to nd empty relations.

DDB 2008/09

J. Gamper

Page 26