Orthogonal Optimization of Subqueries and Aggregation

César A. Galindo-Legaria Milind M. Joshi

{cesarg,milindj}@microsoft.com
Microsoft Corp.
One Microsoft Way
Redmond, WA 98052

ABSTRACT menting more primitive, independent optimizations that col-

There is considerable overlap between strategies proposed lectively generate efficient execution plans.
for subquery evaluation, and those for grouping and aggre- In this paper, we present subquery and aggregation tech-
gation. In this paper we show how a number of small, inde- niques implemented in Microsoft SQL Server. It is organized
pendent primitives generate a rich set of efficient execution as follows. First, we review standard proposals for subquery
strategies —covering standard proposals for subquery evalu- evaluation, and describe how they reduce to other primitive
ation suggested in earlier literature. These small primitives optimizations. Then we review the algebraic representation
fall into two main, orthogonal areas: Correlation removal, of correlation, or use of parameterized subexpressions. In
and efficient processing of outerjoins and GroupBy. An op- Section 2 we focus on representation and normalization of
timization approach based on these pieces provides syntax- subqueries, with the goal of replacing correlations by stan-
independence of query processing with respect to subqueries, dard relational algebra operators. In Section 3 we describe
i. e. equivalent queries written with or without subquery techniques for efficient execution of queries with aggrega-
produce the same efficient plan. tion. In Section 4 we give an overview of how techniques
We describe techniques implemented in Microsoft SQL described earlier fit into the architecture of our query pro-
Server (releases 7.0 and 8.0) for queries containing sub- cessor. We present performance results of our approach in
queries and/or aggregations, based on a number of orthog- Section 5, with data from current published TPC-H results.
onal optimizations. We concentrate separately on removing Section 6 concludes the paper.
correlated subqueries, also called “query flattening,” and on 1.1 Standard subquery execution strategies
efficient execution of queries with aggregations. The end re-
sult is a modular, flexible implementation, which produces Before describing subquery strategies in detail, it is impor-
very efficient execution plans. To demonstrate the validity tant to clarify the two forms of aggregation in SQL, whose
of our approach, we present results for some queries from behavior diverges on an empty input. “Vector” aggregation
the TPC-H benchmark. From all published TPC-H results specifies grouping columns as well as aggregates to com-
in the 300GB scale, at the time of writing (November 2000), pute.1 For example, obtaining the total sales per date:
SQL Server has the fastest results on those queries, even on select o orderdate, sum(o totalprice)
a fraction of the processors used by other systems. from orders
group by o orderdate
1. INTRODUCTION If orders is empty, the result of the query is also empty.
Subqueries are a convenient and succinct construct of the “Scalar” aggregation on the other hand, does not specify
SQL language, which has been implemented by commercial grouping columns. For example, get the total sales in the
database systems for a number of years. They are a stan- table:
dard mechanism used frequently by real applications, and
researchers have worked on their efficient evaluation, propos- select sum(o totalprice) from orders
ing powerful techniques. Here, we make the observation that
there is significant overlap between techniques proposed for This second query always returns exactly one row. The re-
subquery execution and others such as GroupBy evaluation. sult value on an empty input depends on the aggregate; for
Therefore we take the approach of identifying and imple- sum it is null, while for count it is 0 [13]. In algebraic ex-
pressions we denote vector aggregate as GA,F , where A are
the grouping columns and F are the aggregates to compute;
and denote scalar aggregate as GF1 .
Permission to make digital or hard copies of all or part of this work for We review standard subquery execution strategies using
personal or classroom use is granted without fee provided that copies are the following SQL query, which finds customers who have
not made or distributed for profit or commercial advantage and that copies ordered more than $1,000,000. The subquery uses a scalar
bear this notice and the full citation on the first page. To copy otherwise, to 1
republish, to post on servers or to redistribute to lists, requires prior specific The use of distinct for duplicate removal is a special case
permission and/or a fee. of “vector aggregate,” collapsing groups with equal values
ACM SIGMOD 2001 May 21-24, Santa Barbara, California, USA into a single row, but without actual aggregate functions to
Copyright 2001 ACM 1-58113-332-4/01/05 ...$5.00. compute. We normalize distinct as GroupBy.

571
aggregate to compute the total ordered by a customer. It SQL query
is called a “correlated subquery” because it uses parameters
resolved from a table outside of the subquery, in this case
column c custkey. Our examples use the straightforward Algebrize
data schema of the TPC-H benchmark.
Correlated strategy
Q1: select c custkey
from customer
where 1000000 < Remove correlations Reorder GroupBy
(select sum(o totalprice)
from orders Agg/Outerjoin Outerjoin/Agg
where o custkey = c custkey)
Simplify Outerjoin Reorder GroupBy
Correlated execution. The execution that is closest to the
SQL formulation is to take each customer and compute a Agg/Join Join/Agg
total amount, as specified in the subquery, then filter out
customers who have ordered less than the specified amount. Figure 1: Primitives connecting different execution
This is commonly considered an inferior strategy, as it in- strategies.
volves per-row processing of customers, instead of a set ori-
ented execution —but it can actually be the best strategy,
if the outer table is small, and appropriate indices exist.

Outerjoin, then aggregate. This execution strategy was a subquery usage to either of the above strategies, we use or-
original proposed by Dayal [5]. To use set-oriented algo- thogonal, reusable primitives that lead to those listed above,
rithms, we can first collect all orders for each customer, then and more. And use cost-estimation to choose between them.
aggregate grouping by customer, and finally filter based on Figure 1 shows how primitive optimizations lead to differ-
the aggregate result. The corresponding SQL formulation is ent “subquery strategies.” The above standard strategies
as follows. are only two of the possible boxes in the diagram; the other
strategies shown are also feasible, and will sometimes be
select c custkey superior, depending on data distribution and indices avail-
from customer left outer join able. There are a number of optimizations that can be done
orders on o custkey = c custkey on GroupBy queries that are not shown in this figure, but
group by c custkey are discussed later in Section 3. By implementing all these
having 1000000 < sum(o totalprice) orthogonal techniques, the query processor should then pro-
duce the same efficient execution plan for the various equiv-
The use of outerjoin is faithful to the correlated execution alent SQL formulations we have listed above, achieving a
semantics: The correlated subquery uses scalar aggregation degree of syntax-independence. The basic transformations
and therefore returns exactly one row for each customer, shown in the figure are described next.
even if there are no qualifying orders. Using outerjoin, cus-
tomers with no matching orders are preserved, and the ag- Algebrize into initial operator tree. Our algebraic rep-
gregation result on such non-matching row is null. resentation of correlations is based on the Apply operator.
Apply is a second-order relational construct that abstracts
Aggregate, then join. This execution strategy was original parameterized execution of subexpressions. We elaborate
proposed by Kim [11]. It is possible to aggregate directly more on this operator below, in Section 1.3.
over the orders table to obtain the total sales per customer
and later join with the customer table. This alse allows Remove correlations. Removal of correlation consists of
pushing the aggregate condition below the join. The SQL rewriting Apply into other operators such as outerjoin. In
formulation uses a derived table, not a subquery. Section 2.3 we go into the details of how this is achieved.
The result of correlational removal, on the example query
select c custkey we used early is exactly the strategy proposed by Dayal.
from customer,
(select o custkey from orders Simplify outerjoin. Simplification of outerjoin into join,
group by c custkey under null-rejecting conditions, is elaborated in [7]. We use
having 1000000 < sum(o totalprice)) the same framework, but add derivation of null-rejection
as AggResult in GroupBy operators, which is not covered in that work.
where o custkey = c custkey Correlation removal typically results in outerjoins, which are
then simplified into joins, when possible. In our earlier ex-
1.2 Our technique: Use primitive, orthogonal ample, the condition 1000000 < x rejects a null value of
pieces x, which triggers simplification of outerjoin into join.
From those listed above, the most efficient strategy de-
pends on size of tables, selectivity of conditions, and reduc- Reorder GroupBy. Reordering GroupBy around joins was
tion factor of aggregation. Rather than going directly from presented in [3, 18]. In our implementation, we build on that

572
 SELECT(1000000<X) join [1]; at Oregon Graduate Institue and Portland State
University, called d-join [17]; and at Microsoft [6]. It is in-
teresting to note that those three groups were all working
with an optimizer developed from the Cascades query opti-
APPLY(bind:C_CUSTKEY) mizer of Goetz Graefe [8].

Apply works on expressions that take scalar (or row-

valued) parameters. A second useful construct is Segmen-
CUSTOMER SGb(X=SUM(O_TOTALPRICE) tApply, which deals with expressions using table-valued pa-
rameters. It takes a relational input R, a parameterized ex-
pression E(S), and a set of segmenting columns A from R. It
creates segments of R using columns A, much like GroupBy
SELECT(O_CUSTKEY=C_CUSTKEY) does, and for each such segment S it executes E(S). For-
mally,
[
R SAA E = ({a} × E(σA=a R)) ,
a
ORDERS
where a takes all values in the domain of A.
Figure 2: Subquery execution using Apply. These higher order constructs do not add expressive power
to the standard relational operators ×, σ, π, −, ∪, plus op-
erator GroupBy required for SQL. SegmentApply can be
rewritten in terms of Apply, and any expression containing
work by adding reordering around outerjoins, and operating standard operators plus Apply can be rewritten in terms of
based on abstract properties of aggregate functions, rather standard operators only [6]. The constructs do remain a
than considering the five standard SQL aggregates. useful tool for query processing, facilitating query represen-
tation (Apply maps directly to a tuple-at-a-time formulation
1.3 A useful tool: Represent parameterized style) and significantly enhancing the space of alternatives
execution algebraically considered for execution. This is achieved in an algebraic
A convenient tool in the development of our orthogonal fashion, allowing both formal algebraic manipulation as well
optimizations is the algebraic representation of parameter- as smooth integration in algebraic query processors.
ized subquery execution. The basic idea is similar to the
APPLY or MAPCAR operators of LISP: Evaluate an ex-
pression on a collection of items, and gather the results.
2. REPRESENTING AND NORMALIZING
The first construct we use is Apply. Relational database SUBQUERIES
practitioners will see this as nested loops with correlations, In this section we go into the detail of taking an SQL
while researchers on object-oriented databases might employ subquery and generating an equivalent operator tree that
lambda-calculus notation directly, e. g. [16, 4, 12]. Apply does not make use of correlated execution. The result is a
takes a relational input R and a parameterized expression normal form corresponding to a query formulation without
E(r); it evaluates expression E for each row r ∈ R, and subqueries. We consider all SQL subqueries, and describe
collects the results. Formally, what makes their replacement by standard operators easy,
[ or hard, leading to broad subquery classes.
R A⊗ E = ({r} ⊗ E(r)) ,
r∈R 2.1 Direct algebraic representation with mu-
where ⊗ is either cross product, left outerjoin, left semi- tual recursion
join, or left antijoin. The most primitive form is A× , As a first step, the parser/algebrizer takes the SQL for-
and cross product is assumed if no join variant is speci- mulation and generates an operator tree, which contains
fied. Since we deal with SQL, all operators used in this pa- both relational and scalar operators. For example, an SQL
per are bag-oriented, and we assume no automatic removal where clause is translated into relational select, which has
of duplicates. In particular, the union operator above is two subexpressions in the operator tree. The first subex-
UNION ALL. Duplicates are removed explicitly using dis- pression has a relational operator as root, which computes
tinct, which as mentioned earlier is just a special case of the relational input to be filtered. The second subexpression
GroupBy. has a scalar operator as root, and it represents the predicate
For example, the correlated execution strategy for the sub- to be use for filtering. Figure 3 shows the operator tree gen-
query example Q1 shown earlier is represented algebraically erated by the algebrizer for the example query Q1 of Section
in Figure 2. In this case, each invocation of the parameter- 1.1. Relational nodes are shown in bold.
ized expression returns exactly one row, so the cardinality On this representation, scalar operators may have rela-
of customer is preserved through Apply. In general, the tional subexpressions as children, as shown in the figure.
number of output rows depends on the cardinality of E(r), Straightforward execution of this operator tree implies a
as well as the join variant used to combine the result with “nested loops style” of execution for the subquery, as well as
r. mutual recursion between the relational and the scalar exe-
The role of Apply in (sub-)query processing has been inde- cution components —per-row execution of relational select
pendently explored by at least three research/development calls scalar evaluation of a predicate, which in turns calls
groups: At Tandem, where it was called tuple substitution relational execution of a subquery.

573
 SELECT
R A⊗ E = R ⊗true E, (1)
if no parameters in E resolved from R
⊗
R A (σp E) = R ⊗p E, (2)
CUSTOMER <
if no parameters in E resolved from R
R A (σp E) = σp (R A× E)
×
(3)
R A× (πv E) = πv ∪ columns(R) (R A× E) (4)
1000000 SUBQUERY(X)
R A× (E1 ∪ E2 ) = (R A× E1 ) ∪ (R A× E2 ) (5)
R A× (E1 − E2 ) = (R A× E1 ) − (R A× E2 ) (6)
R A× (E1 × E2 ) = (R A× E1 ) ./R.key (R A× E2 ) (7)
ScalarGb
R A× (GA,F E) = GA ∪ columns(R),F (R A× E) (8)
R A× (GF1 E) = Gcolumns(R),F 0 (R ALOJ E) (9)

X:= Identities 7 through 9 require that R contain a key R.key.

SELECT
In identity (7), Join on R.key is used as a shorthand for the
obvious predicate. In identity (9), F 0 contains aggregates
in F expressed over a single-column —for example, if F
is count(*), then F 0 is count(c) for some not-nullable
ORDERS = SUM
column c from E. Identity (9) is valid for all aggregates such
that agg(∅) = agg({null}), which is true for SQL aggregates.

Figure 4: Rules to remove correlations.

O_CUSTKEY C_CUSTKEY O_TOTALPRICE

Figure 3: Direct algebraic representation of sub- possible to obtain an equivalent expression that does not
query. use Apply. The process consists of pushing down Apply in
the operator tree, towards the leaves, until the right child of
Apply is no longer parameterized off the left child. Figure
4 describes the properties that allow this pushing —see [17,
2.2 Algebraic representation with Apply 6] for additional details, and discussion on these properties.
Mutual recursion between scalar and relation nodes in the For example, consider the expression shown in Figure 2.
algebrizer output is removed by introducing Apply opera- On the right child of Apply there is a scalar aggregation,
tors. The general scheme is to evaluate the subquery explic- then a select, and below that point there are no more outer
itly before the operator whose scalar expressions requires references. Apply removal is shown in Fig 5. Identity (9)
the subquery result. Say there is a relational operator Πon is used to push Apply below Scalar GroupBy; then Identity
input R, with a scalar argument e using a subquery Q. We (2) is used to absorb the last parameterized select and re-
execute the subquery first using Apply, such that the sub- move the Apply. This results in the strategy of outerjoin
query result is available as a (new) column q; then replace followed by aggregate. In addition, due to the predicate on
the subquery utilization by such variable: the aggregate result, the left outerjoin (denoted LOJ) can
be simplified to join.
Œe(Q) R ; Œe(q) (R A⊗ Q).
2.4 All SQL subqueries
As an example, removing mutual recursion from the op- So far we have described transformations that normalize
erator tree in Figure 3 results in the tree shown in Figure 2. subqueries into standard relational operators, and exempli-
An operator A× is introduced below the relational select fied the procedure using a scalar aggregate subquery. We
to compute the subquery, whose result is stored in column now describe additional SQL subquery scenarios, and how
X. Figure 2 no longer shows the expanded operator trees our scheme is affected on those.
for scalar expressions.
We showed how to remove one subquery from a scalar For boolean-valued subqueries, i. e. exists, not exists,
expression, but the technique naturally applies to multi- in subquery, and quantified comparisons, the subquery can
ple subqueries, in which case a sequence of Apply operators be rewritten as a scalar count aggregate. From the utiliza-
compute the various subqueries over the relational input. tion context of the aggregate result, either equal to zero or
Straightforward execution at this point is still based on greater than zero, it is possible for the aggregate operator
nested loops. However, recursive calls between scalar and re- to stop requesting rows as soon as one has been found, since
lational execution are removed, since scalar evaluation never additional rows do not affect the result of the comparison.
needs to call back into the relational engine. Removing mu- A common case that is further optimized is when a re-
tual recursion not only can have an impact on performance, lational select has an existential subquery as its only pred-
but it also simplifies implementation. icate (or when such select can be created by splitting an-
other that ANDs an existential subquery with other condi-
2.3 Removal of Apply tions). In this case, the complete select operator is turned
Given a relational expression with Apply operators, it is into Apply-semijoin for exists, or Apply-antisemijoin for not

574
 σ1000000<X (customer A× GX=
1
sum(o price) σo custkey=c custkey orders)

= σ1000000<X Gc custkey,X=sum(o price) (customer ALOJ σo custkey=c custkey orders), by identity (9)
= σ1000000<X Gc custkey,X=sum(o price) (customer LOJo custkey=c custkey orders), by identity (2)
= σ1000000<X Gc custkey,X=sum(o price) (customer ./o custkey=c custkey orders), by outerjoin simplification.

Figure 5: Example of correlation removal.

exists. Such Apply is then converted into a non-correlated of Apply with conditional execution of the parameterized
expression, if possible, using Identity (2). For the resulting expression, based on a predicate. Implementing this is re-
semijoin, we consider execution as join followed by GroupBy quired for completeness, but in our experience this scenario
(distincting), which follows from the definition of semijoin. is very rare in practice.
This GroupBy is also subject to reordering, covering the
semijoin strategies suggested in [14].
2.5 Subquery classes
There are two scenarios where normalization into stan- Our approach delineates three broad classes of subquery
dard relational algebra operators is hindered in a fundamen- usage in SQL, which grant different treatment from the
tal way. We call those exception subqueries and they require query processor.
scalar-specific features. Consider the following query.

Q2: select c name,

Class 1. Subqueries that can be removed with no ad-
(select o orderkey from orders
ditional common subexpressions. In general, removing
Apply requires introduction of additional common subex-
where o custkey = c custkey)
pressions —e. g. see Identity (5), which introduces two
from customer
copies of R. Cases that do not require introducing com-
For every customer, output the customer name, and the mon subexpressions are easier. In particular, the com-
result of a subquery that retrieves an oderkey. There are mon case of subqueries that are formed by a simple se-
three cases: If exactly one row is returned from the subquery, lect/project/join/aggregate block are easy to handle. An
then such value is used in the scalar expression; if no rows are example query for this class is Q1 from Section 1.1. Our
returned, then null is used; finally, if more than one row is normalization scheme produces an operator tree with stan-
returned, then a run-time error is generated [13]. The above dard relational operators and no correlations involved. Af-
query is valid, but will generate a run-time error if there terwards, cost-based optimization will consider a number of
happens to be a customer with more two or more orders. alternatives, including re-introduction of a correlated exe-
Since there are no run-time errors in standard relational cution, which can be very effective if few outer rows are
algebra, we need an additional operator to represent these processed and appropriate indices exist. Earlier research on
subqueries. We call such operator Max1row. It takes a SQL subqueries has implicitly focused on a subset of this
relational input and passes input rows unmodified, and it class of subqueries.
generates a run-time error if the input has more than one Any subquery processing strategy that applies on Class 1
row. It is placed in the right subexpression of Apply, to subqueries must have a more primitive formulation, appli-
verify the SQL subquery semantics. cable on expressions without correlations. For example, the
There is some amount of reordering that can be done on “magic” strategy for subquery evaluation described in [15]
Max1row, but we find this case uninteresting. In our experi- has a primitive formulation on join and aggregation [17].
ence, at most one row is returned in most meaningful cases,
and the compiler can detect this from information about Class 2. Subqueries that are removed by introducing
keys. There is no need for Max1row then. For example, additional common subexpressions. Achieving optimal-
we reverse the roles of the tables next, retrieving customer ity and syntax-independence in this class requires an under-
name for each order. The resulting query is more meaning- standing of the plan space and mechanisms to generate plans
ful, and the compiler avoids the use of Max1row, as long as of interest, for queries with common subexpressions, which
s custkey is a declared key. we believe requires additional research. In our current imple-
mentation these subqueries are not removed during normal-
select o orderkey, ization, but we do still consider unnesting transformations
(select c name from customer during cost-based optimization. This can lead to improve-
where c custkey = o custkey) ments over the original subquery form, and cost-based de-
from orders cisions will be used to choose appropriate execution plans.
However, no syntax-independence is provided, and there is
Another problematic construct is conditional scalar ex- no simple characterization of the space of interest. We are
ecution, expressed in SQL as case when <cond> then not aware of any work that has attempted to optimize this
<value1> else <value2> end. The point is, <value2> class of subqueries.
should not be evaluated when <cond> is true. Therefore, It is hard to formulate a short, meaningful query that fits
eager execution of a subquery, say contained in <value2>, in this class, using the TPCH schema. The next query is
is incorrect, in particular if it happens to generate a run-time a valid (but meaningless) SQL example for the class, using
error. To deal with this scenario, we use a modified version UNION ALL. Removing Apply requires Identity (5), which

575
introduces multiple copies of the outer table. that generated this row after it is pushed below the aggre-
gate. The only characteristic shared by this group of rows is
select *, the values of the grouping columns. Therefore we can move
from partsupp a filter around a GroupBy if and only if all the columns
where 100 > used in the filter are functionally determined by the group-
(select sum(s acctbal) from ing columns in the input relation.
(select s acctbal Moving aggregates around a join is a little more com-
from supplier plicated. A GroupBy can be pushed below a join if the
where s suppkey = ps suppkey grouping columns, the aggregate calculations and the join
union all predicate each satisfy certain conditions. Suppose we have
select p retailprice a GroupBy above a join of two relations, i.e. GA,F (S ./p R),
from part and we want to push the GroupBy below the join so that
where p partkey = ps partkey) the relation R is aggregated before it is joined i.e. S ./p
as unionresult) (GA∪columns(p)−columns(S),F R). This is feasible if and only
if the following three conditions are met -
1. If a column used in the join predicate p is defined by
Class 3. Exception subqueries. These subqueries are the relation R then it is part of the grouping columns.
fundamentally non-relational, as they require scalar-specific
2. The key of the relation S is part of the grouping
features such as generating run-time errors. We consider
columns.
these cases relatively uninteresting, and rare in practice. To
our knowledge, no research work has addressed queries in 3. The aggregate expressions only use columns defined by
this class. An example query for this class is Q2 described the relation R.
in Section 2.4.
To see why this is correct, think of a join as a cross product
followed by a filter. The first two condition ensure that all
3. COMPREHENSIVE OPTIMIZATION the columns of the predicate are functionally determined by
OF AGGREGATION the grouping columns. This allows us to push the GroupBy
In this section we describe techniques for efficient pro- below the filter. The second condition implies that no two
cessing of queries with aggregations. We extend, and for- rows from the relation S are included in the same group
mulate in algebraic terms, earlier work related to reordering during aggregation. The last condition ensures that the ag-
of GroupBy/Aggregate, and segmented execution of queries. gregate expressions can be calculated with just the relation
The result is a number of transformation rules to generate R. These two conditions together enable us to push the
interesting execution strategies, to be used in the context of GroupBy below the cross product.
cost-based optimization. Pulling a GroupBy above a join is a lot easier. All that is
required is that the relation being joined has a key and that
3.1 Reordering GroupBy the join predicate does not use the results of the aggregate
Aggregating a relation reduces its cardinality. This may functions. These two restrictions follow directly from the
give us the impression that we can use the same early evalu- discussion above. Formally,
ation strategy that we use for filters. But that is not always
S ./p (GA,F R) = GA∪columns(S),F (S ./p R)
the case because aggregation can be quite expensive and the
cost depends heavily on the number of rows being aggre- Even these two conditions are not as restrictive as they ap-
gated. Take the case of joins. If the join predicate reduces pear at first sight. If the relation S does not have a key, one
the cardinality dramatically it may be better to perform can always be manufactured during execution. As for the
the GroupBy after the join. Doing it later avoids unnec- second condition, conjuncts of a join predicate can always
essary calculation of aggregates which will be thrown away be separated out into a filter performed after the join. We
by the join. Another reason can be the existence of appro- can apply this strategy to the predicate p if it uses results
priate indices, which allows the join to be performed as an of the aggregate functions.
index-lookup. This may not be possible when the aggre- One can think of semijoins and antisemijoins as filters
gate obstructs it. Therefore, it is best to generate both the since they include or exclude rows of a relation based on the
alternatives and leave the choice to the cost based optimizer. column values. The conditions necessary to reorder these
In this section we study the conditions necessary to move operators around a GroupBy can therefore be easily de-
a GroupBy around filters, joins, semijoins etc. As we men- duced from those for a filter. Suppose we have an aggregate
tioned earlier some of these ideas were developed in [3, 18]. followed by a semijoin i.e. (GA,F R)ˆ Hp S we can push the
We formulate them here in a way that they can be imple- semijoin below if and only if p does not use the result of
mented as primitive optimization rules. In the next section any aggregate expressions and every column of predicate p,
we do the same for outerjoins. In our discussion we will say c, satisfies the condition that if c ∈
/ columns(S) then c
formally denote a GroupBy as GA,F where A is the set of is functionally determined by the grouping columns(i.e. the
grouping columns and F are the aggregate functions. set A). The condition for antisemijoin is exactly the same.
Let us begin with the discussion of a primitive to reorder
a filter and an aggregate. For a GroupBy operator, rows in 3.2 Moving GroupBy around an outerjoin
the input relation that have same values for the grouping Removing correlations for scalar valued subqueries results
columns generate exactly one output row. If a filter above in an outerjoin followed by a GroupBy. Therefore it is es-
an aggregate rejects a row, it needs to reject the whole group pecially important for our optimizer to have primitives that

576
allow reordering of these two operators. None of the litera- below a join. But sometimes it may be possible to do part
ture cited above discusses this issue. of the aggregations before the join and then combine these
In order to push a GroupBy below an outer join the join partially aggregated rows afterwards to get the final result.
predicate, grouping columns and the aggregate calculations This can be efficient because it reduces the cardinality of the
once again have to meet the three conditions mentioned join inputs. Some of these ideas are discussed in [3, 18]. Here
above. The only difference is that an extra project may have we introduce a new operator called LocalGroupBy (formally
to be added above the outerjoin if the aggregate expressions LG) and develop primitives that allow us to push it below
do not meet a certain condition. other relational operators giving us this ability to aggregate
Let us see why this works and why a project may be nec- early.
essary. The result of an outerjoin has two types of rows viz. In order to introduce a LocalGroupBy, the aggregate func-
those that match, and those that do not and are padded tion has to be split into two new functions - one which does
with NULLs. We know that our grouping columns include the early partial aggregation, called a local aggregate in this
a key of the outer relation. This implies that a group can paper, and another which combines these aggregates to gen-
never have both matched and unmatched rows. For the rows erate the final result, called a global aggregate in this paper.
that match, correctness can be proved by using the same Formally if we have an aggregate function f , we need a local
argument as join. For the rows that do not match, early aggregate function fl and a global aggregate function fg such
aggregation means that they will not be aggregated at all! that for any set S and for any partition of it {S1 , S2 , . . . , Sn }
This is where the extra condition and the optional project we have
comes in. [n [
n
In the result of an outerjoin, an unmatched row appears f ( Si ) = fg ( fl (Si ))
exactly once. Therefore given that our grouping columns i=1 i=1
include the key of the outer relation, a group that has an Note that the implementation, whether hash based or sort
unmatched row cannot have any other row. The aggregate based, of aggregate functions in a query execution engine
functions use only the columns from the non-outer relation. requires this ability of splitting an aggregate into local and
For an unmatched row all these columns are NULL. There- global components, if it has to spill data to disk and then
fore the property of aggregate expressions that is important recombine it.3
to us is the result of applying it to NULLs. If the result is If all the aggregate functions used in a GroupBy can be
NULL as it is for most simple aggregate expressions, we need split this way, which should almost always be true, we can
do nothing more. The outerjoin will automatically provide replace a “standard” GroupBy with a LocalGroupBy fol-
the NULLs we need. For the aggregate expressions which lowed by a “global” GroupBy. Formally we have
do not result in a NULL, we need to add a project which
for each unmatched row sets the aggregate result to the ap- GA,F R = GA,Fg LG A,Fl R
propriate constant value.2 Note that this constant can be
where Fl and Fg are the local and global aggregate expres-
calculated at compile time. For count(*), the value of this
sions corresponding to F .
constant is zero.
LocalGroupBy has the interesting property that its group-
Formally we have
ing columns can be extended without affecting the final re-
GA,F (S LOJp R) = πc (S LOJp (GA−columns(S),F R)) sult. Adding a new column to the set of grouping columns
just partitions the groups further. But since we have a final
where the computing project πc introduces the non-NULL “global” aggregate to combine these partial results, the final
results if necessary. result remains unchanged.
As an example, in the outerjoin/aggregate strategy shown This ability to extend grouping columns gives us infinite
earlier in Section 1.1, the aggregation can be pushed down freedom. Take the case of a join. It is now trivially easy to
below the outerjoin. satisfy the first two restrictions we mentioned in our discus-
select c custkey sion about reordering aggregates with a join. We just add
from customer left outer join the necessary columns. That leaves us with only the third re-
(select o custkey, striction about aggregate functions. The solution once again
sum(o totalprice) as totalorder is to extend the grouping columns. An aggregate function
from orders computation can be removed from a LocalGroupBy using
group by c custkey) as AggResult the following steps: First extend the grouping columns by
on o custkey = c custkey adding the aggregate input column (or expression, in gen-
where 1000000 < totalorder eral); at this point, the aggregate function is operating on a
set of count(*) identical values. Now, replace the aggregate
No computing projects are required here as the aggregate computation by a later project that, in general, computes
expression sum(o totalprice) does result in NULL when cal- the result based on count(*) and the original aggregate in-
culated on a singleton NULL. put, which is a constant for the group. For example, suppose
the grouping columns include the column a and one of the
3.3 Local Aggregates aggregates being calculated is sum(a). We can get the same
The restrictions on the join predicate, grouping column answer with the expression (a×count(*)) calculated after
etc. mean that it is not always possible to push a GroupBy the aggregation. This property of aggregate functions allows
2 3
Detection of unmatched rows requires a non-nullable col- There can be composite aggregates, such as avg, which
umn from the inners side, which can always be manufac- do not have local/global versions. But they are computed
tured, or else changes to outerjoin to provide a match col- based on primitive aggregates that do, since we want to be
umn. able to compute them using algorithms that spill to disk.

577
us to replace any aggregate function used in a LocalGroupBy SegmentApply[L_PARTKEY] use S
with a count(*), thus addressing the third restriction. We
can push a LocalGroupBy below any join and to any side
of the join. More details about reordering LocalGroupBy
with several other operators, and proofs of correctness can LINEITEM JOIN(L_QUANTITY < X)
be found in [10].
Note that the actual implementation of a LocalGroupBy
in the query execution engine need not be different from a SGb(X=0.2*AVG(L_QUANTITY)) S
GroupBy. We use a separate operator only to make the job
of optimizer easy since the transformations described above
are only valid for the LocalGroupBy operator.
S
3.4 Segmented execution
Frequently the process of correlation removal for scalar Figure 6: SegmentApply
valued subqueries results in two almost identical expressions
joined together. The only difference is that one of them is
aggregated and the other is not. A simple example of this
is Query 17 of the TPC-H benchmark. After removing the 3.4.1 Introducing SegmentApply
correlation, the SQL representation of the query is - Whenever we see two instances of an expression connected
by a join, where one of the expressions may optionally have
select sum(l extendedprice)/7.0 as avg yearly an extra aggregate and/or an extra filter, we attempt to
from lineitem, part, generate an alternative that uses SegmentApply. The key
(select l partkey as l2 partkey, thing to look for is a conjunct in the join predicate that is an
0.2 * avg(l quantity) as x equality comparison between two instances of the same col-
from lineitem umn from the two expressions. Such a comparison implies
group by l partkey) as aggresult that rows for which this column differs will never match.
where p partkey = l partkey We can therefore use the column to partition the relation.
and p brand = ’brand#23’ Note that the join predicate can have multiple columns that
and p container = ’med box’ satisfy this criterion allowing us to have finer segments. If
and p partkey = l2 partkey the join predicate does yield such segmenting columns, we
and l quantity < x introduce a correlated execution alternative that uses Seg-
Here we have two instances of the lineitem table joined mentApply.
together where one of them is grouped by l partkey. The Removing correlations for an existential subquery gener-
SQL representation for this join using the implied predicate ates a semijoin, or antisemijoin. The argument in the previ-
is - ous section is valid for those operators too. We can therefore
introduce a SegmentApply in both these cases, if there are
select l partkey, l extendedprice common subexpressions and the required equality conditions
from lineitem, hold. The only difference is in the correlated expression.
(select l partkey as l2 partkey,
0.2 * avg(l quantity) as x 3.4.2 Moving joins around SegmentApply
from lineitem Going back to our TPC-H example, we can see that look-
group by l partkey) as aggresult ing at all the lineitem rows is an overkill because the final
where l partkey = l2 partkey result cares only about parts with a specific brand and a
and l quantity < x specific container. It would be more efficient to only process
lineitem rows for these parts. This optimization is possi-
Semantically this join is trying to find all the lineitem
ble if we add a primitive to reorder a SegmentApply and
rows where the quantity ordered is less than 20% of the
a join that allows us to reduce the lineitem table earlier
average for that part. But this means the selection of a
by joining it with the part table. Algebraic representation
particular lineitem row does not require the whole derived
of SegmentApply allows us to manipulate it like any other
table aggresult. All we need is the average quantity for
operator and makes it very easy to add such a primitive.
the part referenced in that row. That gives rise to an in-
The key condition to check when pushing a join below
teresting correlated execution strategy. We can segment the
a SegmentApply is preservation of the segment. The join
lineitem table based on the part and calculate the join for
predicate needs to be such that it either allows all rows in
each segment independently. A SegmentApply does exactly
a segment to pass through or none of them to do so. If
that.
pushing the join removes only some of the rows, the result
A SegmentApply operator is very similar to the Apply
of the correlated expression will not be same. Suppose we
operator we studied in detail. The only difference is that the
have a SegmentApply expression (R SAA E) ./p T where A
parameter is a set of rows rather than a single row. The inner
is the set of segmenting columns and p is the join predicate.
child of the SegmentApply is an expression that uses this set.
The all or none condition means that the predicate p can
Figure 6 shows the transformed version of the lineitem join.
use only the segmenting columns or columns of the relation
Segmented execution was discussed in [2]. Our contribution
T and nothing else.4
is to formulate it as an algebraic operator so that it can be
used in a cost based optimizer. The formalism also allows 4
As we mentioned in case of GroupBy, this is not as restric-
us to introduce reordering primitives. tive as it appears. A join predicate can be split up into a

578
 A join predicate which satisfies this condition may still special strategies for GroupBy; and introduction of corre-
change the segment. If the join is such that one row of lated execution (the simplest and most common being index-
R matches multiple rows of T all these resulting rows will lookup-join). The architecture of our cost-based optimizer
be included in the segment. There is a simple solution to follows the main lines of the Volcano optimizer [9], so that
this however and it is to add the key of relation T to the generation of interesting reorderings is done by means of
segmenting columns.5 This ensures that each instance of transformation rules.
the row goes to a different segment.
Formally we have 5. PERFORMANCE RESULTS
(R SAA E) ./p T = (R ./p T ) SAA∪columns(T ) E We now present the results of our optimizations on queries
from the TPC-H benchmark. Normalization flattens all sub-
iff columns(p) ⊆ A ∪ columns(T ).
queries in the benchmark, but this does not have a direct im-
The predicate of the join with the part table in TPC-H pact on query performance —it is reordering, and GroupBy
Query 17 uses the column l partkey, which is our segment- optimization techniques that do have an impact. In particu-
ing column. We can therefore push the join below the Seg- lar, our full set of techniques apply on Query2 and Query17.
mentApply. We need to add the key of the part table viz. Figure 8 lists all published TPC-H results on the 300GB
p partkey to the segmenting columns, but since the two scale, as of November 27th, 2000, which we include here
segmenting columns l partkey and p partkey are equal in compliance with the TPC reporting rules. In Figure 9
we can safely remove one of them. The final expression is we plot the published results of elapsed time for Query2
shown in Figure 7. and Query17. The numbers are taken from the information
Remember once again that these alternatives will be available in the TPC web page (http://www.tpc.org), for
costed and used in the final plan only if they appear all eight TPC-H results on the 300GB scale. None of those
cheaper. results use clusters. The x axis plots the number of proces-
sors used in each benchmark result, and the y axis plots the
elapsed execution time on the power run. For example, the
4. COMPILATION IN SQL SERVER lower left point in the graph for Query17 corresponds to an
The following is a brief description of relevant compilation elapsed time of 79.7 sec, obtained on 8 processors; while the
steps in SQL Server, and how different optimizations are lower right point on the same graph is for 210.4 sec time
incorporated. For the most part, the material of Section on 64 processors. Points are separated by DBMS, since the
2 deals with preparing a normalized operator tree, to be different query processors are likely to use different technolo-
used as input for cost-based optimization. The material of gies. On these two queries, SQL Server has published the
Section 3 deals mostly with interesting alternatives that can fastest results, even on a fraction of the processors used by
reduce execution time dramatically, but need to be costed other systems.
to determine when to use them, and are therefore part of TPC-H has strict rules on what indices are allowed, re-
cost-based optimization. ducing the relative impact of physical database design, in
comparison to query processor technology and hardware.
Parse and bind. This step is a relatively direct translation Several factors contribute to the fast SQL Server result, in-
of SQL text into an operator tree containing both relational cluding both query optimization and execution. An essen-
and scalar operators, in the form shown in Section 2.1. Note tial component are the techniques described earlier in this
that current SQL allows the use of (correlated) subqueries paper.
anywhere scalar expressions are allowed, including SELECT
and WHERE clauses. Any scalar expression may have a 6. CONCLUSIONS
relational expression as children. In this paper we described the approach used in Microsoft
SQL Server to process queries with subqueries and/or ag-
Query normalization. This step transforms an operator gregations. The approach is based on several ideas:
tree into a simplified/normalized form. Simplifications in-
clude, for example, turning outerjoins into joins, when pos- Subqueries and aggregation should be handled by or-
sible, and detecting empty subexpressions. For subqueries, thogonal optimizations. Earlier work has sometimes com-
mutual recursion between relational and scalar execution is bined multiple, independent primitives to derive strategies
removed, which is always possible; and correlations are re- that are suitable for some cases. What we do instead is
moved, which is usually possible. At the end of normaliza- to separate out those independent, small primitives. This
tion, most common forms of subqueries have been turned allows finer granularity of their application; it generates a
into some join variant. richer set of execution plans; it makes for more modular
proofs; and it simplifies implementation.
Cost-based optimization. Execution alternatives are gen-
erated using transformation rules, and the plan with cheap- Algebraic constructs for parameterized subexpressions
est estimated cost is selected for execution. Important are a useful query processing tool. The Apply and Seg-
classes of optimizations include: Reordering of join vari- mentApply constructs do not add expressive power to re-
ants; reordering of GroupBy with join variants; considering lational algebra. However, they facilitate query representa-
filter applied after the join. tion for some query language constructs, and significantly
5
This is exactly what we did when pushing a join below a enhance the space of alternatives considered for execution.
GroupBy, which should not be surprising since a Segmen- This is achieved in an algebraic fashion, allowing both for-
tApply is just a more general version of a GroupBy. mal algebraic manipulation as well as smooth integration in

579
 SegmentApply[L_PARTKEY] use S

JOIN(L_PARTKEY = P_PARTKEY) JOIN(L_QUANTITY < X)

LINEITEM FILTER(P_BRAND‘Brand#23’ AND SGb(X=0.2*AVG(L_QUANTITY)) S

P_CONTAINER = ‘MED BOX’)

PART S

Figure 7: Join with PART pushed below SegmentApply

System Database QphH Price/QphH System Date

@300GB @300GB, Availability Submitted
in US$
COMPAQ ProLiant 8000-8P Microsoft SQL Server 2000 1506 280 11/17/00 11/17/00
HP NetServer LXr 8500 Microsoft SQL Server 2000 1402 207 08/18/00 08/18/00
HP 9000 N4000 Informix Extended 1592 973 05/02/00 05/02/00
Enterprise Server Parallel Server 8.30 FC2
COMPAQ AlphaServer Informix XPS 8.30 FC3 4951 983 08/31/00 07/13/00
GS320 Model 6/731
HP 9000 V2500 Informix Extended 3714 1119 12/17/99 10/21/99
Enterprise Server Parallel Server 8.30 FC2
IBM NUMA-Q 2000 IBM DB2 UDB 7.1 4027 652 09/05/00 09/05/00
IBM NUMA-Q 2000 IBM DB2 UDB 7.1 5923 653 09/05/00 09/05/00
IBM NUMA-Q 2000 IBM DB2 UDB 7.1 7334 616 08/15/00 05/03/00

Figure 8: Published TPC-H results for 300GB, as of November 27th, 2000. Included here in compliance with
the TPC reporting rules.

300 1800
1600
250
1400
Time (sec)

200
Time (sec)

1200
MS SQL Server
1000
150 Informix
800
100 600 IBM DB2
400
50
200
0 0
0 8 16 24 32 40 48 56 64 72 0 8 16 24 32 40 48 56 64 72
Number of processors Number of processors
Results for Query 2. Results for Query 17.

Figure 9: Query performance reported in TPC-H results, 300GB scale.

580
algebraic query processors. 2001. MSR-TR-2000-31.
[7] C. A. Galindo-Legaria and A. Rosenthal. Outerjoin
Parameterized expressions (i. e. correlated subqueries) simplification and reordering for query optimization.
should be removed during query normalization, for ACM Transactions on Database Systems, 22(1):43–73,
syntax-independence. In Section 2 we described algebraic Mar. 1997.
removal of correlations. We considered all SQL subqueries, [8] G. Graefe. The Cascades framework for query
and described what makes their replacement by standard op- optimization. Data Engineering Bulletin, 18(3):19–29,
erators easy, or hard, leading to three broad classes. On one 1995.
side of the scale, subqueries whose body consists of a sim- [9] G. Graefe and W. J. McKenna. The volcano optimizer
ple SQL query block always can and should be normalized generator: Extensibility and efficient. In Proceedings
out. In the next level, removal of subqueries requires in- of the Ninth International Conference on Data
troducing common subexpressions, which impose additional Engineering, Viena, Austria, pages 209–218, 1993.
requirements on the query processor on the resulting “flat” [10] M. M. Joshi and C. A. Galindo-Legaria. Properties of
expressions. Finally, subqueries that need checking for er- the GroupBy/Aggregate relational operator. Technical
rors at execution time, such as verification that a scalar- report, Microsoft, 2001. MSR-TR-2001-13.
valued subquery returns at most one row, are not relational [11] W. Kim. On optimizing an SQL-like nested query.
in nature. From what we know, earlier work on subqueries ACM Transactions on Database Systems,
has implicitly focused on the simplest class. 7(3):443–469, Sept. 1982.
[12] T. Leung, G. Mitchell, B. Subramanian, B. Vance,
A rich set of alternatives for GroupBy/Aggregate ex- S. L. Vandenberg, and S. B. Zdonik. The AQUA data
ecution should be considered during cost-based op- model and algebra. In DBPL, pages 157–175, 1993.
timization. GroupBy/Aggregate appears frequently, either [13] J. Melton and A. R. Simon. Understanding the new
writen directly in the original query, or as a result of query SQL: A complete guide. Morgan Kaufmann, San
normalization. In Section 3 we described two powerful tech- Francisco, 1993.
niques: Reordering of GroupBy, and segmented execution.
[14] H. Pirahesh, J. M. Hellerstein, and W. Hasan.
We extend, and formulate in algebraic terms, earlier work,
Extensible/rule based query rewrite optimization in
resulting in a number of transformation rules to generate in-
starburst. In Proceedings of ACM SIGMOD 1992,
teresting execution strategies. It is these optimizations that
pages 39–48, 1992.
make for the order-of-magnitude performance improvements
that we report. [15] P. Seshadri, H. Pirahesh, and T. Y. C. Leung.
Complex query decorrelation. In Proceedings of the
We showed the effectiveness of our overall approach with Twelfth International Conference on Data
figures from published TPC-H results, in Section 5. In Engineering, New Orleans, Luisiana, pages 450–458,
particular, our full set of techniques apply on Query2 and 1996.
Query17. From all published TPC-H results in the 300GB [16] G. Shaw and S. Zdonik. An object-oriented query
scale, at the time of writing (November 2000), SQL Server algebra. In Proceedings of the Second International
has the fastest results on those queries, even on a fraction Workshop on Database Programming Languages,
of the processors used by other systems. pages 249–225, 1989.
[17] Q. Wang, D. Maier, and L. Shapiro. Algebraic
7. REFERENCES unnesting of nested object queries. Technical report,
[1] P. Celis and H. Zeller. Subquery elimination: A Oregon Graduate Institute, 1999. CSE-99-013.
complete unnesting algorithm for an extended [18] Y. P. Yan and P. A. Larson. Eager aggregation and
relational algebra. In Proceedings of the Thirteenth lazy aggregation. In Proceedings of the 21st
International Conference on Data Engineering, April International Conference on Very Large Databases,
7-11, 1997 Birmingham U.K, page 321, 1997. Zurich, pages 345–357, 1995.
[2] D. Chatziantoniou and K. A. Ross. Groupwise
processing of relational queries. In Proceedings of the
23rd International Conference on Very Large
Databases, Athens, pages 476–485, 1997.
[3] S. Chaudhuri and K. Shim. Including Group-By in
query optimization. In Proceedings of the Twentieth
International Conference on Very Large Databases,
Santiago, pages 354–366, 1994.
[4] S. Cluet and C. Delobel. A general framework for the
optimization of object-oriented queries. In Proceedings
of ACM SIGMOD 1992, pages 383–392, 1992.
[5] U. Dayal. Of nests and trees: A unified approach to
processing queries that contain nested subqueries,
aggregates, and quantifiers. In Proceedings of the
Thirteenth International Conference on Very Large
Databases, Brighton, pages 197–208, 1987.
[6] C. A. Galindo-Legaria. Parameterized queries and
nesting equivalences. Technical report, Microsoft,

581