Chapter 5: Advanced SQL

Database System Concepts, 7th Ed.

©Silberschatz, Korth and Sudarshan
See www.db-book.com
www.db-book.com for conditions on re-use
 Outline

 Accessing SQL From Python

 Functions and Procedures
 Triggers
 Recursive Queries
 Advanced Aggregation Features

Database System Concepts - 7th Edition 5.2 ©Silberschatz, Korth and Sudarshan
 Accessing SQL from a Programming Language

A database programmer must have access to a general-purpose programming

language for at least two reasons
 SQL does not provide the full expressive power of a general-purpose
language.
 Non-declarative actions -- such as printing a report, interacting with a
user, or sending the results of a query to a graphical user interface --
cannot be done from within SQL.

Database System Concepts - 7th Edition 5.3 ©Silberschatz, Korth and Sudarshan
 MySQL with Python
 Many open source libraries allow Python to interact with a MySQL database.
One of the most popular of these is PyMySQL, which can be installed using
pip install PyMySQL
The connection/cursor model is
used here.

• A connection object conn is

responsible for connecting
to the database, sending
the database information,
handling rollbacks, and
creating cursor objects.

• A cursor object cur which

contains the result of the
latest query it has executed.

There are many books to learn

this. One is Chapter 9 of Web
Scarping with Python: Data
Extraction from the Modern
Web (3rd Edition) by R. Mitchell

Database System Concepts - 7th Edition 5.4 ©Silberschatz, Korth and Sudarshan
 SQLite with Python

SQLite is a database
engine written in the C
programming language. It is
not a standalone app;
rather, it is a
library that software
developers embed in their
apps. As such, it belongs to
the family of embedded
databases.

The csv files are downloaded

from
https://www.kaggle.com/datasets
/olistbr/brazilian-ecommerce

Database System Concepts - 7th Edition 5.5 ©Silberschatz, Korth and Sudarshan
 Functions and Procedures

Database System Concepts - 7th Edition 5.6 ©Silberschatz, Korth and Sudarshan
 Functions and Procedures

Database System Concepts - 7th Edition 5.6 ©Silberschatz, Korth and Sudarshan
 Functions and Procedures

 Functions and procedures allow “business logic” to be stored in the

database and executed from SQL statements.
 The syntax we present here is defined by the SQL standard.
• Most databases implement nonstandard versions of this syntax.

Database System Concepts - 7th Edition 5.7 ©Silberschatz, Korth and Sudarshan
 Declaring SQL Functions

 Define a function that, given the name of a department, returns the count of
the number of instructors in that department.
create function dept_count (dept_name varchar(20))
returns integer
begin
declare d_count integer;
select count (* ) into d_count
from instructor
where instructor.dept_name = dept_name
return d_count;
end
 The function dept_count can be used to find the department names and
budget of all departments with more that 12 instructors.
select dept_name, budget
from department
where dept_count (dept_name ) > 12

Database System Concepts - 7th Edition 5.8 ©Silberschatz, Korth and Sudarshan
 Table Functions

 The SQL standard supports functions that can return tables as results; such
functions are called table functions
 Example: Return all instructors in a given department
create function instructor_of (dept_name varchar(20))
returns table (
ID varchar(5),
name varchar(20),
dept_name varchar(20),
salary numeric(8,2))
return table
(select ID, name, dept_name, salary
from instructor
where instructor.dept_name = instructor_of.dept_name)
 Usage
select *
from table (instructor_of ('Music'))

Database System Concepts - 7th Edition 5.9 ©Silberschatz, Korth and Sudarshan
 MySQL Procedures

it is unnecessary to
use delimiter because the
function definition contains no
internal ; statement delimiters:
Database System Concepts - 7th Edition 5.10 ©Silberschatz, Korth and Sudarshan
 Delimiter in MySQL
 https://dev.mysql.com/doc/refman/9.0/en/stored-programs-
defining.html

 https://dev.mysql.com/doc/refman/9.0/en/create-procedure.html

 If you use the mysql client program to define a stored procedure

containing semicolon characters, a problem arises. By default, mysql itself
recognizes the semicolon as a statement delimiter, so you must redefine
the delimiter temporarily to cause mysql client to pass the entire stored
procedure definition to the server.
 To redefine the mysql delimiter, use the delimiter command. The delimiter
is changed to // to enable the entire definition to be passed to the server
as a single statement, and then restored to ; before invoking the
procedure. This enables the ; delimiter used in the procedure body to
be passed through to the server rather than being interpreted by
mysql itself.
 You can redefine the delimiter to a string other than //, and the delimiter
can consist of a single character or multiple characters.

Database System Concepts - 7th Edition 5.11 ©Silberschatz, Korth and Sudarshan
 SQL Procedures

 The dept_count function could instead be written as procedure:

create procedure dept_count_proc (in dept_name varchar(20),
out d_count integer)
begin
select count(*) into d_count
from instructor
where instructor.dept_name = dept_count_proc.dept_name
end
 The keywords in and out are parameters that are expected to have
values assigned to them and parameters whose values are set in the
procedure in order to return results.
 Procedures can be invoked either from an SQL procedure or from
embedded SQL, using the call statement.
declare d_count integer;
call dept_count_proc( 'Physics', d_count);

Database System Concepts - 7th Edition 5.12 ©Silberschatz, Korth and Sudarshan
 MySQL Procedures

Database System Concepts - 7th Edition 5.13 ©Silberschatz, Korth and Sudarshan
 Language Constructs for Procedures & Functions
 SQL supports constructs that gives it almost all the power of a general-
purpose programming language.
• Warning: most database systems implement their own variant of the
standard syntax below.
 Compound statement: begin … end,
• May contain multiple SQL statements between begin and end.
• Local variables can be declared within a compound statements
 While and repeat statements:
• while boolean expression do
sequence of statements ;
end while

• repeat
sequence of statements ;
until boolean expression
end repeat

Database System Concepts - 7th Edition 5.14 ©Silberschatz, Korth and Sudarshan
 Language Constructs (Cont.)

 For loop
• Permits iteration over all results of a query
 Example: Find the budget of all departments

declare n integer default 0;

for r as
select budget from department
where dept_name = 'Music'
do
set n = n + r.budget
end for

Database System Concepts - 7th Edition 5.15 ©Silberschatz, Korth and Sudarshan
 Procedures: MySQL Example
 While statements:

1 1 2 3 5 8 13
, 2
........
, , ,
,
,
,

n = 5

i= 2 answer = 2
.

i=3 answer =
3 .

i: 4 answer :'

Database System Concepts - 7th Edition 5.16 ©Silberschatz, Korth and Sudarshan
 Procedures: MySQL Example

 Repeat statements:

Database System Concepts - 7th Edition 5.17 ©Silberschatz, Korth and Sudarshan
 Procedures: MySQL Example

 loop statements:

Database System Concepts - 7th Edition 5.18 ©Silberschatz, Korth and Sudarshan
 Language Constructs – if-then-else

 Conditional statements (if-then-else)

if boolean expression
then statement or compound statement
elseif boolean expression
then statement or compound statement
else statement or compound statement
end if

Database System Concepts - 7th Edition 5.19 ©Silberschatz, Korth and Sudarshan
 Example procedure

 Registers student after ensuring classroom capacity is not exceeded

• Returns 0 on success and -1 if capacity is exceeded
• See book (page 202) for details
 Signaling of exception conditions, and declaring handlers for exceptions
declare out_of_classroom_seats condition
declare exit handler for out_of_classroom_seats
begin
…
end
 The statements between the begin and the end can raise an exception by
executing “signal out_of_classroom_seats”
 The handler says that if the condition arises the action to be taken is to exit
the enclosing the begin end statement.

Database System Concepts - 7th Edition 5.20 ©Silberschatz, Korth and Sudarshan
 Procedures: MySQL Example (Page 203)

Database System Concepts - 7th Edition 5.21 ©Silberschatz, Korth and Sudarshan
 Procedures: MySQL Example (Page 203)

Database System Concepts - 7th Edition 5.22 ©Silberschatz, Korth and Sudarshan
 Procedures: MySQL Example (from its manual)

Error Code: 1062;

SQLSTATE: 23000
Message: Duplicate entry '%s' for key %d
SQLSTATE values are taken from ANSI SQL and
ODBC (Open Database Connectivity) and are
more standardized than the numeric error codes.

This means execution continued

to the end of the procedure
after the error occurred

Database System Concepts - 7th Edition 5.23 ©Silberschatz, Korth and Sudarshan
 Triggers

Database System Concepts - 7th Edition 5.24 ©Silberschatz, Korth and Sudarshan
 Triggers

 A trigger is a statement that is executed automatically by the system as a

side effect of a modification to the database.
 To design a trigger mechanism, we must:
• Specify the conditions under which the trigger is to be executed.
• Specify the actions to be taken when the trigger executes.
 Triggers introduced to SQL standard in SQL:1999, but supported even
earlier using non-standard syntax by most databases.
• Syntax illustrated here may not work exactly on your database
system; check the system manuals

Database System Concepts - 7th Edition 5.25 ©Silberschatz, Korth and Sudarshan
 Triggering Events and Actions in SQL

 Triggering event can be insert, delete or update

 Triggers on update can be restricted to specific attributes
• For example, after update of takes on grade
 Values of attributes before and after an update can be referenced
• referencing old row as : for deletes and updates
• referencing new row as : for inserts and updates
 Triggers can be activated before an event, which can serve as extra
constraints. For example, convert blank grades to null.

create trigger setnull_trigger before update of takes

referencing new row as nrow
for each row
when (nrow.grade = ' ')
begin atomic
set nrow.grade = null;
end;

Database System Concepts - 7th Edition 5.26 ©Silberschatz, Korth and Sudarshan
 Trigger in MySQL

delimiter //
create trigger trigger_name
trigger_time triger_event on table_name for each row
beign
…
end; //
delimiter ;

trigger_time trigger_event on Table_name

BEFORE INSERT
UPDATE
AFTER
DELETE

Database System Concepts - 7th Edition 5.27 ©Silberschatz, Korth and Sudarshan
 Trigger in MySQL before insert

after insert (only one is inserted)

Database System Concepts - 7th Edition 5.28 ©Silberschatz, Korth and Sudarshan
 Trigger to Maintain credits_earned value

 create trigger credits_earned after update of takes on (grade)

referencing new row as nrow
referencing old row as orow
for each row
when nrow.grade <> 'F' and nrow.grade is not null
and (orow.grade = 'F' or orow.grade is null)
begin atomic
update student
set tot_cred= tot_cred +
(select credits
from course
where course.course_id= nrow.course_id)
where student.id = nrow.id;
end;

Database System Concepts - 7th Edition 5.29 ©Silberschatz, Korth and Sudarshan
 Trigger to Maintain credits_earned value

Database System Concepts - 7th Edition 5.30 ©Silberschatz, Korth and Sudarshan
 Recursive Queries

Database System Concepts - 7th Edition 5.31 ©Silberschatz, Korth and Sudarshan
 Recursion in SQL

 SQL:1999 permits recursive view definition

 Example: find which courses are a prerequisite, whether directly or
indirectly, for a specific course

with recursive rec_prereq(course_id, prereq_id) as (

select course_id, prereq_id
from prereq
union
select rec_prereq.course_id, prereq.prereq_id,
from rec_rereq, prereq
where rec_prereq.prereq_id = prereq.course_id
)
select ∗
from rec_prereq;
This example view, rec_prereq, is called the transitive closure of the prereq
relation

Database System Concepts - 7th Edition 5.32 ©Silberschatz, Korth and Sudarshan
 Recursion Example in MySQL
CTE

course-id
preregial

C3319

CS319
13315
Cs 190
/
Cs 101
29319

Database System Concepts - 7th Edition 5.33 ©Silberschatz, Korth and Sudarshan
 Find all the direct and indirect supervisors of Bob

or 'create temporary table

Database System Concepts - 7th Edition 5.34 ©Silberschatz, Korth and Sudarshan
 Advanced Aggregation Features

Database System Concepts - 7th Edition 5.35 ©Silberschatz, Korth and Sudarshan
 Ranking

 Ranking is done in conjunction with an order by specification.

 Suppose we are given a relation
student_grades(ID, GPA)
giving the grade-point average of each student
 Find the rank of each student.
select ID, rank() over (order by GPA desc) as s_rank
from student_grades
 An extra order by clause is needed to get them in sorted order
select ID, rank() over (order by GPA desc) as s_rank
from student_grades
order by s_rank
 Ranking may leave gaps: e.g. if 2 students have the same top GPA, both
have rank 1, and the next rank is 3
• dense_rank does not leave gaps, so next dense rank would be 2

Database System Concepts - 7th Edition 5.36 ©Silberschatz, Korth and Sudarshan
 Modify the corresponding part in DDL.sql and smallRelationsInsertFile.sql

Database System Concepts - 7th Edition 5.37 ©Silberschatz, Korth and Sudarshan
 Ranking: MySQL Example

null is the smallest

null is the largest
Database System Concepts - 7th Edition 5.38 ©Silberschatz, Korth and Sudarshan
 Ranking

 Ranking can be done using basic SQL aggregation, but resultant query is
very inefficient
select ID, (1 + (select count(*)
from student_grades B
where B.GPA > A.GPA)) as s_rank
from student_grades A
order by s_rank;

Database System Concepts - 7th Edition 5.39 ©Silberschatz, Korth and Sudarshan
 Ranking (Cont.)

 Ranking can be done within partition of the data.

 “Find the rank of students within each department.”
select ID, dept_name,
rank () over (partition by dept_name order by GPA desc)
as dept_rank
from dept_grades
order by dept_name, dept_rank;
 Multiple rank clauses can occur in a single select clause.
 Ranking is done after applying group by clause/aggregation
 Can be used to find top-n results
• More general than the limit n clause supported by many databases,
since it allows top-n within each partition

Database System Concepts - 7th Edition 5.40 ©Silberschatz, Korth and Sudarshan
 Ranking (Cont.)

Database System Concepts - 7th Edition 5.41 ©Silberschatz, Korth and Sudarshan
 Ranking (Cont.)

 Other ranking functions:

• percent_rank (within partition, if partitioning is done)
• cume_dist (cumulative distribution)
 fraction of tuples with preceding values
• row_number (non-deterministic in presence of duplicates)

• https://dev.mysql.com/doc/refman/9.0/en/window-function-descriptions.html

 SQL:1999 permits the user to specify nulls first or nulls last. (Not supported
by MySQL, but supported by PostgreSQL, SQLite)
select ID,
rank ( ) over (order by GPA desc nulls last) as s_rank
from student_grades

Database System Concepts - 7th Edition 5.42 ©Silberschatz, Korth and Sudarshan
 Ranking (Cont.)

 For a given constant n, the ranking function ntile(n) takes the tuples in
each partition in the specified order and divides them into n buckets with
equal numbers of tuples.
 E.g.,
select ID, ntile(4) over (order by GPA desc) as quartile
from student_grades;

Database System Concepts - 7th Edition 5.43 ©Silberschatz, Korth and Sudarshan
 Windowing

 Used to smooth out random variations.

 E.g., moving average: “Given sales values for each date, calculate for
each date the average of the sales on that day, the previous day, and the
next day”
 Window specification in SQL:
• Given relation sales(date, value)
select date, sum(value) over
(order by date between rows 1 preceding and 1 following)
from sales

Database System Concepts - 7th Edition 5.44 ©Silberschatz, Korth and Sudarshan
 Windowing

 Examples of other window specifications:

• between rows unbounded preceding and current
• rows unbounded preceding
• range between 10 preceding and current row
All rows with values between current row value –10 to current value

• range interval 10 day preceding
 Not including current row

Database System Concepts - 7th Edition 5.45 ©Silberschatz, Korth and Sudarshan
 Windowing: MySQL

16
23

28

the window
do in
window
-

Database System Concepts - 7th Edition 5.46 ©Silberschatz, Korth and Sudarshan
 Windowing (Cont.)

 Can do windowing within partitions

 E.g., Given a relation transaction (account_number, date_time, value),
where value is positive for a deposit and negative for a withdrawal
• “Find total balance of each account after each transaction on the
account”
select account_number, date_time,
sum (value) over
(partition by account_number
order by date_time
rows unbounded preceding)
as balance
from transaction
order by account_number, date_time

Database System Concepts - 7th Edition 5.47 ©Silberschatz, Korth and Sudarshan
 Windowing (Cont.)

Database System Concepts - 7th Edition 5.48 ©Silberschatz, Korth and Sudarshan
 Aggregation on Multidimensional Data

Database System Concepts - 7th Edition 5.49 ©Silberschatz, Korth and Sudarshan
 Example sales relation

... ... ... ...

Database System Concepts - 7th Edition ... ... ...
5.50 ... ©Silberschatz, Korth and Sudarshan
Database System Concepts - 7th Edition 5.51 ©Silberschatz, Korth and Sudarshan
 Cross Tabulation of sales by item_name and color

 The table above is an example of a cross-tabulation (cross-tab), also

referred to as a pivot-table.
• Values for one of the dimension attributes form the row headers
• Values for another dimension attribute form the column headers
• Other dimension attributes are listed on top
• Values in individual cells are (aggregates of) the values of the
dimension attributes that specify the cell.

Database System Concepts - 7th Edition 5.52 ©Silberschatz, Korth and Sudarshan
 Pivot table by MySQL

Database System Concepts - 7th Edition 5.53 ©Silberschatz, Korth and Sudarshan
 Data Cube

 A data cube is a multidimensional generalization of a cross-tab

 Can have n dimensions; we show 3 below
 Cross-tabs can be used as views on a data cube

Database System Concepts - 7th Edition 5.54 ©Silberschatz, Korth and Sudarshan
 Hierarchies on Dimensions
Hierarchy on dimension attributes: lets dimensions to be viewed at
different levels of detail
•E.g., the dimension DateTime can be used to aggregate by hour of
day, date, day of week, month, quarter or year

Database System Concepts - 7th Edition 5.55 ©Silberschatz, Korth and Sudarshan
 Cross Tabulation With Hierarchy

 Cross-tabs can be easily extended to deal with hierarchies

• Can drill down or roll up on a hierarchy

Database System Concepts - 7th Edition 5.56 ©Silberschatz, Korth and Sudarshan
 Relational Representation of Cross-tabs

 Cross-tabs can be represented

as relations
• The value all is used to
represent aggregates.
• The SQL standard actually
uses null values in place of
all despite confusion with
regular null values.

Database System Concepts - 7th Edition 5.57 ©Silberschatz, Korth and Sudarshan
 Extended Aggregation to Support OLAP

 The cube operation computes union of group by’s on every subset of the
specified attributes
 Example relation for this section
sales(item_name, color, size, quantity)
 E.g., consider the query
select item_name, color, size, sum(quantity)
from sales
group by cube(item_name, color, size)
This computes the union of eight different groupings of the sales relation:
{ (item_name, color, size), (item_name, color),
(item_name, size), (color, size),
(item_name), (color),
(size), ()}
where ( ) denotes an empty group by list.
 For each grouping, the result contains the null value
for attributes not present in the grouping.

Database System Concepts - 7th Edition 5.58 ©Silberschatz, Korth and Sudarshan
 Extended Aggregation to Support OLAP
(Cont.)

 The rollup construct generates union on every prefix of specified list of

attributes
 E.g.,
select item_name, color, size, sum(quantity)
from sales
group by rollup(item_name, color, size)
• Generates union of four groupings:
{ (item_name, color, size), (item_name, color), (item_name), ( ) }

Database System Concepts - 7th Edition 5.59 ©Silberschatz, Korth and Sudarshan
 rollup in MySQL

Database System Concepts - 7th Edition 5.60 ©Silberschatz, Korth and Sudarshan
 cube and rollup in PostgreSQL

Database System Concepts - 7th Edition 5.61 ©Silberschatz, Korth and Sudarshan
 Online Analytical Processing Operations

 Relational representation of cross-tab that we saw earlier, but with null in

place of all, can be computed by
 select item_name, color, sum(quantity)
from sales
group by cube(item_name, color)
 The function grouping() can be applied on an attribute
• Returns 1 if the value is a null value representing all, and returns 0 in all
other cases.
select item_name, color, size, sum(quantity),
grouping(item_name) as item_name_flag,
grouping(color) as color_flag,
grouping(size) as size_flag,
from sales
group by cube(item_name, color, size)

Database System Concepts - 7th Edition 5.62 ©Silberschatz, Korth and Sudarshan
 PostgreSQL:

Database System Concepts - 7th Edition 5.63 ©Silberschatz, Korth and Sudarshan
Database System Concepts - 7th Edition 5.64 ©Silberschatz, Korth and Sudarshan
 Online Analytical Processing Operations

Let me summarize the concepts of OLAP and OLTP which you may have heard
(https://questions.finalroundai.com/1967/what-is-oltp-vs-olap).
OLTP and OLAP are two types of data processing that serve different purposes.
OLTP, or Online Transaction Processing, is a class of systems that manage
transaction-oriented applications on the internet. OLTP systems are primarily
used for recording day-to-day transactions such as updating, inserting, and
deleting data. They are characterized by a large number of short online
transactions and are designed to ensure data integrity in multi-access
environments. Examples are ATM transactions, order entry, retail sales, etc.
On the other hand, OLAP, or Online Analytical Processing, is a category of
software tools that enables analysts, managers, or executives to gain insight into
data through fast, interactive access to a variety of possible views of information.
Unlike OLTP where transactions are short and fast, OLAP is characterized by
complex queries with multi-dimensional view of data for analysis. It is typically
used in data mining or retrieving multi-dimensional information. OLAP helps top-
level executives to ascertain business-driving factors and trends.

Database System Concepts - 7th Edition 5.65 ©Silberschatz, Korth and Sudarshan
 End of Chapter 5

Database System Concepts - 7th Edition 5.66 ©Silberschatz, Korth and Sudarshan
 Homework:

• Write and run those MySQL example codes in the slides.

• Optional (won’t be collected and won’t be marked):

If you have more time, you may want to try some of the
Questions 5.8, 5.9, 5.22, 5.23 on Page 232 of the textbook.
Read the solutions (on canvas) afterwards.

Database System Concepts - 7th Edition 5.67 ©Silberschatz, Korth and Sudarshan