SQL Games We Can Play

inside PL/SQL

Steven Feuerstein
[email protected]
Quest Software

Copyright 2000-2006 Steven Feuerstein - Page 1

 Software used in presentation

 You can download all my training

materials and demonstration scripts
from:
http://oracleplsqlprogramming.com/resources.html

 All scripts I run may be obtained from:

http://oracleplsqlprogramming.com/downloads/demo.zip

name of file
Copyright 2000-2006 Steven Feuerstein - Page 2
 Let the Games Begin!

 Turbocharged SQL with BULK

COLLECT and FORALL
 Deceptively simple SQL with table
functions
 Code above the fray with record-level
DML.

Copyright 2000-2006 Steven Feuerstein - Page 3

 Turbo-charged SQL with
BULK COLLECT and FORALL

 Improve the performance of multi-row SQL

operations by an order of magnitude or more
with bulk/array processing in PL/SQL!
CREATE OR REPLACE PROCEDURE upd_for_dept (
dept_in IN employee.department_id%TYPE
,newsal_in IN employee.salary%TYPE)
IS
CURSOR emp_cur IS
SELECT employee_id,salary,hire_date
FROM employee WHERE department_id = dept_in;
BEGIN
FOR rec IN emp_cur LOOP
UPDATE employee SET salary = newsal_in
WHERE employee_id = rec.employee_id; “Conventional
END LOOP; binds” (and lots
END upd_for_dept;
Copyright 2000-2006 Steven Feuerstein - Page 4
of them!)
 Conventional Bind
Oracle server

PL/SQL Runtime Engine SQL Engine

PL/SQL block
Procedural
statement
FOR rec IN emp_cur LOOP executor
UPDATE employee SQL
SET salary = ... statement
WHERE employee_id = executor
rec.employee_id;
END LOOP;
Performance penalty
for many “context
switches”

Copyright 2000-2006 Steven Feuerstein - Page 5

 Enter the “Bulk Bind”
Oracle server

PL/SQL Runtime Engine SQL Engine

PL/SQL block
Procedural
statement
FORALL indx IN executor
deptlist.FIRST.. SQL
deptlist.LAST statement
UPDATE employee executor
SET salary = ...
WHERE employee_id =
deptlist(indx);
Much less overhead for
context switching

Copyright 2000-2006 Steven Feuerstein - Page 6

 Use the FORALL Bulk Bind Statement
 Instead of executing repetitive, individual DML
statements, you can write your code like this:
PROCEDURE remove_emps_by_dept (deptlist dlist_t)
IS
BEGIN
FORALL aDept IN deptlist.FIRST..deptlist.LAST
DELETE FROM emp WHERE deptno = deptlist(aDept);
END;

 Things to be aware of:

– You MUST know how to use collections to use this feature!
– Only a single DML statement is allowed per FORALL.
– SQL%BULK_ROWCOUNT returns the number of rows affected by each
row in the binding array.
– Prior to Oracle10g, the binding array must be sequentially filled.
– Use SAVE EXCEPTIONS to continue past errors. bulktiming.sql
bulk_rowcount.sql
Copyright 2000-2006 Steven Feuerstein - Page 7 bulkexc.sql
 Use BULK COLLECT INTO for Queries

DECLARE
Declare a TYPE employees_aat IS TABLE OF employees%ROWTYPE
collection of INDEX BY BINARY_INTEGER;
records to hold
l_employees employees_aat;
the queried data.
BEGIN
SELECT *
BULK COLLECT INTO l_employees
Use BULK FROM employees;
COLLECT to
retrieve all rows. FOR indx IN 1 .. l_employees.COUNT
LOOP
process_employee (l_employees(indx));
Iterate through the END LOOP;
END;
collection
contents with a
loop.
bulkcoll.sql
Copyright 2000-2006 Steven Feuerstein - Page 8
 Limit the number of rows returned by
BULK COLLECT
CREATE OR REPLACE PROCEDURE bulk_with_limit
(deptno_in IN dept.deptno%TYPE)
IS Use the LIMIT clause with the
CURSOR emps_in_dept_cur IS INTO to manage the amount
SELECT *
FROM emp
of memory used with the
WHERE deptno = deptno_in; BULK COLLECT operation.

TYPE emp_tt IS TABLE OF emp%ROWTYPE;

emps emp_tt;
BEGIN
OPEN three_cols_cur; WARNING!
LOOP
FETCH emps_in_dept_cur BULK COLLECT will not raise
BULK COLLECT INTO emps
LIMIT 100;
NO_DATA_FOUND if no rows
are found.
EXIT WHEN emps.COUNT = 0;
Best to check contents of
process_emps (emps);
END LOOP; collection to confirm that
END bulk_with_limit; something was retrieved.
bulklimit.sql
Copyright 2000-2006 Steven Feuerstein - Page 9
 Tips and Fine Points

 Use bulk binds in these circumstances:

– Recurring SQL statement in PL/SQL loop. Oracle
recommended threshold: five rows!
 Bulk bind rules:
– Can be used with any kind of collection; Collection subscripts
cannot be expressions; The collections must be densely filled
(pre-10g); If error occurs, prior successful DML statements
are NOT ROLLED BACK.
 Bulk collects: emplu.pkg
cfl_to_bulk*.*
– Can be used with implicit and explicit cursors
– Collection is always filled sequentially, starting at row 1
Copyright 2000-2006 Steven Feuerstein - Page 10
 Oracle9i
Dynamic FORALL Example
 This example shows the use of bulk binding and
collecting, plus application of the RETURNING clause.
CREATE TYPE NumList IS TABLE OF NUMBER;
CREATE TYPE NameList IS TABLE OF VARCHAR2(15);

PROCEDURE update_emps (
col_in IN VARCHAR2, empnos_in IN numList) IS
enames NameList;
BEGIN
FORALL indx IN empnos_in.FIRST .. empnos_in.LAST
EXECUTE IMMEDIATE
'UPDATE emp SET ' || col_in || ' = ' || col_in
|| ' * 1.1 WHERE empno = :1
RETURNING ename INTO :2'
USING empnos_in (indx ) Notice that empnos_in
RETURNING BULK COLLECT INTO enames; is indexed, but enames
... is not.
END;
Copyright 2000-2006 Steven Feuerstein - Page 11
 Oracle9i
Dynamic BULK COLLECT
 Now you can even avoid the OPEN FOR and just
grab your rows in a single pass!
CREATE OR REPLACE PROCEDURE fetch_by_loc (loc_in IN VARCHAR2)
IS
TYPE numlist_t IS TABLE OF NUMBER;
TYPE namelist_t IS TABLE OF VARCHAR2 (15);
emp_cv sys_refcursor;
empnos numlist_t;
enames namelist_t;
sals numlist_t;
BEGIN
OPEN emp_cv FOR 'SELECT empno, ename FROM emp_' || loc_in;
FETCH emp_cv BULK COLLECT INTO empnos, enames;
CLOSE emp_cv;

EXECUTE IMMEDIATE 'SELECT sal FROM emp_' || loc_in

BULK COLLECT INTO sals; With Oracle9iR2
END; you can also fetch
into collections of
records.
Copyright 2000-2006 Steven Feuerstein - Page 12
 Excellent Exception Handling
for Bulk Operations in Oracle 9i Database R2

 Allows you to continue past errors and obtain error

information for each individual operation (for dynamic
and static SQL).

CREATE OR REPLACE PROCEDURE load_books (books_in IN book_obj_list_t)

IS
bulk_errors EXCEPTION;
PRAGMA EXCEPTION_INIT ( bulk_errors, -24381 );
BEGIN
FORALL indx IN books_in.FIRST..books_in.LAST Allows processing of all
SAVE EXCEPTIONS
rows, even after an error
INSERT INTO book values (books_in(indx));
EXCEPTION
occurs.
WHEN BULK_ERRORS THEN
FOR indx in 1..SQL%BULK_EXCEPTIONS.COUNT New cursor
LOOP attribute, a pseudo-
log_error (SQL%BULK_EXCEPTIONS(indx)); collection
END LOOP;
END;

Copyright 2000-2006 Steven Feuerstein - Page 13 bulkexc.sql

 Should you ever use a cursor FOR loop?

 Oracle10g will automatically optimize cursor

FOR loops to perform at levels comparable to
BULK COLLECT!
– So what the heck, keep or write your cursor FOR
loops -- at least if they contain no DML.
 Also stick with a cursor FOR loop....
– If you want to do complex DML processing on each
row as it is queried – and possibly halt further
fetching, and you can't use SAVE EXCEPTIONS.
 Otherwise, moving to BULK COLLECT and
FORALL is a smart move!
10g_optimize_cfl.sql
cfl_vs_bulkcollect.sql
cfl_to_bulk.sql
Copyright 2000-2006 Steven Feuerstein - Page 14
 Oracle10g More flexibility with FORALL

 In Oracle10g, the FORALL driving array no

longer needs to be processed sequentially.
 Use the INDICES OF clause to use only the
row numbers defined in another array.
 Use the VALUES OF clause to use only the
values defined in another array.

Copyright 2000-2006 Steven Feuerstein - Page 15

 Oracle10g Using INDICES OF

 It only DECLARE
TYPE employee_aat IS TABLE OF employee.employee_id%TYPE
processes INDEX BY PLS_INTEGER;
l_employees employee_aat;
the rows TYPE boolean_aat IS TABLE OF BOOLEAN
INDEX BY PLS_INTEGER;
with row l_employee_indices boolean_aat;

numbers BEGIN
l_employees (1) := 7839;
matching l_employees (100) := 7654;
l_employees (500) := 7950;
the defined --
l_employee_indices (1) := TRUE;
rows of the l_employee_indices (500) := TRUE;
--
driving FORALL l_index IN INDICES OF l_employee_indices

array. UPDATE employee

SET salary = 10000
WHERE employee_id = l_employees (l_index);
END;

Copyright 2000-2006 Steven Feuerstein - Page 16 10g_indices_of.sql

 Oracle10g Using VALUES OF

DECLARE
 It only TYPE employee_aat IS TABLE OF employee.employee_id%TYPE

processes INDEX BY PLS_INTEGER;

l_employees employee_aat;
the rows TYPE indices_aat IS TABLE OF PLS_INTEGER
INDEX BY PLS_INTEGER;
with row l_employee_indices
BEGIN
indices_aat;

numbers l_employees (-77) := 7820;

l_employees (13067) := 7799;
matching l_employees (99999999) := 7369;

the content --
l_employee_indices (100) := -77;
of a row in l_employee_indices (200) := 99999999;
--
the driving FORALL l_index IN VALUES OF l_employee_indices
UPDATE employee
array. SET salary = 10000
WHERE employee_id = l_employees (l_index);
END;

Copyright 2000-2006 Steven Feuerstein - Page 17 10g_values_of.sql

 Table Functions

 A table function is a function that you can call in the

FROM clause of a query, and have it be treated as if
it were a relational table.
 Table functions allow you to perform arbitrarily
complex transformations of data and then make that
data available through a query.
– Not everything can be done in SQL.
 Combined with REF CURSORs, you can now more
easily transfer data from within PL/SQL to host
environments.
– Java, for example, works very smoothly with cursor variables

Copyright 2000-2006 Steven Feuerstein - Page 18

 Applications for pipelined functions

 Execution functions in parallel.

– In Oracle9i Database Release 2 and above, you can use the
PARALLEL_ENABLE clause to allow your pipelined function to
participate fully in a parallelized query.
– Critical in data warehouse applications.
 Improve speed of delivery of data to web pages.
– Use a pipelined function to "serve up" data to the webpage and
allow users to being viewing and browsing, even before the
function has finished retrieving all of the data.

Copyright 2000-2006 Steven Feuerstein - Page 19

 Building a table function

 A table function must return a nested table or

varray based on a schema-defined type, or type
defined in a PL/SQL package.
 The function header and the way it is called
must be SQL-compatible: all parameters use
SQL types; no named notation.
– In some cases (streaming and pipelines functions), the IN
parameter must be a cursor variable -- a query result set.

Copyright 2000-2006 Steven Feuerstein - Page 20

 Simple table function example

 Return a list of names as a nested table, and

then call that function in the FROM clause.
CREATE OR REPLACE FUNCTION lotsa_names (
base_name_in IN VARCHAR2, count_in IN INTEGER
)
RETURN names_nt
SELECT column_value
IS
FROM TABLE (
retval names_nt := names_nt ();
lotsa_names ('Steven'
BEGIN
, 100)) names;
retval.EXTEND (count_in);
COLUMN_VALUE
FOR indx IN 1 .. count_in
LOOP
retval (indx) :=
------------
base_name_in || ' ' || indx;
Steven 1
END LOOP;

RETURN retval;
... tabfunc_scalar.sql
END lotsa_names;
Copyright 2000-2006 Steven Feuerstein - Page 21
Steven 100
 Streaming data with table functions

 You can use table functions to "stream" data through

several stages within a single SQL statement.
CREATE TYPE tickertype AS OBJECT (
ticker VARCHAR2 (20)
, pricedate DATE
, pricetype VARCHAR2 (1)
, price NUMBER
);

CREATE TYPE tickertypeset AS TABLE OF tickertype;

/

CREATE TABLE tickertable (

ticker VARCHAR2(20),
pricedate DATE,
pricetype VARCHAR2(1),
price NUMBER)
/

tabfunc_streaming.sql

Copyright 2000-2006 Steven Feuerstein - Page 22

 Streaming data with table functions - 2

 In this example, transform each row of the

stocktable into two rows in the tickertable.
CREATE OR REPLACE PACKAGE refcur_pkg
IS
TYPE refcur_t IS REF CURSOR
RETURN stocktable%ROWTYPE;
END refcur_pkg;
/

CREATE OR REPLACE FUNCTION stockpivot (dataset refcur_pkg.refcur_t)

RETURN tickertypeset ...

BEGIN
INSERT INTO tickertable
SELECT *
FROM TABLE (stockpivot (CURSOR (SELECT *
FROM stocktable)));
END;
/

Copyright 2000-2006 Steven Feuerstein - Page 23 tabfunc_streaming.sql

 Use pipelined functions to enhance
performance.
CREATE FUNCTION StockPivot (p refcur_pkg.refcur_t)
RETURN TickerTypeSet PIPELINED

 Pipelined functions allow you to return data

iteratively, asynchronous to termination of the
function.
– As data is produced within the function, it is passed back to the calling
process/query.
 Pipelined functions can be defined to support parallel
execution.
– Iterative data processing allows multiple processes to work on that data
simultaneously.

Copyright 2000-2006 Steven Feuerstein - Page 24

Piping rows out from a pipelined function

CREATE FUNCTION stockpivot (p refcur_pkg.refcur_t)

RETURN tickertypeset
Add PIPELINED
keyword to header PIPELINED
IS
out_rec tickertype :=
tickertype (NULL, NULL, NULL);
in_rec p%ROWTYPE;
BEGIN
LOOP
FETCH p INTO in_rec;
EXIT WHEN p%NOTFOUND;
out_rec.ticker := in_rec.ticker;
Pipe a row of data out_rec.pricetype := 'O';
back to calling block out_rec.price := in_rec.openprice;
or query
PIPE ROW (out_rec);
END LOOP;
CLOSE p;
RETURN...nothing at
all! RETURN;
END;

tabfunc_setup.sql
Copyright 2000-2006 Steven Feuerstein - Page 25 tabfunc_pipelined.sql
 Enabling Parallel Execution

 The table function's parameter list must consist only

of a single strongly-typed REF CURSOR.
 Include the PARALLEL_ENABLE hint in the program
header.
– Choose a partition option that specifies how the function's execution
should be partitioned.
– "ANY" means that the results are independent of the order in which the
function receives the input rows (through the REF CURSOR).

{[ORDER | CLUSTER] BY column_list}

PARALLEL_ENABLE ({PARTITION p BY
[ANY | (HASH | RANGE) column_list]} )

Copyright 2000-2006 Steven Feuerstein - Page 26

 Table functions - Summary

 Table functions offer significant new flexibility

for PL/SQL developers.
 Consider using them when you...
– Need to pass back complex result sets of data through the SQL
layer (a query);
– Want to call a user defined function inside a query and execute
it as part of a parallel query.

Copyright 2000-2006 Steven Feuerstein - Page 27

 Oracle9i Record-based DML

 PL/SQL records (similar in structure to a row

in a table) offer powerful ways to manipulate
data
– Prior to Oracle9i R2, however, records could not
be used in DML statements
 That restriction has now been lifted
– You can INSERT specifying a record rather than
individual fields of the record
– You can UPDATE an entire row with a record

Copyright 2000-2006 Steven Feuerstein - Page 28

 Record-based Inserts

DECLARE
TYPE book_list_t IS TABLE OF books%ROWTYPE;
my_books book_list_t := book_list_t();
BEGIN
my_books.EXTEND (2);

my_books(1).isbn := '1-56592-335-9';
my_books(1).title := 'ORACLE PL/SQL PROGRAMMING';

my_books(2).isbn := '0-596-00121-5';
my_books(2).title := 'ORACLE PL/SQL BEST PRACTICES';

FORALL indx IN my_books.FIRST .. my_books.LAST

INSERT INTO books VALUES my_books(indx);
END;

 This example shows a record-based insert

inside the high-speed FORALL statement
Copyright 2000-2006 Steven Feuerstein - Page 29
 Record-based Updates

DECLARE
my_book books%ROWTYPE;
BEGIN
my_book.isbn := '1-56592-335-9';
my_book.title := 'ORACLE PL/SQL PROGRAMMING';
my_book.summary := 'General user guide and reference';
my_book.author := 'FEUERSTEIN, STEVEN AND BILL PRIBYL';
my_book.page_count := 950; -- new page count for 3rd edition
 
UPDATE books
SET ROW = my_book
WHERE isbn = my_book.isbn;
END;

 You can only update the entire ROW, and not a

subset via, say, a programmer-defined record type
Copyright 2000-2006 Steven Feuerstein - Page 30
 Stop taking SQL for granted!

 Every SQL statement is a hard-coding of your

data structures in your code, so...
– Control your SQL.
– Avoid writing SQL.
– And fully leverage PL/SQL when you do write SQL.
 Take advantage of key PL/SQL functionality.
– FORALL
– BULK COLLECT
– Table functions
– Record-based DML
Copyright 2000-2006 Steven Feuerstein - Page 31