Bryn Llewellyn

Product Manager for PL/SQL Oracle Corporation

PL/SQL enhancements in Oracle9i

paper #129, Oracle OpenWorld, San Francisco, Tue 4-Dec-2001

But before I start

OTN homepage Technologies PL/SQL

otn.oracle.com/tech/pl_sql

What are the Benefits?

y Speed and scalability y Functionality y Usability for the developer

Whats been enhanced?

y Implementation y Language features y Supplied packages

What kind of enhancements?

y Transparent y Semi-transparent y New features New constructs in the language Richer APIs in the supplied packages

Here comes the list

Implementation
Feature Native compilation of PL/SQL Manipulating records faster by up to 5x Inter -package calls faster by up to 1.5x Inter-package Utl_Tcp native implementation Common SQL parser Perf 3 3 3 3 3 3 Funct Use

Language features
Feature Table functions Cursor expressions Multilevel collections Exception handling in bulk binding DML operations Bulk binding in native dynamic SQL CASE statements and CASE expressions 3 Perf 3 3 3 Funct 3 3 3 3 Use 3 3 3 3 3 3

Exception handling in bulk DML

y Consider giving employees a 10% raise where employee_id among the values in a table of numbers y forall j in id.first..id.last y There might be a per department salary cap so some intended updates will fail y Pre-Oracle9i the whole bulk statement failed y Now the OK updates will fail and the exceptions can be reported
y save exceptions sql%bulk_exceptions collection

Multilevel collection syntax

for j in my_multi.first..my_multi.last loop for k in my_multi(j).first..my_multi(j).last loop Show ( my_multi(j)(k) ); end loop; end loop;

y Collections can be nested to arbitrary depth

CASE statement syntax

case n when 1 when 2 when 3 else end case; then Action1; then Action2; then Action3; ActionOther;

CASE statement syntax

case when p = 1 then Action1; when q = 1 then Action2; when r > 1 then Action3; else ActionOther; end case;

y searched variety

CASE expression syntax

text := case when when when else end; n 1 2 3 then 'one' then 'two' then 'three' 'other'

CASE expression syntax

text := case when p = 1 then 'one' when r = 2 then 'two' when q > 1 then 'three' else 'other' end;

y searched variety

CASE_NOT_FOUND exception
begin p:=0; q:=0; r:=0; case when p = 1 then Action1; when r = 2 then Action2; when q > 1 then Action3; end case; exception when case_not_found then Show ('Exception: case_not_found' ); end;

Language features cont.

Feature VARCHAR2 < -> NVARCHAR2 (etc) assignment <-> VARCHAR2 < -> CLOB assignment <-> SUBSTR and INSTR w/ CLOB Seamless access to new SQL features e.g. MERGE, multitable insert, new time datatypes datatypes OO : Schema evolution OO: inheritance support a.k.a. subtyping and polymorphism 3 3 3 Perf Funct 3 3 3 Use

Supplied packages
Feature Utl_Http enhanced Utl_ Raw enhanced Utl_Raw Utl_File enhanced Nineteen new packages Perf 3 Funct 3 3 3 3 Use

Utl_Http enhancements
y Used to send HTTP request and handle the reply mechanically especially in B2B applications y Oracle9i adds

POST method arbitarily long request Authentication Access to return status code RAW reply cookie support

y I.e. provides full support for the semantics that can be expressed via HTTP y Full functionality for character set conversion for request and reply

I cant possibly talk about everything on that list in detail!

So instead Im going to pick just a couple to look at in depth

 Native compilation Table functions and cursor expressions

Native compilation of PL/SQL

y When PL/SQL is used as a thin wrapper for SQL its execution speed is rarely an issue y But we see an increasing trend to use PL/SQL for computationally intensive database independent tasks y Here it is the execution speed of the PL/SQL code that determines the performance

Background
y Pre-Oracle9i, compilation of PL/SQL source code always results in bytecode which is stored in the database and interpreted at run-time by a virtual machine implemented within ORACLE

Whats new?
y In Oracle9i PL/SQL source code may optionally be compiled into native object code which is linked into ORACLE

Whats the benefit?

y Speed increased by up to 40% y Thin wrapper programs wont speed up so much but they wont slow down!

How does it work?

y The code generator translates the PL/SQL source code into C source code y This is compiled on the given platform and stored as object files on the filesystem y and then linked into ORACLE

How do you do it?

y One-time DBA setup y The developer chooses at compile time via the session parameter
plsql_compiler_flags

One-time DBA setup

y Specify 3rd party utilities for compiling and linking Should be owned by the ORACLE owner Only the ORACLE owner should have write access y Specify directories for the compiled object libraries y Done via system parameters

Developer choice
alter session set plsql_compiler_flags = 'NATIVE' /* or 'INTERPRETED' */;

y sets the compilation mode for subsequently compiled PL/SQL library units

Oracle recommends
y All PL/SQL library units that are called from a given natively compiled top-level unit should also be compiled natively there is a cost for the context switch when a library unit compiled in native mode invokes one compiled in interpreted mode y Recommendation includes the Oracle-supplied library units (by default these are compiled in interpreted mode)

Oracle9i in Action
y 170 Systems, Inc have been an Oracle Partner for eleven years and participated in the Beta Program for the Oracle9i Database with particular interest in PL/SQL Native Compilation y They have now certified their 170 MarkView Document Management and Imaging System against Oracle9i y They have updated the install scripts to optionally turn on Native Compilation

170 MarkView Document Management and Imaging System

y Provides Content Management, Document Management, Imaging and Workflow solutions y Tightly integrated with the Oracle9i Database, Oracle9i Application Server and the Oracle E-Business Suite y Enables businesses to capture and manage all of their information online in a single, unified system

170 MarkView
y Large-scale multi-user, multi-access system y Customers include

British Telecommunications E*TRADE Group the Apollo Group the University of Pennsylvania

y Very large numbers of documents, images, concurrent users, and high transaction rates y Performance and scalability especially important

170 MarkView
y Business logic, including preparation of data for presentation, is implemented in the database in PL/SQL y Involves complex logic and intensive string processing supported by stacks and lists of values modeled as PL/SQL collections.

y Visit 170 Systems at Booth # 1914

170 Systems
y Have observed a performance increase of up to 40% for computationally intensive routines and no performance degradation

Native Compilation - summary

y Its a semi-transparent enhancement y It gives you improved performance y Try it !

 Table functions and cursor expressions

Table functions and cursor expressions

y Cursor variables recap y Manipulating cursor expressions in PL/SQL y Using a cursor expression as an actual parameter to a PL/SQL function y Table functions pre-Oracle9i y Oracle9i pipelined table functions y Daisy-chaining table functions y Parallelization

Cursor variables - recap

y Supported pre-Oracle9i y Allow encapsulation of logic specific to tuples of a particular format to be independent of the concrete SELECT statement which retrieves them

Cursor variables - recap

procedure Bulk_Fetch_From_Cursor ( p_cursor in sys_refcursor ) is... y SYS_REFCURSOR is a new Oracle9i

usability feature y Defines a generic weak cursor once and for all

Set up the target

y Declare an index-by PL/SQL table for the fetched rows
...is type names_t is table of varchar2(4000) index by binary_integer; the_names names_t; begin...

Do the bulk fetch

fetch p_cursor bulk collect into the_names;

y Works pre-Oracle9i y Performance boost because reduces context switching between PL/SQL and SQL engines

Process the results

for j in the_names.first..the_names.last loop Show ( the_names(j) ); end loop;

y Works pre-Oracle9i

Invoke the procedure

declare the_cursor sys_refcursor; begin open the_cursor for 'select last_name from employees order by last_name'; Bulk_Fetch_From_Cursor ( the_cursor ); close the_cursor;

y Note were using Native Dynamic SQL

New in Oracle9i
y This is just one example of bulk binding working with native dynamic SQL y The combination now works in all situations Defining
SELECT

In-binding

FORALL USING

Out-binding

FORALL USING RETURNING

Table functions and cursor expressions

y Cursor variables recap

y Manipulating cursor expressions in PL/SQL

y Using a cursor expression as an actual parameter to a PL/SQL function y Table functions pre-Oracle9i y Oracle9i pipelined table functions y Daisy-chaining table functions y Parallelization

Cursor expressions and PL/SQL

y Suppose we want to make a pretty print listing departments and under each department list its employees

Explicit 3GL approach

for department in ( select ... ) loop Show ( department.department_name ); for employee in ( select ... where department_id = department.department_id ) loop Show ( employee.last_name );

This SQL does the job in one go

select department_name, cursor ( select last_name from employees e where e.department_id = d.department_id ) from departments d;

Use in it PL/SQL new in Oracle9i

declare cursor depts is select department_name, cursor ( select ... ) from departments d;

y Since theres only one SQL statement (we had two in the explicit approach) the CBO has a better chance

Set up the fetch targets

y Just for fun well do some bulk fetching
v_dname emps type enames_t v_enames dep...%type; sys_refcursor; is table of emp...%type index by...; enames_t;

Outer loop
y Fetch the cursor into the ref cursor target and then fetch it (bulk) into its target
open depts; loop fetch depts into v_dname, emps; exit when depts%notfound; Show ( v_dname ); fetch emps bulk collect into v_ename;

Inner loop
y Process the bulk fetched results
for j in v_ename.first..v_ename.last loop Show ( v_ename(j) ); end loop;

Table functions and cursor expressions

y Cursor variables recap y Manipulating cursor expressions in PL/SQL

y Using a cursor expression as an actual parameter to a PL/SQL function

y y y y Table functions pre-Oracle9i Oracle9i pipelined table functions Daisy-chaining table functions Parallelization

Cursor terminology
y A cursor variable (i.e. a variable of type REF CURSOR) points to an actual cursor y It may be used as a formal parameter to a PL/SQL procedure or function y A cursor expression defines an actual cursor and is legal in a SQL statement y So youd expect to write
Func ( cursor ( select c from t ) )

Cursor expression as an actual parameter to a PL/SQL function

y Pre-Oracle9i this was never allowed y At Oracle9i it is allowed when a function is invoked in a top level SQL statement

Cursor expression in WHERE clause function

select ... from employees managers where Func ( cursor ( < select stuff for this manager's reports > ), managers.hire_date ) = 1;

Cursor expression in WHERE clause function

y Were deciding on the basis of elaborate procedural logic y Writing the whole thing as a procedure you need to output to a table y With the logic in a WHERE clause function you can define a VIEW for dynamic reuse

A cursor expression can be an actual parameter to a PL/SQL function in a top level SQL statement
Hold that thought

Table functions and cursor expressions

y Cursor variables recap y Manipulating cursor expressions in PL/SQL y Using a cursor expression as an actual parameter to a PL/SQL function

y Table functions pre-Oracle9i

y Oracle9i pipelined table functions y Daisy-chaining table functions y Parallelization

Pre-Oracle9i table function

y We define our row, and a table of our rows
create type my_row as object ( idx text

number, varchar2(20) );

create type my_tab as table of my_row;

Pre-Oracle9i table function

function Func return my_tab is v_tab my_tab; begin v_tab := my_tab ( my_row ( ... ) ); < extend the table and compute the rows > return v_tab; end Func;

y All the rows are created before the function returns

Pre-Oracle9i table function

select * from table ( cast ( Func() as my_tab ) );

y Nice, but the response characterstics are not what were used to with a rowsource

Table functions and cursor expressions

y Cursor variables recap y Manipulating cursor expressions in PL/SQL y Using a cursor expression as an actual parameter to a PL/SQL function y Table functions pre-Oracle9i

y Oracle9i pipelined table functions

y Daisy-chaining table functions y Parallelization

Oracle9i pipelined table function

function Func return my_tab pipelined is v_row my_row; begin for ... loop v_row := my_row ( ... ); pipe row ( v_row ); end loop; return; end Func;

As soon as a row is computed, its delivered rowsource-style

Hold that thought too

Note:
y The pipelined and pipe row keywords can only be used together as shown y A pipelined table function can be invoked only in the FROM list of a SELECT clause y We can now use a simpler syntax
select * from table ( Func() );

y We can now use package-level types for the row and for the table

Table functions and cursor expressions

y Cursor variables recap y Manipulating cursor expressions in PL/SQL y Using a cursor expression as an actual parameter to a PL/SQL function y Table functions pre-Oracle9i y Oracle9i pipelined table functions

y Daisy-chaining table functions

y Parallelization

Remember
y A cursor expression can be an actual parameter to a PL/SQL function in a top level SQL statement y So a table function may now be defined with an input parameter of type REF CURSOR and invoked with a cursor expression as the actual parameter

function Func_2 ( p_cur in sys_refcursor ) return Pkg.my_tab pipelined is v_in_row Pkg.my_in_row; v_out_row Pkg.my_row; begin loop fetch p_cur into v_in_row; exit when p_cur%notfound;
< compute out-row(s) from in-row(s) >

pipe row ( v_out_row ); end loop; close p_cur; return; end Func_2;

Were consuming in-rows, Transforming them procedurally to out-rows with an arbitrarily complex M:N algorithm, and piping out-rows as soon as theyre ready

How do we use it?

y Suppose t is a table which supports a select list compatible with Pkg.my_row y We can now invoke the table function thus
select * from table ( Func_2 ( cursor ( select * from t ) ) );

Think about it
y We now have a very powerful generic transformation technique y It starts with the results from a SELECT and transforms them into something you can SELECT from y Sounds familiar? y Note: dont forget, the table function can have ordinary input parameters

Think of a table function as a deluxe, parameterizable view

t might have been a view

create view t as select * from table ( Func() );

y So wed actually be doing

select * from table ( Func_2 ( cursor ( select * from table ( Func() ) ) ) );

Now were daisy-chaining

Daisy-chaining
y We can plug any number of transformations back-to-back y Each can perform an arbitrarily complex transformation y The starting point could be an Oracle table y Or it could be a table function that accesses external data via Utl_File or via the call-out framework

Table functions and cursor expressions

y Cursor variables recap y Manipulating cursor expressions in PL/SQL y Using a cursor expression as an actual parameter to a PL/SQL function y Table functions pre-Oracle9i y Oracle9i pipelined table functions y Daisy-chaining table functions

y Parallelization

Whos going to use them?

y The driver for developing table functions was datawarehousing, especially the ETL phase y So of course the story wouldnt be complete without a parallel capability y Lets look at the syntax

PARALLEL_ENABLE syntax
function Func_2 ( p_cur in sys_refcursor ) return Pkg.my_tab pipelined parallel_enable ( partition p_cur by any ) is...

y Must have an input REF CURSOR parameter to drive the partitioning y ANY is the simple, special case results dont depend on the order of processing

Caution
y If you use ANY and your assertion is wrong, the results will be unpredictable y The developer must design the algorithm so that the results dont depend on the degree of parallelism

What if the results do depend on the order of processing?

y We can be more explicit when we declare how we want the input rows to be partitioned y This requires that we have a strongly typed input cursor, e.g
package Pkg is type my_row is record ( n number, ... ); type cur_t is ref cursor return my_row; end Pkg;

CLUSTER BY
function Func_3 ( p_cur in Pkg.cur_t ) return Pkg.my_tab pipelined parallel_enable ( partition p_cur by hash (n) ) cluster p_cur by (n) is...

y The algorithm requires that all rows for a given value of n come together to a single slave y but doesnt care about the order

ORDER BY
function Func_4 ( p_cur in Pkg.cur_t ) return Pkg.my_tab pipelined parallel_enable ( partition p_cur by range (n) ) order p_cur by ( n, m ) is...

y Not only must all rows for a given value of n come together to a single slave y but also they must be ordered by n, m

Before

t1 oltp

stage1

t2

stage2

t3 data warehouse

Using Oracle9i table functions

t1 t1 t1 oltp t1 t1 t2 t2 t2 t2 t2 t3 t3 t3 t3 t3 data warehouse

pipelined 3

parallelized 3

Comparison
y y y y y y Sun Ultra-Enterprise 4500 3 GB RAM, 10 CPUs of 168 MHz 1,000,000 row source table 1 row in to 7 rows out pivot transform Oracle8i PL/SQL cursor loop with 7 INSERTs Oracle9i table function with 7 PIPE ROWs
Performance and Scalability in DSS Environment with Oracle9i, Neil Thombre, Oracle Corp OOW paper #822 also on otn.oracle.com/deploy/performance/

Oracle8i 87 min

Oracle9i pipelined 37 min 2.4 x

Oracle9i pipelined parallel 20 12 min

7.5 x

Table Functions - summary

y An exciting new language feature y Generic applicability think deluxe, parameterizable view y Especially beneficial in Extraction Load Transformation phase in warehousing applications

Oracle9i PL/SQL gives you

y Improved performance y Increased functionality y Better usability for the developer

you should upgrade !

Q U E S T I O N S A N S W E R S