Hands on Collections

Copyright 2006 Steven Feuerstein -

Hands-on PL/SQL:Focus on Collections

Steven [email protected]

www.oracleplsqlprogramming.comwww.qnxo.com www.unit-test.com


Ten Years Writing Ten Books on the Oracle PL/SQL Language


The Mechanics

Lectures followed by lots of flex-time exercises.– Tell me your status: green = ready, red=help

needed Use Toad (or your own alternative) to build,

edit, run exercises. Login: qnxo/qnxo Qnxo contains your exercise repository.

– Or work from c:\hoc\hoc_exercises.html or c:\hoc\hoc_exercises_no_sol.html Generally, all files found in c:\hoc unless indicated otherwise.

Use Oracle doc: http://tahiti.oracle.com


Outline of class and exercises

01. Declare and define types, instances, and elements 02. Intro to BULK COLLECT and FORALL 03. Scan the contents of collections 04. Pass collections as parameters 05. Modify contents of collections with collection methods 06. String-indexed collections 07. Multi-level collections 08. Working with collections in SQL statements 09. BULK COLLECT for high performance querying 10. FORALL for high performance DML 11. Table Functions 12. Multiset operations on nested tables A. Collection utilities, plus mutating table errors


Software used in this seminar (or recommended)

You can download all my training materials and demonstration scripts from:– http://oracleplsqlprogramming.com/resources.html

Toad: thanks to Quest for giving me permission! Qnxo: repository of exercises and useful PL/SQL

code MasterMind and Set: let's have fun while

keeping our brains tuned up. Qute: the Quick Unit Test Engine

plsql_ides.txt


Introduction to Toad

You all have installed and available Toad 8.6 to help you write and run your exercises.– Username: qnxo– Password: qnxo

Let's do a quick review of the main features of Toad you will be using...


Introduction to Qnxo

I used Qnxo to define a repository of all the exercises and solutions for the class.– Find the "Hands-on Training on PL/SQL

Collections" root script in the Script Browser.

Drill down to the current chapter, and then into the exercises for that chapter.– The corresponding "s" script (CH-Ns) offers a

solution that you can use to help you solve the exercise.


00. Resources for exercises

Here are a few scripts to help you complete your exercises over the next two days....


Calculating elapsed time

You will usually have more than one possible implementation of your challenges. Which is fastest?– DBMS_UTILITY.GET_CPU_TIME helps you

answer that question down to the 100th of a second.

Two scripts for your use:– test_varieties procedure to compare multiple

implementations mult_imp.sql– stand-alone script to perform timings elapsed.sql


Recreating database objects and contents

Your schema has defined within it the full set of standard HR objects, such as employees, departments, etc.

We have also added a department_denorms table with collection columns.

If at any point you want to refresh these tables...– Copy the code from the "Script to create tables

referenced in the exercises"– Paste into the SQL Editor and run it.


A. Collection utilities and applications

The A(ppendix) challenges you to construct commonly-needed functionality when working with collections.– Since PL/SQL is a strongly-typed language, it is very difficult

to write programs that work for all types of collections. – If you find yourself waiting for others to finish, you can work

on these!


PL/SQL Collections

Collections are single-dimensioned lists of information, similar to 3GL arrays.

They are an invaluable data structure; all PL/SQL developers should be familiar with them -- and use them a lot.

They take some getting used to, especially when you want to leverage the latest features, such as multi-level collections.


What is a collection?

A collection is an "ordered group of elements, all of the same type." (PL/SQL User Guide and Reference)– That's a very general definition; lists, sets, arrays and similar

data structures are all types of collections.– Each element of a collection may be addressed by a unique

subscript, usually an integer but in some cases also a string.– Collections are single-dimensional, but you can create

collections of collections to emulate multi-dimensional structures.

abc def sf q rrr swq...1 2 3 4 22 23


Why use collections?

Emulate bi-directional cursors, which are not yet supported within PL/SQL.

Bypass mutating table restrictions in DB triggers. Avoid many scenarios that produce "Snapshot too

old" and "Rollback segment too small" errors. Using BULK COLLECT and FORALL....

Parallelize execution of PL/SQL functions inside SQL statements. With table functions....

Dramatically improve multi-row querying, inserting, updating and deleting the contents of tables. Combined with BULK COLLECT and FORALL....

Cache data in program memory for faster access. The difference between PGA and SGA....


System Global Area (SGA) of RDBMS Instance

Refresher: PL/SQL in Shared Memory

Shared Pool

Large Pool

Reserved Pool

show_empscalc_totals upd_salaries

Select * from emp

Shared SQL

Pre-parsedUpdate emp Set sal=...

Library cache

Session 1 memory (PGA/UGA)

emp_rec emp%rowtype;tot_tab tottabtype;

Session 2 memory (PGA/UGA)

emp_rec emp%rowtype;tot_tab tottabtype;Session 1

Session 2


Three Types of Collections

Associative arrays (aka index-by tables) – Can be used only in PL/SQL blocks.– Similar to hash tables in other languages, allows you to

access elements via arbitrary subscript values.

Nested tables and Varrays – Can be used in PL/SQL blocks, but also can be the

datatype of a column in a relational table. – Part of the object model in PL/SQL.– Required for some features, such as table functions– With Varrays, you specify a maximum number of elements

in the collection, at time of definition.


About Associative Arrays

Unbounded, practically speaking. – Valid row numbers range from -2,147,483,647 to

2,147,483,647.

– This range allows you to employ the row number as an intelligent key, such as the primary key or unique index value, because AAs also are:

Sparse– Data does not have to be stored in consecutive rows, as is

required in traditional 3GL arrays and VARRAYs.

Index values can be integers or strings (Oracle9i R2 and above).

assoc_array_example.sql


About Nested Tables

No pre-defined limit on a nested table.– Valid row numbers range from 1 to

2,147,483,647.

Part of object model, requiring initialization.

Is always dense initially, but can become sparse after deletes.

Can be defined as a schema level type and used as a relational table column type.

nested_table_example.sql


About Varrays

Has a maximum size, associated with its type. – Can adjust the size at runtime in Oracle10g R2.

Part of object model, requiring initialization.

Is always dense; you can only remove elements from the end of a varray with TRIM.

Can be defined as a schema level type and used as a relational table column type.

varray_example.sql


First Set of Collection Methods

COUNT returns number of rows currently defined in collection.

EXISTS returns TRUE if the specified row is defined. FIRST/LAST return lowest/highest numbers of defined

rows. NEXT/PRIOR return the closest defined row

after/before the specified row. LIMIT tells you the max. number of elements allowed

in a VARRAY.


Useful reminders for PL/SQL collections

Memory for collections comes out of the PGA or Process Global Area– One per session, so a program using collections

can consume a large amount of memory.

Don't always fill collections sequentially. Think about how you need to manipulate the contents.

Oracle raises NO_DATA_FOUND if you try to read an index that does not exist.

mysess.pkgsess2.sql


Exercises for "01. Declare and define types, instances, and elements"

Time to get comfortable with declaring types and variables based on those types!


02. Introduction to BULK COLLECT / FORALL; high performance SQL

Oracle8i and Oracle9i offer groundbreaking new syntax to improve the performance of both DML and queries.

In Oracle8, updating from a collection (or, in general, performing multi-row DML) meant writing code like this:

CREATE TYPE dlist_t AS TABLE OF INTEGER;/PROCEDURE remove_emps_by_dept (deptlist dlist_t)ISBEGIN FOR aDept IN deptlist.FIRST..deptlist.LAST LOOP DELETE emp WHERE deptno = deptlist(aDept); END LOOP;END;

“Conventional binds” (and lots of them!)


Oracle server

PL/SQL Runtime Engine SQL Engine

PL/SQL blockProcedural statement executor

SQL statement executor

FOR aDept IN deptlist.FIRST.. deptlist.LASTLOOP DELETE emp WHERE deptno = deptlist(aDept);END LOOP;

Performance penalty Performance penalty for many “context for many “context switches”switches”

Conventional Bind


Enter the “Bulk Bind”

Oracle server

PL/SQL Runtime Engine SQL Engine

PL/SQL blockProcedural statement executor

SQL statement executor

FORALL aDept IN deptlist.FIRST.. deptlist.LAST DELETE emp WHERE deptno = deptlist(aDept);

Much less overhead for Much less overhead for context switchingcontext switching


Use the FORALL Bulk Bind Statement

Instead of executing repetitive, individual DML statements, you can write your code like this:

Things to be aware of:– Only a single DML statement is allowed. If you want to

INSERT and then UPDATE, two different FORALL statements.

– SQL%BULK_ROWCOUNT returns the number of rows affected by each row in the binding array.

– Prior to Oracle10g, the binding array must be sequentially filled.

PROCEDURE remove_emps_by_dept (deptlist dlist_t)ISBEGIN FORALL aDept IN deptlist.FIRST..deptlist.LAST DELETE FROM emp WHERE deptno = deptlist(aDept);END;

bulk_rowcount.sql


CREATE OR REPLACE PROCEDURE process_emps (deptno_in IN dept.depno%TYPE)IS TYPE three_cols_rt IS RECORD ( empno emp.empno%TYPE, ename emp.ename%TYPE, hiredate emp.hiredate%TYPE); TYPE three_cols_tt IS TABLE OF three_cols_rt INDEX BY PLS_INTEGER; three_cols_t three_cols_tt;BEGIN SELECT empno, ename, hiredate BULK COLLECT INTO three_cols_t FROM emp WHERE deptno = deptno_in; ...END;

Use BULK COLLECT INTO for Queries

CREATE OR REPLACE PROCEDURE process_emps (deptno_in IN dept.depno%TYPE)IS TYPE numTab IS TABLE OF NUMBER; TYPE charTab IS TABLE OF VARCHAR2(12); TYPE dateTab IS TABLE OF DATE; enos numTab; names charTab; hdates dateTab;BEGIN SELECT empno, ename, hiredate BULK COLLECT INTO enos, names, hdates FROM emp WHERE deptno = deptno_in; FOR i IN enos.FIRST..enos.LAST LOOP do_stuff (enos(i), names(i), hiredates(i)); END LOOP;END;

Oracle9i R2 supports fetching into a collection of

records.

Oracle8i requires fetching into individual collections of

scalars.

bulkcoll.sql


Exercises for Chapter 2

Let's get you comfortable with the basics of BULK COLLECT and FORALL.– Later we will drill down into more nuances of these

great constructs.


03. Scan the contents of collections

You can use any kind of PL/SQL loop to iterate through the elements of a collection.

Here are some recommendations:– Use a numeric FOR loop only when the

collection is densely-filled and indexed by integer.

– Use a WHILE loop and NEXT/PRIOR with sparsely-filled collections.



When you are done with this section, choosing the right kind of loop should be automatic for you.


04. Pass collections as parameters

You can pass collections as parameters in functions and procedures, just as with any other type of data.

But keep these items in mind...– The collection type on which the parameter is

based must already be defined.– Passing collections as IN OUT arguments can

involve lots of overhead. That's why Oracle offers the NOCOPY hint.


Example of passing collections as arguments - 1

Define a schema-level collection type and then use that to pass an argument to a procedure.

CREATE OR REPLACE TYPE numbers_ntt IS TABLE OF NUMBER;/

CREATE OR REPLACE PROCEDURE process_numbers ( numbers_in IN numbers_ntt )ISBEGIN -- Assumes densely-filled collection! FOR indx IN 1 .. numbers_in.COUNT LOOP DBMS_OUTPUT.put_line ( numbers_in ( indx )); END LOOP;END process_numbers;/



Define a collection type in a package specification and then use that to pass an argument to a procedure.

CREATE OR REPLACE PACKAGE types_pkgIS TYPE numbers_ntt IS TABLE OF NUMBER;END types_pkg;/CREATE OR REPLACE PROCEDURE process_numbers ( numbers_in IN types_pkg.numbers_ntt) ISBEGIN -- Assumes densely-filled collection! FOR indx IN 1 .. numbers_in.COUNT LOOP DBMS_OUTPUT.put_line ( numbers_in ( indx )); END LOOP;END process_numbers;/



Define a collection type in a local block and then use that to pass an argument to a procedure.

DECLARE TYPE numbers_ntt IS TABLE OF NUMBER; l_numbers numbers_ntt;

PROCEDURE process_numbers ( numbers_in IN types_pkg.numbers_ntt ) IS BEGIN -- Assumes densely-filled collection! FOR indx IN 1 .. numbers_in.COUNT LOOP DBMS_OUTPUT.put_line ( numbers_in ( indx )); END LOOP; END process_numbers;BEGIN process_numbers ( l_numbers );END;


NOCOPY and parameter passing by reference

PL/SQL by default passes all parameters by value.– OUT and IN OUT are copied when the subprogram

is executed.– Temporary variables hold output parameter values.– No error, temporaries are copied to actuals,

otherwise no change is made. Can lead to excessive overhead and memory

for large structures and particularly collections and objects.– Enter the NOCOPY compiler hint.


NOCOPY: Just a request - and also potentially quite problematic!

Under some circumstances, the compiler will ignore your request to not copy, including:– Actual parameter is element of assoc. array– Actual parameter is constrained (eg, NOT NULL)

And if it does as requested and an exception is raised, your data may be corrupted.– The exception will not cause a rollback of

changes to a parameter that was passed by reference.

nocopy*.*



Let's make sure you are comfortable with all the variations, and familiar with NOCOPY.


05. Modify contents of collections with collection methods

DELETE deletes elements from the index-by table.– Delete 1, a range of, or all rows.

EXTEND adds elements to a nested table or VARRAY only.– Can extend 1 or multiple rows.– Can set initial value to value in existing row.

TRIM removes elements from the end a nested table or VARRAY only.

delete.sqlextend.sql

trim.sql



Go ahead, see if you can mess up those collections by deleting, trimming, extending!


06. String-indexed collections

Prior to Oracle9iR2, you could only index by BINARY_INTEGER.

You can now define the index on your associative array to be:– Any sub-type derived from BINARY_INTEGER– VARCHAR2(n), where n is between 1 and 32767– %TYPE against a database column that is consistent with

the above rules– A SUBTYPE against any of the above.

This means that you can now index on string values! (and concatenated indexes and...)

Oracle9i Release 2


Examples of new TYPE variants

All of the following are now valid TYPE declarations in Oracle9i Release 2– You cannot use %TYPE against an INTEGER column,

because INTEGER is not a subtype of BINARY_INTEGER.

DECLARE TYPE array_t1 IS TABLE OF NUMBER INDEX BY BINARY_INTEGER; TYPE array_t2 IS TABLE OF NUMBER INDEX BY PLS_INTEGER; TYPE array_t3 IS TABLE OF NUMBER INDEX BY POSITIVE; TYPE array_t4 IS TABLE OF NUMBER INDEX BY NATURAL; TYPE array_t5 IS TABLE OF NUMBER INDEX BY VARCHAR2(64); TYPE array_t6 IS TABLE OF NUMBER INDEX BY VARCHAR2(32767); TYPE array_t7 IS TABLE OF NUMBER INDEX BY employee.last_name%TYPE; TYPE array_t8 IS TABLE OF NUMBER INDEX BY types_pkg.subtype_t;


Working with VARCHAR2-Indexed Collections

Specifying a row via a string takes some getting used to, but if offers some very powerful advantages.

DECLARE TYPE population_type IS TABLE OF NUMBER INDEX BY VARCHAR2(64);

country_population population_type; continent_population population_type;

howmany NUMBER;BEGIN country_population('Greenland') := 100000; country_population('Iceland') := 750000;

howmany := country_population('Greenland');

continent_population('Australia') := 30000000;END;

assoc_array*.sqlassoc_array_perf.tst

Oracle9i Release 2


Rapid Access to Data Via String Keys

One of the most powerful applications of this features is to construct very fast pathways to static data from within PL/SQL programs. – If you are repeatedly querying the same data from the

database, why not cache it in your PGA inside collections?

Emulate the various indexing mechanisms (primary key, unique indexes) with collections.

Demonstration package:assoc_array5.sql

Oracle9i Release 2

Comparison of performance of different approaches:

vocab*.*

Generate a caching package:genaa.sqlgenaa.tst



Fun with strings! On the one hand, there is no substantive difference

in the way you work with string-indexed collections. Other the other hand, you can do some really cool

things very easily....


07. Multi-level collections

Oracle9i allows you to create collections of collections, or collections of records that contain collections, or...

Applies to all three types of collections. Two scenarios to be aware of:

– Named sub-collections– Anonymous sub-collections

Oracle9i


Collections with Named, Multi-level Collections

When a collection is based on a record or object that in turn contains a collection, that collection has a name.

CREATE TYPE vet_visit_t IS OBJECT ( visit_date DATE, reason VARCHAR2 (100));/CREATE TYPE vet_visits_t IS TABLE OF vet_visit_t/CREATE TYPE pet_t IS OBJECT ( tag_no INTEGER, NAME VARCHAR2 (60), petcare vet_visits_t, MEMBER FUNCTION set_tag_no (new_tag_no IN INTEGER) RETURN pet_t);/

Continued...multilevel_collections.sql

Collection nested inside object type

Oracle9i


Collections with Named, Multi-level Collections, continued

DECLARE TYPE bunch_of_pets_t IS TABLE OF pet_t INDEX BY BINARY_INTEGER; my_pets bunch_of_pets_t;BEGIN my_pets (1) := pet_t ( 100, 'Mercury', vet_visits_t ( vet_visit_t ( '01-Jan-2001', 'Clip wings'), vet_visit_t ( '01-Apr-2002', 'Check cholesterol') ) ); DBMS_OUTPUT.put_line (my_pets (1).petcare (2).reason);END;

Outer collection

Inner collection

Oracle9i


Anonymous Collection Columns

If your nested collections do not rely on "intermediate" records or objects, you simply string together index subscripts.– To demonstrate this syntax, let's take a look at how to

emulate a three-dimensional array using nested collections. First, we cannot directly reference or populate an

individual cell, as one would do in a 3GL.– Instead we have to build an interface between the underlying

arrays and the user of the "three dimensional array."

Oracle9i

BEGIN gps_info (1, 45, 605) := l_value;

BEGIN gps_info (605) (45) (1) := l_value;

Can't do this... Have to do something like this instead...


Multi-dimensional array emulationCREATE OR REPLACE PACKAGE multdim IS TYPE dim1_t IS TABLE OF VARCHAR2 (32767) INDEX BY BINARY_INTEGER; TYPE dim2_t IS TABLE OF dim1_t INDEX BY BINARY_INTEGER; TYPE dim3_t IS TABLE OF dim2_t INDEX BY BINARY_INTEGER; PROCEDURE setcell ( array_in IN OUT dim3_t, dim1_in PLS_INTEGER, dim2_in PLS_INTEGER, dim3_in PLS_INTEGER, value_in IN VARCHAR2 ); FUNCTION getcell ( array_in IN dim3_t, dim1_in PLS_INTEGER, dim2_in PLS_INTEGER, dim3_in PLS_INTEGER ) RETURN VARCHAR2;END multdim; multdim.*

multdim2.*gen_multcoll.sp

Oracle9i

Three levels of collections

Set a cell value

Get a cell value


Multi-dimensional array emulation

CREATE OR REPLACE PACKAGE BODY multdimIS PROCEDURE setcell ( array_in IN OUT dim3_t, dim1_in PLS_INTEGER, dim2_in PLS_INTEGER, dim3_in PLS_INTEGER, value_in IN VARCHAR2 ) IS BEGIN array_in(dim3_in )(dim2_in )(dim1_in) := value_in; END; FUNCTION getcell ( array_in IN dim3_t, dim1_in PLS_INTEGER, dim2_in PLS_INTEGER, dim3_in PLS_INTEGER) RETURN VARCHAR2 IS BEGIN RETURN array_in(dim3_in )(dim2_in )(dim1_in); END;

Oracle9i

As close as you can get...


Applying multi-level and string-based indexes

Careful -- and creative! -- application of this functionality can greatly simplify the code you need to write to handle complex requirements.

Let's step through an application of this capability to a programming challenge.

overloadings

naming_conventions

bad_datatypes

CodecheckpackageOverloadCheck:

A QA packagefor PL/SQL


The problem of ambiguous package overloadings

Oddly and sadly, it is possible to compile overloadings which are not usable.– You see an obvious example below, but there are many

more subtle circumstances, usually involving defaulted parameters.

So I will build a program to identify such ambiguous overloadings. But how can I do this?

BEGIN salespkg.calc_total ('ABC');END;/

PACKAGE salespkgIS PROCEDURE calc_total ( dept_in IN VARCHAR2); PROCEDURE calc_total ( dept_in IN CHAR);END salespkg;

?ambig_overloading.sql


ALL_ARGUMENTS to the rescue!

Parsing is too complicated for me, but the ALL_ARGUMENTS data dictionary view contains information about all the arguments of all the procedures and functions to which I have access. That sounds pretty good!

As usual, Oracle offers us a whole lot of pleasure, mixed with a little bit of pain.– The organization of data in ALL_ARGUMENTS is a bit

bizarre, plus it is incomplete, necessitating the use also of DBMS_DESCRIBE.DESCRIBE_COLUMNS.

all_arguments.tstall_arguments.sql

allargs.*


First Inclination: Same Old, Same Old

All right then, I will grab all the information from ALL_ARGUMENTS and dump it into a collection based on that view! Very easy...

CREATE OR REPLACE PROCEDURE get_all_arguments ( package_in IN VARCHAR2)IS TYPE all_arguments_tt IS TABLE OF all_arguments%ROWTYPE INDEX BY BINARY_INTEGER; l_arguments all_arguments_tt;BEGIN FOR rec IN ( SELECT * FROM all_arguments WHERE owner = USER AND package_name = package_in) LOOP l_arguments (SQL%ROWCOUNT) := rec; END LOOP;END;

Load it up!

Emulate the view.


Then what? Write lots of code to interpret the contents...

Which programs are overloaded? Where does one overloading end and another start?

l_last_program all_arguments.object_name%TYPE; l_is_new_program BOOLEAN := FALSE; l_last_overload PLS_INTEGER := -1;BEGIN FOR indx IN l_arguments.FIRST .. l_arguments.LAST LOOP IF l_arguments (indx).object_name != l_last_program OR l_last_program IS NULL THEN l_last_program := l_arguments (indx).object_name; l_is_new_program := TRUE; do_new_program_stuff; END IF; ...

IF l_arguments (indx).overload != l_last_overload OR l_last_overload = -1 THEN IF l_is_new_program THEN do_first_overloading_stuff; ELSE do_new_overloading_stuff; END IF; END IF; END LOOP;END;


Discovery: there is a natural hierarchy to ALL_ARGUMENTS data!

Each program has zero or more overloadings, each overloading has N arguments, and each argument can have multiple "breakouts" (my term - applies to non-scalar parameters, such as records or object types).

RUN_TEST

SHOW_RESULTS

RESET_FLAGS

Program name

Overloading 1

Overloading 2

Overloading

Argument 1

Argument 2

Argument 3

Argument 4

Argument 5

ArgumentBreakout 1

Breakout 1

Breakout 2

Breakout 3

Breakout


What if I reflect this hierarchy in a collection of collections?

Have to build from the bottom up:

TYPE breakouts_t IS TABLE OF all_arguments%ROWTYPE INDEX BY BINARY_INTEGER;

TYPE arguments_t IS TABLE OF breakouts_t INDEX BY BINARY_INTEGER;

TYPE overloadings_t IS TABLE OF arguments_t INDEX BY BINARY_INTEGER;

TYPE programs_t IS TABLE OF overloadings_t INDEX BY all_arguments.object_name%type;

1. Set of rows from ALL_ARGUMENTS

String-based index

2. All the "breakout" info for a single argument

3. All the argument info for a single overloading

4. All the overloadings for a distinct program name


Then I can populate it very easily

Assigning a single record to the "lowest" level also defines each of the upper levels.

Notice the automatic "SELECT DISTINCT" on program name that results!

FOR rec IN (SELECT * FROM all_arguments)LOOP l_arguments (NVL (l_arguments.LAST, 0) + 1) := rec; l_programs (rec.object_name) (NVL (rec.overload, 0)) (rec.position) (rec.data_level) := rec;END LOOP;

I can still do the typical sequential

load.

But I will now also add the multi-level

load in single assignment

show_all_arguments.spshow_all_arguments.tst


And then I can "query" the contents with a minimum of code

l_programs ('TOP_SALES') (2).EXISTS (0)

Is the TOP_SALES program overloaded?

l_programs ('TOP_SALES') (2)(0)(0).datatype

l_programs ('TOP_SALES').COUNT > 1

Is the 2nd overloading of TOP_SALES a

function?

What is the datatype of the RETURN clause of the 2nd overloading

of TOP_SALES?

And, of course, I know the beginning and end points of each program, overloading, and argument. I just use the

FIRST and LAST methods on those collections!


Encapsulate these complex structures!

As you can see, you can easily and rapidly arrive at completely unreadable and un-maintainable code.

What' s a developer to do?– Hide complexity -- and all data structures -- behind

small modules.– Work with and through functions to retrieve

contents and procedures to set contents.

cc_smartargs.pkb:cc_smartargs.next_overloading

cc_smartargs.add_new_parameter



These structures can get very tough to follow, very quickly.

Take it step by step and encapsulate the structures to hide the details and avoid confusion.

Oracle10g


08. Working with collections in SQL statements

When you define a collection as the column of a table, you can manipulate its contents with SQL statements.

You can also query the contents of a PL/SQL variable collection (nested table or varray).– Which means that you sort the contents.

These operators will come in handy:– TABLE : convert a collection into a query result set – MULTISET: convert the result set of a subquery into a

collection


Using Collections inside SQL

Lisa MarieGary Richard

BOND

BOLZ

Barbara Annedb_family

surname children

Max RichardEric Thomas

Lisa MarieGary RichardBarbara Annecolumn_value

SELECT column_valueFROM TABLE (SELECT children FROM db_family WHERE surname = 'BOLZ');

UPDATE TABLE (SELECT children FROM db_family WHERE SURNAME = 'BOLZ)SET column_value = 'Lisa Nadezhka'WHERE column_value = 'Lisa Marie');

Lisa NadezhkaGary RichardBarbara Anne

. . .

BOLZ

db_family

surname children


Inserting into a table from a collection

INSERT-SELECT from a local collection - no problem with the TABLE operator!

DECLARE lvnt nt_vc2_20x := nt_vc2_20x ( );BEGIN lvnt.EXTEND ( 3 ); lvnt ( 1 ) := 'ONE'; lvnt ( 2 ) := 'TEN'; lvnt ( 3 ) := 'HUNDRED';

INSERT INTO t_vc2 SELECT relational_lvnt.COLUMN_VALUE FROM TABLE (lvnt AS nt_vc2_20x) relational_lvnt;END;

collsql1.sqlcollsql4.sql


Querying from a collection using TABLE

Use TABLE to (at least conceptually) convert a collection into a database table.

DECLARE nyc_devolution cutbacks_for_taxcuts := cutbacks_for_taxcuts ('Stop rat extermination programs', 'Fire building inspectors', 'Close public hospitals');BEGIN DBMS_OUTPUT.PUT_LINE ( 'How to Make the NYC Rich Much, Much Richer:'); FOR rec IN ( SELECT COLUMN_VALUE ohmy FROM TABLE (nyc_devolution)) LOOP DBMS_OUTPUT.PUT_LINE (rec.ohmy); END LOOP;END;

collsql2.sqlcollsql3.sql


Using the MULTISET Operator

MULTISET is the inverse of TABLE, converting a set of table, view, query) into a VARRAY or nested table.– Use MULTISET to emulate or transform relational joins into

collections, with potential client-server performance impact.

DECLARE CURSOR bird_curs IS SELECT b.genus, b.species, CAST(MULTISET(SELECT bh.country FROM bird_habitats bh WHERE bh.genus = b.genus AND bh.species = b.species) AS country_tab_t) FROM birds b; bird_row bird_curs%ROWTYPE;BEGIN OPEN bird_curs; FETCH bird_curs into bird_row;END;

multiset.sql

Retrieves all detail information for the master in one trip.



Make sure you are comfortable with applying various kinds of SQL statements to collections.

Advanced: 08-4...populate the denormalization table.– Let's take a look at that structure....


09. BULK COLLECT for high performance querying

Let's go back to BULK COLLECT and talk a bit more about using it in "real life."

BULK COLLECT is fast, but can also cause large consumption of memory.

Use the LIMIT clause to manage memory and still achieve high performance.


Limit the number of rows returned by BULK COLLECT

CREATE OR REPLACE PROCEDURE bulk_with_limit (deptno_in IN dept.deptno%TYPE)IS CURSOR emps_in_dept_cur IS SELECT * FROM emp WHERE deptno = deptno_in;

TYPE emp_tt IS TABLE OF emp%ROWTYPE; emps emp_tt;BEGIN OPEN three_cols_cur; LOOP FETCH emps_in_dept_cur BULK COLLECT INTO emps LIMIT 100;

EXIT WHEN emps.COUNT = 0;

process_emps (emps); END LOOP;END bulk_with_limit;

Use the LIMIT clause with the INTO to manage the amount

of memory used with the BULK COLLECT operation.

WARNING!

BULK COLLECT will not raise NO_DATA_FOUND if no rows

are found.

Best to check contents of collection to confirm that something was retrieved.

bulklimit.sql


Dynamic BULK COLLECT

Now you can even avoid the OPEN FOR and just grab your rows in a single pass!CREATE OR REPLACE PROCEDURE fetch_by_loc (loc_in IN VARCHAR2)IS TYPE numlist_t IS TABLE OF NUMBER; TYPE namelist_t IS TABLE OF employee.name%TYPE; TYPE employee_t IS TABLE OF employee%ROWTYPE;

emp_cv sys_refcursor;

empnos numlist_t; enames namelist_t; l_employees employee_t;BEGIN OPEN emp_cv FOR 'SELECT empno, ename FROM emp_' || loc_in; FETCH emp_cv BULK COLLECT INTO empnos, enames; CLOSE emp_cv;

EXECUTE IMMEDIATE 'SELECT * FROM emp_' || loc_in BULK COLLECT INTO l_employees;END;

With Oracle9iR2 you can also fetch into collections of

records.

Oracle9i


Tips and Fine Points

Use bulk queries whenever you need to execute individual row queries within a PL/SQL loop.– Can be used with implicit and explicit cursors

Fills collection sequentially, starting at row 1 Avoid "Snapshot too old" errors...

– Caused: a cursor is held open too long and Oracle can no longer maintain the snapshot information.

– Solution: open-close cursor, or use BULK COLLECT to retrieve information more rapidly bulktiming.sql

emplu.pkgcfl_to_bulk*.sql


Cursor FOR Loop ... or BULK COLLECT?

Why would you ever use a cursor FOR loop (or other LOOP) now that you can perform a BULK COLLECT?– If you want to do complex processing on each

row as it is queried – and possibly halt further fetching.

– You are retrieving many rows and cannot afford to use up the memory (large numbers of users).

Otherwise, moving to BULK COLLECT is a smart move!

cfl_vs_bulkcollect.sqlcfl_to_bulk*.sql


10. FORALL for high performance DML

Let's now return to FORALL and fill out our understanding of this powerful construct:– Use with dynamic SQL– SAVE EXCEPTIONS for improved error

handling– Using FORALL with sparsely-filled collections


Dynamic FORALL Example

This example shows the use of bulk binding and collecting, plus application of the RETURNING clause.

CREATE TYPE NumList IS TABLE OF NUMBER;CREATE TYPE NameList IS TABLE OF VARCHAR2(15);

PROCEDURE update_emps ( col_in IN VARCHAR2, empnos_in IN numList) IS enames NameList;BEGIN FORALL indx IN empnos_in.FIRST .. empnos_in.LAST EXECUTE IMMEDIATE 'UPDATE emp SET ' || col_in || ' = ' || col_in || ' * 1.1 WHERE empno = :1 RETURNING ename INTO :2' USING empnos_in (indx ) RETURNING BULK COLLECT INTO enames; ...END;

Notice that empnos_in is indexed, but enames is not.

Oracle9i


Better Exception Handlingfor Bulk Operations

Allows you to continue past errors and obtain error information for each individual operation (for dynamic and static SQL).

CREATE OR REPLACE PROCEDURE load_books (books_in IN book_obj_list_t)IS bulk_errors EXCEPTION; PRAGMA EXCEPTION_INIT ( bulk_errors, -24381 );BEGIN FORALL indx IN books_in.FIRST..books_in.LAST SAVE EXCEPTIONS INSERT INTO book values (books_in(indx));EXCEPTION WHEN BULK_ERRORS THEN FOR indx in 1..SQL%BULK_EXCEPTIONS.COUNT LOOP log_error (SQL%BULK_EXCEPTIONS(indx)); END LOOP;END;

Allows processing of all rows, even after an error

occurs.

New cursor attribute, a pseudo-

collection

bulkexc.sql

Oracle9i


More flexibility with FORALL

In Oracle10g, the FORALL driving array no longer needs to be processed sequentially.

Use the INDICES OF clause to use only the row numbers defined in another array.

Use the VALUES OF clause to use only the values defined in another array.

Oracle10g


Using INDICES OF

It only processes the rows with row numbers matching the defined rows of the driving array.

Oracle10g

10g_indices_of.sql10g_indices_of2.sql

DECLARE TYPE employee_aat IS TABLE OF employee.employee_id%TYPE INDEX BY PLS_INTEGER; l_employees employee_aat; TYPE boolean_aat IS TABLE OF BOOLEAN INDEX BY PLS_INTEGER; l_employee_indices boolean_aat;BEGIN l_employees (1) := 7839; l_employees (100) := 7654; l_employees (500) := 7950; -- l_employee_indices (1) := TRUE; l_employee_indices (500) := TRUE; l_employee_indices (799) := TRUE -- FORALL l_index IN INDICES OF l_employee_indices BETWEEN 1 AND 500 UPDATE employee SET salary = 10000 WHERE employee_id = l_employees (l_index);END;


Using VALUES OF

It only processes the rows with row numbers matching the content of a row in the driving array.

Oracle10g

DECLARE TYPE employee_aat IS TABLE OF employee.employee_id%TYPE INDEX BY PLS_INTEGER; l_employees employee_aat;

TYPE values_aat IS TABLE OF PLS_INTEGER INDEX BY PLS_INTEGER; l_employee_values values_aat;BEGIN l_employees (-77) := 7820; l_employees (13067) := 7799; l_employees (99999999) := 7369; -- l_employee_values (100) := -77; l_employee_values (200) := 99999999; -- FORALL l_index IN VALUES OF l_employee_values UPDATE employee SET salary = 10000 WHERE employee_id = l_employees (l_index);END;

10g_values_of.sql


Rollback segment too small and incremental committing with FORALL

Rollback segment too small...– Cause: so many uncommitted changes, the

rollback segment can't handle it all.– Solution: incremental commits. You can do this

with normal DML but also with FORALL DML.

forall_incr_commit.sql


Exercises for Chapters 9 and 10

Deepen your knowledge of both bulk SQL options.


11. Table Functions

A table function is a function that you can call in the FROM clause of a query, and have it be treated as if it were a relational table.

Table functions allow you to perform arbitrarily complex transformations of data and then make that data available through a query.– Not everything can be done in SQL.

Combined with REF CURSORs, you can now more easily transfer data from within PL/SQL to host environments.– Java, for example, works very smoothly with cursor

variables


Building a table function

A table function must return a nested table or varray based on a schema-defined type, or type defined in a PL/SQL package.

The function header and the way it is called must be SQL-compatible: all parameters use SQL types; no named notation.– In some cases (streaming and pipelines

functions), the IN parameter must be a cursor variable -- a query result set.


Simple table function example

Return a list of names as a nested table, and then call that function in the FROM clause.

CREATE OR REPLACE FUNCTION lotsa_names ( base_name_in IN VARCHAR2, count_in IN INTEGER) RETURN names_ntIS retval names_nt := names_nt ();BEGIN retval.EXTEND (count_in);

FOR indx IN 1 .. count_in LOOP retval (indx) := base_name_in || ' ' || indx; END LOOP;

RETURN retval;END lotsa_names; tabfunc_scalar.sql

SELECT column_value FROM TABLE ( lotsa_names ('Steven' , 100)) names;

COLUMN_VALUE ------------Steven 1 ... Steven 100


Streaming data with table functions

You can use table functions to "stream" data through several stages within a single SQL statement.

CREATE TYPE tickertype AS OBJECT ( ticker VARCHAR2 (20) , pricedate DATE , pricetype VARCHAR2 (1) , price NUMBER);

CREATE TYPE tickertypeset AS TABLE OF tickertype;/

CREATE TABLE tickertable ( ticker VARCHAR2(20), pricedate DATE, pricetype VARCHAR2(1), price NUMBER)/

tabfunc_streaming.sql


Streaming data with table functions - 2

In this example, transform each row of the stocktable into two rows in the tickertable.

CREATE OR REPLACE PACKAGE refcur_pkgIS TYPE refcur_t IS REF CURSOR RETURN stocktable%ROWTYPE;END refcur_pkg;/

CREATE OR REPLACE FUNCTION stockpivot (dataset refcur_pkg.refcur_t) RETURN tickertypeset ...

BEGIN INSERT INTO tickertable SELECT * FROM TABLE (stockpivot (CURSOR (SELECT * FROM stocktable)));END;/

tabfunc_streaming.sql


Use pipelined functions to enhance performance.

Pipelined functions allow you to return data iteratively, asynchronous to termination of the function.– As data is produced within the function, it is passed

back to the calling process/query.

Pipelined functions can be defined to support parallel execution.– Iterative data processing allows multiple processes to

work on that data simultaneously.

CREATE FUNCTION StockPivot (p refcur_pkg.refcur_t) RETURN TickerTypeSet PIPELINED


Piping rows out from a pipelined function

CREATE FUNCTION stockpivot (p refcur_pkg.refcur_t) RETURN tickertypeset PIPELINED IS out_rec tickertype := tickertype (NULL, NULL, NULL); in_rec p%ROWTYPE;BEGIN LOOP FETCH p INTO in_rec; EXIT WHEN p%NOTFOUND; out_rec.ticker := in_rec.ticker; out_rec.pricetype := 'O'; out_rec.price := in_rec.openprice;

PIPE ROW (out_rec); END LOOP; CLOSE p;

RETURN;END;

tabfunc_setup.sqltabfunc_pipelined.sql

Add PIPELINED keyword to header

Pipe a row of data back to calling block

or query

RETURN...nothing at all!


Applications for pipelined functions

Execution functions in parallel.– In Oracle9i Database Release 2 and above, you can

use the PARALLEL_ENABLE clause to allow your pipelined function to participate fully in a parallelized query.

– Critical in data warehouse applications.

Improve speed of delivery of data to web pages.– Use a pipelined function to "serve up" data to the

webpage and allow users to being viewing and browsing, even before the function has finished retrieving all of the data.


Enabling Parallel Execution

The table function's parameter list must consist only of a single strongly-typed REF CURSOR.

Include the PARALLEL_ENABLE hint in the program header.– Choose a partition option that specifies how the function's

execution should be partitioned. – "ANY" means that the results are independent of the order

in which the function receives the input rows (through the REF CURSOR).

{[ORDER | CLUSTER] BY column_list} PARALLEL_ENABLE ({PARTITION p BY [ANY | (HASH | RANGE) column_list]} )


Table functions - Summary

Table functions offer significant new flexibility for PL/SQL developers.

Consider using them when you...– Need to pass back complex result sets of data

through the SQL layer (a query);– Want to call a user defined function inside a

query and execute it as part of a parallel query.



Go from simple table functions to more complex formulations.


13. Multiset operations on nested tables

Oracle10g introduces high-level set operations on nested tables (only).– Nested tables are multisets, meaning that

theoretically there is no order to their elements. – This makes set operations of critical importance for

manipulating nested tables.

You can now....– Check for equality and inequality– Obtain UNION, INTERSECT and MINUS of two NTs– Determine if there are duplicates, remove them, etc.


Check for equality and inequality

Just use the basic operators….DECLARE TYPE clientele IS TABLE OF VARCHAR2 (64); group1 clientele := clientele ('Customer 1', 'Customer 2'); group2 clientele := clientele ('Customer 1', 'Customer 3'); group3 clientele := clientele ('Customer 3', 'Customer 1');BEGIN IF group1 = group2 THEN DBMS_OUTPUT.put_line ('Group 1 = Group 2'); ELSE DBMS_OUTPUT.put_line ('Group 1 != Group 2'); END IF;

IF group2 != group3 THEN DBMS_OUTPUT.put_line ('Group 2 != Group 3'); ELSE DBMS_OUTPUT.put_line ('Group 2 = Group 3'); END IF;END;

10g_compare.sql10g_compare2.sql

10g_compare_old.sql

Watch out for NULLs! If the collections have the same number of elements, then if at least one element is NULL, the operators

return NULL.


UNION, INTERSECT, MINUS

Straightforward, with the MULTISET keyword.

Oracle10g

BEGIN our_favorites := my_favorites MULTISET UNION dad_favorites; show_favorites ('MINE then DAD', our_favorites); our_favorites := dad_favorites MULTISET UNION my_favorites; show_favorites ('DAD then MINE', our_favorites); our_favorites := my_favorites MULTISET UNION DISTINCT dad_favorites; show_favorites ('MINE then DAD with DISTINCT', our_favorites); our_favorites := my_favorites MULTISET INTERSECT dad_favorites; show_favorites ('IN COMMON', our_favorites); our_favorites := dad_favorites MULTISET EXCEPT my_favorites; show_favorites ('ONLY DAD''S', our_favorites); END;

10g_setops.sql10g_string_nt.sql10g_favorites.sql

10g*union*.sql


Distinct sets of values

Use the SET operator to work with distinct values, and determine if you have a set of distinct values.

Oracle10g

DECLARE keep_it_simple strings_nt := strings_nt ();BEGIN keep_it_simple := SET (favorites_pkg.my_favorites);

favorites_pkg.show_favorites ('FULL SET', favorites_pkg.my_favorites);

p.l (favorites_pkg.my_favorites IS A SET, 'My favorites distinct?'); p.l (favorites_pkg.my_favorites IS NOT A SET, 'My favorites NOT distinct?'); favorites_pkg.show_favorites ( 'DISTINCT SET', keep_it_simple); p.l (keep_it_simple IS A SET, 'Keep_it_simple distinct?'); p.l (keep_it_simple IS NOT A SET, 'Keep_it_simple NOT distinct?');

END;

10g_set.sql10g_favorites.pkg


Determining subsets of data

Use the SUBMULTISET operator to determine if a nested table contains only elements that are in another nested table.

Oracle10g

BEGIN p.l (favorites_pkg.my_favorites SUBMULTISET OF favorites_pkg.eli_favorites , 'Father follows son?');

p.l (favorites_pkg.eli_favorites SUBMULTISET OF favorites_pkg.my_favorites , 'Son follows father?');

p.l (favorites_pkg.my_favorites NOT SUBMULTISET OF favorites_pkg.eli_favorites , 'Father doesn''t follow son?');

p.l (favorites_pkg.eli_favorites NOT SUBMULTISET OF favorites_pkg.my_favorites , 'Son doesn''t follow father?');END; 10g_submultiset.sql

10g_favorites.pkg


Mutating table errors and collections (Appendix exercises)

A mutating table error occurs when a row-level trigger attempts to query or change the table from which it fired.

Collections offer a very nice way to work around these errors.

Use exercise A-5 to apply the information provided in the next two pages.


Dealing with Mutating Table errors

Row level triggers cannot query from or change the contents of the table to which it is attached; it is "mutating".

But statement level triggers do not have this restriction.

So what are you supposed to do when a row-level operation needs to "touch" that table?

UPDATE row 1

UPDATE row N

UPDATE emp SET sal = 1000

Database triggers can be associated with both the DML statement as a whole and individual rows affected by that statement.

Note: in Oracle8i, you can use autonomous transactions to relax

restrictions associated with queries.

mutating.sql

Statement level

Row level


A Solution Based on Associative Arrays Tables

Since you cannot perform the processing desired in the row-level trigger, you need to defer the action until you get to the statement level.

If you are going to defer the work, you have to remember what you needed to do. – A collection is an ideal repository for this reminder list.

1st row trigger fires

Nth row trigger fires

Work List(collection)

Statement Trigger

Writes to list

Writes to list

Process datain the list.

mutating_trigger.pkgranking.pkg


Acknowledgements and Resources

Very few of my ideas are truly original. I have learned from every one of these books and authors – and you can, too!


A guide to my mentors/resources

A Timeless Way of Building – a beautiful and deeply spiritual book on architecture that changed the way many developers approach writing software.

Peopleware – a classic text on the human element behind writing software. Refactoring – formalized techniques for improving the internals of one's code

without affect its behavior. Code Complete – another classic programming book covering many aspects of

code construction. The Cult of Information – thought-provoking analysis of some of the down-

sides of our information age. Patterns of Software – a book that wrestles with the realities and problems with

code reuse and design patterns. Extreme Programming Explained – excellent introduction to XP. Code and Other Laws of Cyberspace – a groundbreaking book that recasts

the role of software developers as law-writers, and questions the direction that software is today taking us.


So Much to Learn...

Don't panic -- but don't stick your head in the sand, either.– You won't thrive as an Oracle7, Oracle8 or Oracle8i

developer!

You can do so much more from within PL/SQL than you ever could before.– Familiarity with new features will greatly ease the

challenges you face.

Hands on Collections

Documents

Transcript of Hands on Collections