XML and DB2XML and DB2By Josephine Cheng, Jane XuBy Josephine Cheng, Jane Xu

IBM Santa Teresa LaboratoryIBM Santa Teresa Laboratory

Slides edited from the original slides of Yunyu Song

OutlineOutline

Net.DataNet.Data

XML ExtenderXML Extender

Create XML document from DB2Create XML document from DB2

Transform XML document to DB2Transform XML document to DB2

Searching XML documentsSearching XML documents

ConclusionConclusion

Net.Data VINet.Data VI

A scripting language to create web A scripting language to create web applications which can access data from applications which can access data from any ODBC compliant databases (e.g. any ODBC compliant databases (e.g. DB2, Oracle, DRDA, ODBC, flat files and DB2, Oracle, DRDA, ODBC, flat files and web registry data)web registry data)

Can generate XML documents from SQL Can generate XML documents from SQL queries.queries.

Its successor product is WebSphere Its successor product is WebSphere Application server. Application server.

Generating XML documents from Generating XML documents from existing DB2 dataexisting DB2 data

Net.Data Macro to generate XML Data

Generating XML documents from Generating XML documents from existing DB2 dataexisting DB2 data

Resultset of XML data generated by Net.Data

OutlineOutline

Net.DataNet.Data

XML ExtenderXML Extender

Create XML document from DB2Create XML document from DB2

Transform XML document to DB2Transform XML document to DB2

Searching XML documentsSearching XML documents

ConclusionConclusion

XML ExtenderXML Extender

Repository for XML documents and their Repository for XML documents and their DTDs.DTDs.

Data management functionalities (integrity, Data management functionalities (integrity, security, recoverability and management).security, recoverability and management).

User has the option to store the entire User has the option to store the entire document as an column or decomposed document as an column or decomposed into multiple tables and columns.into multiple tables and columns.

Users can retrieve the entire document or Users can retrieve the entire document or extract XML elements and attributes extract XML elements and attributes dynamically in an SQL query.dynamically in an SQL query.

XML CollectionXML Collection

SQL Collection MappingSQL Collection MappingIt only works when you can collect all relevant data with a single SELECT statement, It only works when you can collect all relevant data with a single SELECT statement, which is placed in the DAD. If multiple tables are involved you should use primary-which is placed in the DAD. If multiple tables are involved you should use primary-foreign key relationships. foreign key relationships.

RDB_Node Collection MappingRDB_Node Collection MappingIt allows node definitions for multiple tables. All involved tables must have primary It allows node definitions for multiple tables. All involved tables must have primary keys. This approach does not require SQL in the DAD. It does allow imports as well keys. This approach does not require SQL in the DAD. It does allow imports as well as exports.as exports.

XML CollectionXML Collection

XML CollectionXML Collection

XML ColumnXML Column

Store entire XML document in a column of Store entire XML document in a column of XML typeXML type

Allows to query XML contentAllows to query XML content Associate and store DTD in DB2 for one or Associate and store DTD in DB2 for one or

more documentmore document Map element and attribute to DB2 table (side Map element and attribute to DB2 table (side

table)table) Used when your XML storage is mainly Used when your XML storage is mainly

archival or for reference, and document archival or for reference, and document content is updated infrequentlycontent is updated infrequently

XML ColumnXML Column

XML ColumnXML Column

DAD FileDAD File

Define which DB2 tables hold the data Define which DB2 tables hold the data The The mapping methodmapping method: SQL Collection, RDB : SQL Collection, RDB

Node Collection, or XML ColumnNode Collection, or XML Column Whether to validate XML documents with a DTD Whether to validate XML documents with a DTD

before storing or translating the contents before storing or translating the contents Which DTD to use for validation and translation Which DTD to use for validation and translation

Sample DAD (Sample DAD (SQL CollectionSQL Collection))

Sample DAD (Sample DAD (RDB Node CollectionRDB Node Collection))

Sample DAD (XMLColumn)Sample DAD (XMLColumn)

XML in DB2XML in DB2

OutlineOutline

Net.DataNet.Data

XML ExtenderXML Extender

Create XML document from DB2Create XML document from DB2

Transform XML document to DB2Transform XML document to DB2

Searching XML documentsSearching XML documents

ConclusionConclusion

Create XML from DB2Create XML from DB2

Generate XML fragments (SQL Collection)Generate XML fragments (SQL Collection)

Generate whole XML documents (RDB Generate whole XML documents (RDB

Node Collection)Node Collection)

Generate XML fragments (Generate XML fragments (SQL SQL

CollectionCollection))

REC2XMLREC2XML Return XML fragmentsReturn XML fragments

XMLELEMENTXMLELEMENT Return Single elementsReturn Single elements

XMLATTRIBUTESXMLATTRIBUTES Group elements as XML Group elements as XML attributes attributes

XMLAGGXMLAGG Aggregate elements in a Aggregate elements in a hierarchy hierarchy

Examples IExamples I

SELECT XML2CLOB( XMLELEMENT(NAME "Department", SELECT XML2CLOB( XMLELEMENT(NAME "Department", XMLATTRIBUTES(e.workdept AS "name"), XMLATTRIBUTES(e.workdept AS "name"), XMLAGG(XMLELEMENT(NAME "emp", e.lastname) ORDER BY XMLAGG(XMLELEMENT(NAME "emp", e.lastname) ORDER BY e.lastname) )) AS "dept_list" FROM employee e WHERE e.lastname) )) AS "dept_list" FROM employee e WHERE e.workdept IN ('C01','E21') GROUP BY workdepte.workdept IN ('C01','E21') GROUP BY workdept

This query produces the following output. This query produces the following output. dept_list dept_list <Department name = "C01"> <Department name = "C01">

<emp>KWAN</emp> <emp>KWAN</emp> <emp>NICHOLLS</emp> <emp>NICHOLLS</emp> <emp>QUINTANA</emp> <emp>QUINTANA</emp>

</Department> </Department> <Department name = "E21"> <Department name = "E21">

<emp>GOUNOT</emp> <emp>GOUNOT</emp> <emp>LEE</emp> <emp>LEE</emp> <emp>MEHTA</emp><emp>MEHTA</emp> <emp>SPENSER</emp> <emp>SPENSER</emp>

</Department> </Department>

Example IIExample II

REC2XMLREC2XML Fuction Fuction

select rec2xml(1.0, 'COLATTVAL','MyRow',DEPTNO, DEPTNAME, select rec2xml(1.0, 'COLATTVAL','MyRow',DEPTNO, DEPTNAME, MGRNO) AS MYROWS FROM DEPARTMENTMGRNO) AS MYROWS FROM DEPARTMENT MYROWS MYROWS

-------------------------------------------------------------------- --------------------------------------------------------------------

<MyRow> <MyRow>

<column name="DEPTNO">A00</column> <column name="DEPTNO">A00</column>

<column name="DEPTNAME">SPIFFY COMPUTER SERVICE <column name="DEPTNAME">SPIFFY COMPUTER SERVICE DIV.</column>DIV.</column>

<column name="MGRNO">000010</column> <column name="MGRNO">000010</column>

</MyRow> </MyRow>

<MyRow><MyRow>

<column name="DEPTNO">B01</column> <column name="DEPTNO">B01</column>

<column name="DEPTNAME">PLANNING</column> <column name="DEPTNAME">PLANNING</column>

<column name="MGRNO">000020</column><column name="MGRNO">000020</column>

</MyRow> </MyRow>

Compose XML documentsCompose XML documents((RDB NodeRDB Node))

dxxGenXMLdxxGenXML generates an XML document according to the given generates an XML document according to the given DAD and places the result in a table you specify DAD and places the result in a table you specify

used when document update occasionallyused when document update occasionally

dxxGenXMLClobdxxGenXMLClob identical with identical with dxxGenXMLdxxGenXMLexcept that it returns the XML directly as a Character except that it returns the XML directly as a Character Large Object (CLOB)Large Object (CLOB)

dxxRetrieveXMLdxxRetrieveXML Used when composing or decomposing XML Used when composing or decomposing XML documents frequentlydocuments frequently

Using the Using the dxxEnableCollectiondxxEnableCollection stored procedure to stored procedure to create a named collection and stores the DAD within create a named collection and stores the DAD within the databasethe database

Then refer to it by name using Then refer to it by name using dxxRetrieveXMLdxxRetrieveXML and and dxxRetrieveXMLClobdxxRetrieveXMLClob

ExamplesExamples

dxxGenXML(CLOB(100K) DAD, /* input */ dxxGenXML(CLOB(100K) DAD, /* input */

char(resultTabName) resultTabName, /* input */char(resultTabName) resultTabName, /* input */

integer overrideType /* input */integer overrideType /* input */

varchar(1024) override, /* input */ varchar(1024) override, /* input */

integer maxRows, /* input */ integer maxRows, /* input */

integer numRows, /* output */ integer numRows, /* output */

long returnCode, /* output */ long returnCode, /* output */

varchar(1024) returnMsg) /* output */ varchar(1024) returnMsg) /* output */

dxxRetrieveXML(char(collectionName) collectionName, /* input */ dxxRetrieveXML(char(collectionName) collectionName, /* input */

char(resultTabName) resultTabName, /* input */ char(resultTabName) resultTabName, /* input */

integer overrideType, /* input */ integer overrideType, /* input */

varchar_value override, /* input */ varchar_value override, /* input */

integer maxRows, /* input */ integer maxRows, /* input */

integer numRows, /* output */ integer numRows, /* output */

long returnCode, /* output */ long returnCode, /* output */

varchar(1024) returnMsg) /* output */ varchar(1024) returnMsg) /* output */

OutlineOutline

Net.DataNet.Data

XML ExtenderXML Extender

Create XML document from DB2Create XML document from DB2

Transform XML document to DB2Transform XML document to DB2

Searching XML documentsSearching XML documents

ConclusionConclusion

Transform XML into DB2Transform XML into DB2

XML RDB_Node CollectionXML RDB_Node Collection

XML columnXML column

RDB_Node collectionRDB_Node collection In this case it is the reverse of generating XML from an RDB Node In this case it is the reverse of generating XML from an RDB Node

Collection. Collection. The end result of this is pure relational data, in tables, and ready to be The end result of this is pure relational data, in tables, and ready to be accessed with the usual database tools. You can do Queries, indexes, accessed with the usual database tools. You can do Queries, indexes, views, optimizations etc. views, optimizations etc.

dxxShredXMLdxxShredXML

Takes the DAD as a parameter, similar to Takes the DAD as a parameter, similar to dxxGenXMLdxxGenXML

Used for applications that do occasional updates Used for applications that do occasional updates dxxInsertXMLdxxInsertXML

Uses a DAD already stored, similar to Uses a DAD already stored, similar to dxxRetrieveXMLdxxRetrieveXML

Used for applications that make regular updatesUsed for applications that make regular updates

Takes an enabled XML collection as input parameterTakes an enabled XML collection as input parameter

UDTs provided by XML extenderUDTs provided by XML extender((XML ColumnXML Column))

XMLCLOBXMLCLOB XML content stored as a character XML content stored as a character larger object (CLOB) in DB2larger object (CLOB) in DB2

XMLVARCHARXMLVARCHAR XML content stored as VARCHAR in XML content stored as VARCHAR in DB2DB2

XMLDBCLOBXMLDBCLOB XML document stored as double XML document stored as double byte CLOB in DB2byte CLOB in DB2

XMLFILEXMLFILE XML document in a file on a local XML document in a file on a local file systemfile system

Store XML as XML Column in Store XML as XML Column in DB2DB2

1.1. Ensure that the database is XML-enabled Ensure that the database is XML-enabled

2.2. Optional: Locate or create a DTD for your documents and store it Optional: Locate or create a DTD for your documents and store it in in DB2XML.DTD_REFDB2XML.DTD_REF. .

3.3. Create the table to store the XML documents and add the XML Create the table to store the XML documents and add the XML column. Also create any side tables. column. Also create any side tables.

4.4. Create a DAD to map the XML to the relational DB structures. Create a DAD to map the XML to the relational DB structures.

5.5. Enable the XML column, specifying the DAD. Enable the XML column, specifying the DAD.

6.6. Create indexes for the side-tables. Create indexes for the side-tables.

7.7. Insert some XML documents and start performing queries. Insert some XML documents and start performing queries.

Key Features of XML Column I Key Features of XML Column I

The XML extender supports a subset of The XML extender supports a subset of XPathXPath, providing , providing

a way to locate specific data within an XML document. a way to locate specific data within an XML document.

Combined with the provided Combined with the provided extracttypeextracttype functions functions

((extractIntegerextractInteger, , extractTimestampextractTimestamp, , extractVarCharextractVarChar, etc.) , etc.)

you can retrieve values as database types. you can retrieve values as database types.

Example IExample I

Example IIExample II

Key features of XML Column IIKey features of XML Column II

You can select to copy specific element and attribute You can select to copy specific element and attribute

values to values to a a side tableside table. Doing that lets you search and . Doing that lets you search and

manipulate the values directly via SQL just like any other manipulate the values directly via SQL just like any other

column data, linking to the stored XML to pick up related column data, linking to the stored XML to pick up related

values. values.

You can create side table indexes, to improve search You can create side table indexes, to improve search

time. time.

Use side table for fast searchUse side table for fast search

Key features of XML Column IIIKey features of XML Column III

XPathXPath based element or attribute update based element or attribute update

UPDATE sales_tab SET order = Update(order, UPDATE sales_tab SET order = Update(order, '/Order/Customer/Name', 'Customer X') WHERE '/Order/Customer/Name', 'Customer X') WHERE sales_person = 'Salesperson Y' sales_person = 'Salesperson Y'

Using the Update function, you can correctly alters both the stored Using the Update function, you can correctly alters both the stored XML document and any relevant side tables. XML document and any relevant side tables.

OutlineOutline

Net.DataNet.Data

XML ExtenderXML Extender

Create XML document from DB2Create XML document from DB2

Transform XML document to DB2Transform XML document to DB2

Searching XML documentsSearching XML documents

ConclusionConclusion

Searching XML docs (Searching XML docs (XML ColumnXML Column))

ExampleExample

of of

XMLXML

ColumnColumn

Using side tables.Using side tables.

Can make a view of the joint.Can make a view of the joint.

Searching XML documents Searching XML documents (Column)(Column)

Example of query against attribute.Example of query against attribute.

The UDF is expensive. Alternatives:The UDF is expensive. Alternatives: Use DAD to define CUSTOMER as another Use DAD to define CUSTOMER as another index in index in

the side tablethe side table.. Enable XML column to use Enable XML column to use text indexingtext indexing (e.g. section (e.g. section

search with search with db2tx.containsdb2tx.contains).).

Searching XML documents Searching XML documents (Column)(Column)

Search can be done directly against the Search can be done directly against the SQL based tables, or using procedures SQL based tables, or using procedures dxxGenXML()dxxGenXML() and and dxxRetrieveXML()dxxRetrieveXML()..

The DAD is used to specify whether to The DAD is used to specify whether to retrieve the entire document or a fragment.retrieve the entire document or a fragment.

The DAD is also used to specify the The DAD is also used to specify the search criteria which can be based either search criteria which can be based either on tables or SQL query.on tables or SQL query.

Searching XML documents Searching XML documents (Collect)(Collect)

OutlineOutline

Net.DataNet.Data

XML ExtenderXML Extender

Create XML document from DB2Create XML document from DB2

Transform XML document to DB2Transform XML document to DB2

Searching XML documentsSearching XML documents

ConclusionConclusion

Conclusion: XML in DB2Conclusion: XML in DB2

RequirementRequirement SolutionsSolutions

generate XML document from generate XML document from relational data (DB2 tables)relational data (DB2 tables)

Use simple functions (Use simple functions (REC2XML XMLELEMENT etc.)REC2XML XMLELEMENT etc.)

Use a "SQL Collection" to map a query to an XML document Use a "SQL Collection" to map a query to an XML document structure structure

Use an "RDB Node Collection" to map a number of tables to an Use an "RDB Node Collection" to map a number of tables to an XML document structure (this also allows XML import) XML document structure (this also allows XML import)

Use Net.Data Macro to generate XML DataUse Net.Data Macro to generate XML Data

Ignore DB2's capability and use Visual Basic or VB ScriptIgnore DB2's capability and use Visual Basic or VB Script

Transform XML to relational dataTransform XML to relational data Use “RDB_node collection” to store XML in DB2 tablesUse “RDB_node collection” to store XML in DB2 tables

Store XML documents in DB2Store XML documents in DB2

Database for later referenceDatabase for later reference

Use Use an "XML Column" to store entire XML documents.They can an "XML Column" to store entire XML documents.They can still be indexed and queried via DB2 functionsstill be indexed and queried via DB2 functions