1

XQuery XML Databases

Roger L. Costello16 June 2010

2

XQuery Databases

• XQuery is an XML query language.

• It can be used to efficiently and easily extract information from Native XML Databases (NXD).

• It can be used to query XML views of relational data.

3

Database Systems supporting XQuery

• The following database systems offer XQuery support: – Native XML Databases:

• Berkeley DB XML• eXist• MarkLogic• Software AG Tamino• Raining Data TigerLogic• Documentum xDb (X-Hive/DB)

– Relational Databases: • IBM DB2• Microsoft SQL Server• Oracle

4

Native XML Database (NXD)

• Native XML databases have an XML-based internal model, i.e., their fundamental unit of storage is XML.

• To enable Oxygen XML to use a database, you need to download the database's .jar files and then point Oxygen to them.

• For instructions on configuring each database, select the Oxygen Help menu and search for this: Native XML Database (NXD) Support

5

Use Cases for Native XML DatabasesTable of contents•1.0 Got XML? •2.0 A very quick review of native XML databases •3.0 Storing and querying document-centric XML

•3.1 Documents in the real world •3.2 Inside the applications

•3.2.1 Managing documents •3.2.2 Finding documents •3.2.3 Retrieving information •3.2.4 Reusing content

•3.3 Why you need a native XML database •3.4 A peek into the future

•4.0 Data integration •4.1 Data integration in the real world •4.2 Inside the applications

•4.2.1 Query architectures •4.2.2 Handling differences in schemas

•4.3 Why you need a native XML database •4.4 A peek into the future

•5.0 Working with semi-structured data •5.1 Semi-structured data in the real world •5.2 Inside the applications •5.3 Why you need a native XML database •5.4 A peek into the future

•6.0 Schema evolution •6.1 Schema evolution in the real world •6.2 Inside the applications •6.3 Why you need a native XML database •6.4 A peek into the future

•7.0 Long-running transactions •8.0 Handling large documents •9.0 Hierarchical data •10.0 Other uses •11.0 A final peek into the future •12.0 Conclusion •13.0 Thanks •14.0 Resources

http://www.rpbourret.com/xml/UseCases.htm

6

XML and Databases

Table of Contents•1.0 Introduction •2.0 Is XML a Database? •3.0 Why Use a Database? •4.0 Data versus Documents

•4.1 Data-Centric Documents •4.2 Document-Centric Documents •4.3 Data, Documents, and Databases

•5.0 Storing and Retrieving Data •5.1 Mapping Document Schemas to Database Schemas

•5.1.1 Table-Based Mapping •5.1.2 Object-Relational Mapping

•5.2 Query Languages •5.2.1 Template-Based Query Languages •5.2.2 SQL-Based Query Languages •5.2.3 XML Query Languages

•5.3 Storing Data in a Native XML Database •5.4 Data Types, Null Values, Character Sets, and All That Stuff

•5.4.1 Data Types •5.4.2 Binary Data •5.4.3 Null Data •5.4.4 Character Sets •5.4.5 Processing Instructions and Comments •5.4.6 Storing Markup

•5.5 Generating XML Schemas from Relational Schemas and Vice Versa •6.0 Storing and Retrieving Documents

•6.1 Storing Documents in the File System •6.2 Storing Documents in BLOBs •6.3 Native XML Databases

•6.3.1 What is a Native XML Database? •6.3.2 Native XML Database Architectures

•6.3.2.1 Text-Based Native XML Databases •6.3.2.2 Model-Based Native XML Databases

•6.3.3 Features of Native XML Databases •6.3.3.1 Document Collections •6.3.3.2 Query Languages •6.3.3.3 Updates and Deletes •6.3.3.4 Transactions, Locking, and Concurrency •6.3.3.5 Application Programming Interfaces (APIs) •6.3.3.6 Round-Tripping •6.3.3.7 Remote Data •6.3.3.8 Indexes

http://www.rpbourret.com/xml/XMLAndDatabases.htm

7

MarkMail -- An email search site built on XQuery

http://markmail.org/

8

9

eXist

• It is a native XML database• It is open source• More info: http://exist-db.org

• Want to see some sample XQueries run against an eXist database? Check out the eXist sandbox: http://demo.exist-db.org/exist/sandbox/sandbox.xql

• In the drop-down box labeled "Paste example" select "Find books by author". Examine the XQuery then press the Send button.

10

11

Installing eXist

• In your C: folder create a new folder, eXist C: eXist

• In the XQuery folder is a subfolder, eXist-exe-file. Open it. Double click on the .exe file.

• The second prompt will ask: Select the installation path. Enter: C:\eXist

• As the password for user ‘admin’ enter: admin• Select these packs to install: Core, Javadocs (we don’t need

Source)

12

Running eXist

• After it’s finished installing, start the database:– Start All Programs eXist XML Database eXist Database Startup

(alternatively, double click startup.bat in c:\eXist\bin)

– Minimize the DOS window that is generated by starting the database (don’t close it)

– Open a browser and enter: http://localhost:8080/exist/index.xml

• View the Admin page. The link to it is on the left side. You need to scroll down to see the link. Remember the password is: admin.

13

How to configure an eXist datasource in Oxygen

1. Open Oxygen. Go to Options -> Preferences -> Data Sources. In the Data Sources panel click the New button.

2. Enter this name for this data source: eXist-DB Select eXist from the driver type combo box.

3. Press the Add button and add the following eXist specific files. They are located in the eXist installation root directory (c:\eXist):

1. exist.jar2. lib/core/xmldb.jar3. lib/core/xmlrpc-client-3.1.1.jar4. lib/core/xmlrpc-common-3.1.1.jar5. lib/core/ws-commons-util-1.0.2.jar

4. Click OK to finish the data source configuration.

14

How to configure an eXist connection in Oxygen

1. Open Oxygen. Go to Options -> Preferences -> Data Sources. In the Connections panel click the New button.

2. Enter this name for this connection: eXist-Connection Select eXist-DB from the Data Source combo box.

3. Fill in the Connection Details:1. XML DB URI: use this value:

xmldb:exist://localhost:8080/exist/xmlrpc2. User: use this value: guest3. Password: use this value: guest4. Collection: use this value: /db

4. Click OK to finish the connection configuration.

15

eXist Collections

• eXist organizes all documents in hierarchical collections.

• Collections are like directories. They are used to group related documents together.

• On the last slide you set the default collection name to /db

16

Add a new Collectionto the Database

• Switch on Oxygen's Database Perspective: Perspective -> Database

• Right-click on db, and select Add a collection…

17

Add a new Collectionto the Database

• Type in example01

18

Put a Resource (XML File)into the Collection

• Right-click on example01, and select Add Resource …

19

Put a Resource (XML File)into the Collection

• Select FitnessCenter.xml in the xquery/eXist-examples/example01 folder

20

Editing Resources

• Double clicking on FitnessCenter.xml allows you toview and edit it:

21

Run an XQuery on the XML

• Drag and drop FitnessCenter.xq into Oxygen. Modify the XQuery to use the doc() function (i.e., use an explicit input):

22

Specify the Transformer

• Click on the wrench icon then click on the edit button:

23

Specify the Transformer

• Select eXist-Connection in the Transformer drop-down box:

24

Execute the XQuery

• Click on the red triangle:

25

View the Results

26

Remember to Restart eXist

• Suppose that you shut down your computer for the day. Tomorrow you Open Oxygen and try to run an XQuery on FitnessCenter.xml in the eXist database.

• You get an error. Why?

• Answer: you need to restart eXist before you open Oxygen.– Start All Programs eXist XML Database eXist Database Startup

(alternatively, double click startup.bat in c:\eXist\bin)

27

XQuery Update in eXist

• eXist supports XQuery update in a slightly non-standard way.

• You must precede each update operation with update, e.g.,

for $i in doc('FitnessCenter.xml')//(*|@*) return update rename node $i as lower-case(name($i))

Do Lab1