Jonathan Robie
XQuery Technology LeadDataDirect Technologies
2
DataDirect XQuery 2.0
3
Data Integration without DataDirect XQuery
SOAPthrough AXIS
dBASE IVAPIs
JDBC
Java/JSP codeaccessing the various
Java APIs and generating the HTML report
Web Browser
4
The DataDirect XQuery Solution
Java/JSP
5
Dealing with multiple, heterogeneous Data Sources:A dangerous approach
Data Source
Data Source
Data Source
Data Source
Data Source
Data Consumer
Data Consumer
Data Consumer
Data Consumer
Data Consumer
Data Source
Data Source
Data Source
Data Source
Data SourceEDI Message Web Service
RDBMS RDBMSXML Document
<XML>
Data Access Layer
AJAXClient
DynamicHTMLClient
WebServiceClient
PublishingApps
RESTClient
6
The XQuery Vision
Data Source
Data Source
Data Source
Data Source
Data Source
Data Consumer
Data Consumer
Data Consumer
Data Consumer
Data Consumer
Data Source
Data Source
Data Source
Data Source
Data SourceEDI Message Web Service
RDBMS RDBMSXML Document
<XML>
AJAXClient
DynamicHTMLClient
WebServiceClient
PublishingApps
RESTClient
XML
XML
XQuery
7
Querying XML and Relational – the same way!
for $h in doc("holdings.xml")/holdings/entryfor $c in collection("companies")/companieswhere $h/userid = "Minollo" and $c/ticker = $h/stocktickerreturn <company ticker="{ $c/ticker }"> { $c/companyname } { $c/annualrevenues } </company>
8
XQuery Data Integration? DataDirect!
• In-memory XQuery implementations• Only for physical XML
• Database XQuery implementations• Only for their database• Relational Database XQuery implementations• XML Database XQuery implementations• Full-text XQuery implementations
• Server-based XQuery implementations• Only for their server• Application Server• Integration Server
• DataDirect XQuery is Open!• All leading relational databases• XML• Many other data sources• Any app server – or none
9
DataDirect XQuery
**
• Accesses almost any data source
• No dependency on servers
• Standards-based
• High performance
• Scalable
• Embeddable
• Plugs into any Java architecture – supports XQuery for Java API (XQJ)
10
What is XQJ?
• XQuery API for Java (XQJ) – JSR 225
• “The JDBC for XQuery”
11
DataDirect XQuery – Input, Output
• Relational Databases• SQL Server 2000, 2005• Oracle 9i, 10gR1, 10gR2• DB2 Windows/UNIX/Linux: v8.x, v9 • DB2 iSeries: V5R2, V5R3• DB2: z/OS: v8 • Sybase: 12.5.x, 15
• XML sources• http: ftp: and file: schemes • XML in DOM trees• Drastically reduced memory
requirements for large XML documents.
• Custom URI Resolvers
• Legacy sources• Converters for CSV, EDI, and more
• SOA Integration• Web Service calls
• XML Output• XML as text• XML in DOM trees • XML in SAX streams • XML in StAX streams
12
XML Processing without XQuery<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <soap:Body> <GetQuotesResponse xmlns="http://swanandmokashi.com"> <GetQuotesResult> <Quote> <CompanyName>APPLE COMPUTER</CompanyName> <StockTicker>AAPL</StockTicker> <StockQuote>74.17</StockQuote> <LastUpdated>9/14/2006 4:01pm</LastUpdated> <Change>1.17</Change> <PercentChange>1.82%</PercentChange> <OpenPrice>N/A</OpenPrice> <DayHighPrice>N/A</DayHighPrice> <DayLowPrice>N/A</DayLowPrice> <Volume>0</Volume> <MarketCap>63.266B</MarketCap> <YearRange>47.87 - 86.40</YearRange> <ExDividendDate>21-Nov-95</ExDividendDate> <DividendYield>N/A</DividendYield> <DividendPerShare>0.00</DividendPerShare> </Quote> </GetQuotesResult> </GetQuotesResponse> </soap:Body></soap:Envelope>
13
XML Processing without XQuery<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <soap:Body> <GetQuotesResponse xmlns="http://swanandmokashi.com"> <GetQuotesResult> <Quote> <CompanyName>APPLE COMPUTER</CompanyName> <StockTicker>AAPL</StockTicker> <StockQuote>74.17</StockQuote> <LastUpdated>9/14/2006 4:01pm</LastUpdated> <Change>1.17</Change> <PercentChange>1.82%</PercentChange> <OpenPrice>N/A</OpenPrice> <DayHighPrice>N/A</DayHighPrice> <DayLowPrice>N/A</DayLowPrice> <Volume>0</Volume> <MarketCap>63.266B</MarketCap> <YearRange>47.87 - 86.40</YearRange> <ExDividendDate>21-Nov-95</ExDividendDate> <DividendYield>N/A</DividendYield> <DividendPerShare>0.00</DividendPerShare> </Quote> </GetQuotesResult> </GetQuotesResponse> </soap:Body></soap:Envelope>
• Without XQuery• Parse XML• Navigate (DOM) or Capture Events (SAX)• Cast XML values to Java values• Create XML for output
• With XQuery
declare default element namespace "http://swanandmokashi.com";
for $q in doc("aapl.xml")//Quotewhere $q/PercentChange > 1return <Quote> { $q/StockTicker } { $q/StockQuote } { $q/PercentChange } </Quote>
14
DataDirect XQuery is fast for XML files!<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <soap:Body> <GetQuotesResponse xmlns="http://swanandmokashi.com"> <GetQuotesResult> <Quote> <CompanyName>APPLE COMPUTER</CompanyName> <StockTicker>AAPL</StockTicker> <StockQuote>74.17</StockQuote> <LastUpdated>9/14/2006 4:01pm</LastUpdated> <Change>1.17</Change> <PercentChange>1.82%</PercentChange> <OpenPrice>N/A</OpenPrice> <DayHighPrice>N/A</DayHighPrice> <DayLowPrice>N/A</DayLowPrice> <Volume>0</Volume> <MarketCap>63.266B</MarketCap> <YearRange>47.87 - 86.40</YearRange> <ExDividendDate>21-Nov-95</ExDividendDate> <DividendYield>N/A</DividendYield> <DividendPerShare>0.00</DividendPerShare> </Quote> </GetQuotesResult> </GetQuotesResponse> </soap:Body></soap:Envelope>
• General XQuery rewrites• Elimination of common sub-expressions,
loop rewrites, ordering rewrites, etc…
• Document projection• XML construction accounts for much of the
cost• Don’t build parts of the document that the
query doesn’t need!
• Document streaming• Discard parts of the document when no
longer needed• Makes memory usage near constant with
size of file
• Multiple Gigabytes can be queried
15
DataDirect XQuery for Relational Data
HOLDINGS
USERID TICKER SHARES
Jonathan PRGS 23
Minollo PRGS 4000000
Jonathan AMZN 3000
Minollo AMZN 3000
<portfolio > <company ticker="AMZN"> <companyname>Amazon.com, Inc.</companyname> <annualrevenues>7780</annualrevenues> </company> <company ticker="EBAY"> <companyname>eBay Inc.</companyname> <annualrevenues>22600</annualrevenues> </company> <company ticker="IBM"> <companyname>Int'l Business Machines C</companyname> <annualrevenues>128200</annualrevenues> </company> <company ticker="PRGS"> <companyname>Progress Software</companyname> <annualrevenues>493.4</annualrevenues> </company></portfolio>
COMPANIES
TICKER NAME ANNUALREVENUES
AMZN Amazon.com, Inc. 7780
EBAY eBay Inc. 22600
PRGS Progress Software 493.4
YHOO Yahoo! Inc. 10700
16
Database Publishing without DataDirect XQuery
HOLDINGS
USERID TICKER SHARES
Jonathan PRGS 23
Minollo PRGS 4000000
Jonathan AMZN 3000
Minollo AMZN 3000
<portfolio > <company ticker="AMZN"> <companyname>Amazon.com, Inc.</companyname> <annualrevenues>7780</annualrevenues> </company> <company ticker="EBAY"> <companyname>eBay Inc.</companyname> <annualrevenues>22600</annualrevenues> </company> <company ticker="IBM"> <companyname>Int'l Business Machines C</companyname> <annualrevenues>128200</annualrevenues> </company> <company ticker="PRGS"> <companyname>Progress Software</companyname> <annualrevenues>493.4</annualrevenues> </company></portfolio>
COMPANIES
TICKER NAME ANNUALREVENUES
AMZN Amazon.com, Inc. 7780
EBAY eBay Inc. 22600
PRGS Progress Software 493.4
YHOO Yahoo! Inc. 10700
• Without XQuery• Use SQL for Relational Data• Use JDBC for database connection• Use Java + XML API to build XML hierarchies from flat result sets• Use XSLT for transformations
• With XQuery
for $h in collection("HOLDINGS")/holdingsfor $c in collection("COMPANIES")/companieswhere $h/ticker = $c/ticker and $h/userid = "Jonathan"return <portfolio> <company ticker="{$c/ticker}"> <companyname>{string($c/name)}</companyname> <annualrevenues>{string($c/annualrevenues)}</annualrevenues> </company> </portfolio>
17
DataDirect XQuery is fast for Relational Data!
HOLDINGS
USERID TICKER SHARES
Jonathan PRGS 23
Minollo PRGS 4000000
Jonathan AMZN 3000
Minollo AMZN 3000
<portfolio > <company ticker="AMZN"> <companyname>Amazon.com, Inc.</companyname> <annualrevenues>7780</annualrevenues> </company> <company ticker="EBAY"> <companyname>eBay Inc.</companyname> <annualrevenues>22600</annualrevenues> </company> <company ticker="IBM"> <companyname>Int'l Business Machines C</companyname> <annualrevenues>128200</annualrevenues> </company> <company ticker="PRGS"> <companyname>Progress Software</companyname> <annualrevenues>493.4</annualrevenues> </company></portfolio>
COMPANIES
TICKER NAME ANNUALREVENUES
AMZN Amazon.com, Inc. 7780
EBAY eBay Inc. 22600
PRGS Progress Software 493.4
YHOO Yahoo! Inc. 10700
• Highly optimized for relational sources
• Minimizes retrieval of data• No more rows than needed• No more columns than needed
• Uses database functionality • Joins• Sorting• Etc..
• Optimizes for each SQL dialect
• Efficient JDBC retrieval• Embeds DataDirect JDBC technology• Optimizations added to support XQuery
• Supports incremental retrieval
• Optimizes for XML hierarchies• Sort-merge algorithm• Minimal cost of XML construction
• Leverages SQL library
• Recognizes equivalences
• Supports hints
18
XML Converters
EDI File:
ISA+00+DATADIRECT+00+STYLUS2006+01+DATA DIRECT +01+STYLUS STUDIO +060504+1212+~+00503+200654321+0+I+:'GS+BF+DATADIRECT+STYLUS2006+20060504+121212+256+X+005030'ST+105+3389'BGN+28+1024+20060504+121212+GM'NM1+2L+4+Progress Software Corporation'N3+14 Oak Park Drive'N4+Bedford+MA+01730+US+AA'REF+1Z+PRGS'NM1+2L+4+Apple Computer, Inc.'N3+1 Infinite Loop'N4+Cupertino+CA+95014+US+AA'REF+1Z+AAPL'SE+11+3389'GE+1+256'IEA+1+200654321'
19
XML Converters
EDI File:
ISA+00+DATADIRECT+00+STYLUS2006+01+DATA DIRECT +01+STYLUS STUDIO +060504+1212+~+00503+200654321+0+I+:'GS+BF+DATADIRECT+STYLUS2006+20060504+121212+256+X+005030'ST+105+3389'BGN+28+1024+20060504+121212+GM'NM1+2L+4+Progress Software Corporation'N3+14 Oak Park Drive'N4+Bedford+MA+01730+US+AA'REF+1Z+PRGS'NM1+2L+4+Apple Computer, Inc.'N3+1 Infinite Loop'N4+Cupertino+CA+95014+US+AA'REF+1Z+AAPL'SE+11+3389'GE+1+256'IEA+1+200654321'
doc("adapter://EDI?ticker-request.edi")
<X12> <ISA> <ISA01><!--I01: Authorization Information Qualifier-->00<!--No Authorization Information Present (No Meaningful Information in I02)--></ISA01> <ISA02><!--I02: Authorization Information-->DATADIRECT</ISA02> <ISA03><!--I03: Security Information Qualifier-->00<!--No Security Information Present (No Meaningful Information in I04)--></ISA03> <ISA04><!--I04: Security Information-->STYLUS2006</ISA04> <ISA05><!--I05: Interchange ID Qualifier-->01<!--Duns (Dun & Bradstreet)--></ISA05> <ISA06><!--I06: Interchange Sender ID-->DATA DIRECT </ISA06> <ISA07><!--I05: Interchange ID Qualifier-->01<!--Duns (Dun & Bradstreet)--></ISA07> <ISA08><!--I07: Interchange Receiver ID-->STYLUS STUDIO </ISA08> <ISA09><!--I08: Interchange Date-->060504<!--2006-05-04--></ISA09> <ISA10><!--I09: Interchange Time-->1212</ISA10> <ISA11><!--I65: Repetition Separator-->~</ISA11> <ISA12><!--I11: Interchange Control Version Number-->00503<!--Standards Approved for Publication by ASC X12 Procedures Review Board through October 2005--></ISA12> <ISA13><!--I12: Interchange Control Number-->200654321</ISA13> <ISA14><!--I13: Acknowledgment Requested-->0<!--No Interchange Acknowledgment Requested--></ISA14> <ISA15><!--I14: Interchange Usage Indicator-->I<!--Information--></ISA15> <ISA16><!--I15: Component Element Separator-->:</ISA16> </ISA> <GS> <GS01><!--479: Functional Identifier Code-->BF<!--Business Entity Filings (105)--></GS01> <GS02><!--142: Application Sender's Code-->DATADIRECT</GS02> <GS03><!--124: Application Receiver's Code-->STYLUS2006</GS03> <GS04><!--373: Date-->20060504<!--2006-05-04--></GS04> <GS05><!--337: Time-->121212</GS05> <GS06><!--28: Group Control Number-->256</GS06> <GS07><!--455: Responsible Agency Code-->X<!--Accredited Standards Committee X12--></GS07> <GS08><!--480: Version / Release / Industry Identifier Code-->005030<!--Standards Approved for Publication by ASC X12 Procedures Review Board through October 2005--></GS08> </GS>
20
SOA - Web Services
declare function local:amazon-listing($isbn){ <tns:Request> <tns:Condition>All</tns:Condition> <tns:DeliveryMethod>Ship</tns:DeliveryMethod> <tns:FutureLaunchDate/> <tns:IdType>ASIN</tns:IdType> <tns:ItemId>{ $isbn }</tns:ItemId> <tns:ResponseGroup>Medium</tns:ResponseGroup> </tns:Request>};
let $loc := <location address="http://soap.amazon.com/onca/soap?Service=AWSECommerceService" soapaction="http://soap.amazon.com" />let $payload := local:amazon-listing("0395518482")return ws:call($loc, $payload)
21
DataDirect Stylus Studio IDE
• DataDirect Stylus Studio®• General-purpose XML IDE • XQuery Editor• Ad-hoc queries• Database Connections Window • XQuery Mapper • XQuery Debugger• XML Publisher • XML Pipeline
22
<oXygen/> plugin for Eclipse (DataDirect Edition)
• <oXygen/> plugin for Eclipse (DataDirect Edition)• Eclipse-based• XQuery Editor• Ad-hoc queries• Database Connections Window • XQuery Perspective
23
Why DataDirect XQuery?
• Performance• Generates highly optimized SQL code for relational databases• Excellent performance for very large XML files• Programs written with DataDirect XQuery generally outperform programs
written with Java + XML APIs + JDBC
• Programmer productivity• Using XQuery means 7 to 20 times less code• Using XQuery means code is easier to read and maintain• XQuery IDEs increase productivity
• Support for most common data sources• XML, Relational, Flat File, EDI, Web Service calls …• Extensible via Java
• Fits in almost any architecture• Supports all leading relational databases• Easily embeddable - does not require any server• Supports any J2EE Application Server• Runs on any Java platform
24
Getting Started …
www.xquery.com• Free Trial Download• Tutorials
• XQuery• XQJ• DataDirect XQuery
• Support Forums
Top Related