XMLIXMLI

Structure of XML DataStructure of XML Data XML Document SchemaXML Document Schema XPATHXPATH

IntroductionIntroduction XML: Extensible Markup LanguageXML: Extensible Markup Language Defined by the WWW Consortium (W3C)Defined by the WWW Consortium (W3C) Derived from SGML (Standard Generalized Markup Derived from SGML (Standard Generalized Markup

Language), but simpler to use than SGML Language), but simpler to use than SGML Documents have tags giving extra information about Documents have tags giving extra information about

sections of the documentsections of the document E.g. E.g. <title> XML </title> <slide> Introduction <title> XML </title> <slide> Introduction

…</slide>…</slide> ExtensibleExtensible, unlike HTML, unlike HTML

Users can add new tags, and Users can add new tags, and separatelyseparately specify how specify how the tag should be handled for displaythe tag should be handled for display

XML Introduction (Cont.)XML Introduction (Cont.) The ability to specify new tags, and to create nested tag The ability to specify new tags, and to create nested tag

structures make XML a great way to exchange structures make XML a great way to exchange datadata, not , not just documents.just documents. Much of the use of XML has been in data exchange Much of the use of XML has been in data exchange

applications, not as a replacement for HTMLapplications, not as a replacement for HTML Tags make data (relatively) self-documenting Tags make data (relatively) self-documenting

E.g.E.g. <bank><bank>

<account> <account> <account_number> A-101 </account_number><account_number> A-101 </account_number> <branch_name> Downtown </branch_name><branch_name> Downtown </branch_name> <balance> 500 </balance><balance> 500 </balance>

</account></account> <depositor><depositor>

<account_number> A-101 </account_number><account_number> A-101 </account_number> <customer_name> Johnson </customer_name><customer_name> Johnson </customer_name>

</depositor></depositor> </bank></bank>

XML: MotivationXML: Motivation Data interchange is critical in today’s networked worldData interchange is critical in today’s networked world

Examples:Examples: Banking: funds transferBanking: funds transfer Order processing (especially inter-company Order processing (especially inter-company

orders)orders) Scientific dataScientific data

Chemistry: ChemML, …Chemistry: ChemML, … Genetics: BSML (Bio-Sequence Markup Language), Genetics: BSML (Bio-Sequence Markup Language),

…… Paper flow of information between organizations is Paper flow of information between organizations is

being replaced by electronic flow of informationbeing replaced by electronic flow of information Each application area has its own set of standards for Each application area has its own set of standards for

representing informationrepresenting information XML has become the basis for all new generation data XML has become the basis for all new generation data

interchange formatsinterchange formats

XML Motivation (Cont.)XML Motivation (Cont.) Earlier generation formats were based on plain text with line Earlier generation formats were based on plain text with line

headers indicating the meaning of fieldsheaders indicating the meaning of fields Similar in concept to email headersSimilar in concept to email headers Does not allow for nested structures, no standard “type” Does not allow for nested structures, no standard “type”

languagelanguage Tied too closely to low level document structure (lines, Tied too closely to low level document structure (lines,

spaces, etc)spaces, etc) Each XML based standard defines what are valid elements, usingEach XML based standard defines what are valid elements, using

XML type specification languages to specify the syntaxXML type specification languages to specify the syntax DTD (Document Type Definition)DTD (Document Type Definition) XML SchemaXML Schema

Plus textual descriptions of the semanticsPlus textual descriptions of the semantics XML allows new tags to be defined as requiredXML allows new tags to be defined as required

However, this may be constrained by DTDsHowever, this may be constrained by DTDs A wide variety of tools is available for parsing, browsing and A wide variety of tools is available for parsing, browsing and

querying XML documents/dataquerying XML documents/data

Comparison with Relational DataComparison with Relational Data

Inefficient: tags, which in effect represent schema Inefficient: tags, which in effect represent schema information, are repeatedinformation, are repeated

Better than relational tuples as a data-exchange formatBetter than relational tuples as a data-exchange format Unlike relational tuples, XML data is self-documenting Unlike relational tuples, XML data is self-documenting

due to presence of tagsdue to presence of tags Non-rigid format: tags can be addedNon-rigid format: tags can be added Allows nested structuresAllows nested structures Wide acceptance, not only in database systems, but also Wide acceptance, not only in database systems, but also

in browsers, tools, and applicationsin browsers, tools, and applications

Structure of XML DataStructure of XML Data

TagTag: label for a section of data: label for a section of data ElementElement: section of data beginning with <: section of data beginning with <tagnametagname> and > and

ending with matching </ending with matching </tagnametagname>> Elements must be properly Elements must be properly nestednested

Proper nestingProper nesting <account> … <balance> …. </balance> </account><account> … <balance> …. </balance> </account>

Improper nesting Improper nesting <account> … <balance> …. </account> </balance><account> … <balance> …. </account> </balance>

Formally: every start tag must have a unique matching Formally: every start tag must have a unique matching end tag, that is in the context of the same parent element.end tag, that is in the context of the same parent element.

Every document must have a single top-level elementEvery document must have a single top-level element

Example of Nested ElementsExample of Nested Elements <bank-1><bank-1> <customer><customer>

<customer_name> Hayes </customer_name><customer_name> Hayes </customer_name> <customer_street> Main </customer_street><customer_street> Main </customer_street> <customer_city> Harrison </customer_city><customer_city> Harrison </customer_city> <account><account>

<account_number> A-102 </account_number><account_number> A-102 </account_number> <branch_name> Perryridge <branch_name> Perryridge </branch_name></branch_name> <balance> 400 </balance><balance> 400 </balance>

</account></account> <account><account> … … </account></account>

</customer></customer> .. . .

</bank-1></bank-1>

Motivation for NestingMotivation for Nesting Nesting of data is useful in data transferNesting of data is useful in data transfer

Example: elements representing Example: elements representing customer_id, customer_id, customer_namecustomer_name, and address nested within an , and address nested within an orderorder elementelement

Nesting is not supported, or discouraged, in relational Nesting is not supported, or discouraged, in relational databasesdatabases With multiple orders, customer name and address are With multiple orders, customer name and address are

stored redundantlystored redundantly normalization replaces nested structures in each order normalization replaces nested structures in each order

by foreign key into table storing customer name and by foreign key into table storing customer name and address informationaddress information

Nesting is supported in object-relational databasesNesting is supported in object-relational databases But nesting is appropriate when transferring dataBut nesting is appropriate when transferring data

External application does not have direct access to External application does not have direct access to data referenced by a foreign keydata referenced by a foreign key

Structure of XML Data (Cont.)Structure of XML Data (Cont.) Mixture of text with sub-elements is legal in XML. Mixture of text with sub-elements is legal in XML.

Example:Example: <account><account>

This account is seldom used any more.This account is seldom used any more. <account_number> A-102</account_number><account_number> A-102</account_number> <branch_name> Perryridge</branch_name><branch_name> Perryridge</branch_name> <balance>400 </balance><balance>400 </balance></account></account>

Useful for document markup, but discouraged for Useful for document markup, but discouraged for data representationdata representation

AttributesAttributes Elements can have Elements can have attributesattributes

<account <account acct-type = “checking”acct-type = “checking” > > <account_number> A-102 <account_number> A-102 </account_number></account_number> <branch_name> Perryridge </branch_name><branch_name> Perryridge </branch_name> <balance> 400 </balance><balance> 400 </balance>

</account></account> Attributes are specified by Attributes are specified by name=valuename=value pairs inside the pairs inside the

starting tag of an elementstarting tag of an element An element may have several attributes, but each An element may have several attributes, but each

attribute name can only occur onceattribute name can only occur once

<account acct-type = “checking” monthly-<account acct-type = “checking” monthly-fee=“5”>fee=“5”>

Attributes vs. SubelementsAttributes vs. Subelements

Distinction between subelement and attributeDistinction between subelement and attribute In the context of documents, attributes are part of In the context of documents, attributes are part of

markup, while subelement contents are part of the basic markup, while subelement contents are part of the basic document contentsdocument contents

In the context of data representation, the difference is In the context of data representation, the difference is unclear and may be confusingunclear and may be confusing

Same information can be represented in two waysSame information can be represented in two ways <account account_number = “A-101”> …. <account account_number = “A-101”> ….

</account></account> <account> <account>

<account_number>A-101</account_number> … <account_number>A-101</account_number> … </account> </account>

Suggestion: use attributes for identifiers of elements, Suggestion: use attributes for identifiers of elements, and use subelements for contentsand use subelements for contents

NamespacesNamespaces XML data has to be exchanged between organizationsXML data has to be exchanged between organizations Same tag name may have different meaning in different Same tag name may have different meaning in different

organizations, causing confusion on exchanged documentsorganizations, causing confusion on exchanged documents Specifying a unique string as an element name avoids Specifying a unique string as an element name avoids

confusionconfusion Better solution: use Better solution: use unique-name:element-nameunique-name:element-name Avoid using long unique names all over document by using Avoid using long unique names all over document by using

XML NamespacesXML Namespaces

<bank Xmlns:FB=‘<bank Xmlns:FB=‘http://http://www.FirstBank.comwww.FirstBank.com’>’> … …

<FB:branch><FB:branch>

<FB:branchname>Downtown</FB:branchname><FB:branchname>Downtown</FB:branchname> <FB:branchcity> Brooklyn </FB:branchcity><FB:branchcity> Brooklyn </FB:branchcity>

</FB:branch></FB:branch>……

</bank></bank>

More on XML SyntaxMore on XML Syntax Elements without subelements or text content can be Elements without subelements or text content can be

abbreviated by ending the start tag with a /> and abbreviated by ending the start tag with a /> and deleting the end tagdeleting the end tag <account number=“A-101” branch=“Perryridge” <account number=“A-101” branch=“Perryridge”

balance=“200 />balance=“200 /> To store string data that may contain tags, without the To store string data that may contain tags, without the

tags being interpreted as subelements, use CDATA as tags being interpreted as subelements, use CDATA as belowbelow <![CDATA[<account> … </account>]]><![CDATA[<account> … </account>]]>

Here, <account> and </account> are treated as just Here, <account> and </account> are treated as just stringsstrings

CDATA stands for “character data”CDATA stands for “character data”

XML Document SchemaXML Document Schema

Database schemas constrain what information can be Database schemas constrain what information can be stored, and the data types of stored valuesstored, and the data types of stored values

XML documents are not required to have an associated XML documents are not required to have an associated schemaschema

However, schemas are very important for XML data However, schemas are very important for XML data exchangeexchange Otherwise, a site cannot automatically interpret data Otherwise, a site cannot automatically interpret data

received from another sitereceived from another site Two mechanisms for specifying XML schemaTwo mechanisms for specifying XML schema

Document Type Definition (DTD)Document Type Definition (DTD) Widely usedWidely used

XML Schema XML Schema Newer, increasing useNewer, increasing use

Document Type Definition (DTD)Document Type Definition (DTD)

The type of an XML document can be specified using a DTDThe type of an XML document can be specified using a DTD DTD constraints structure of XML dataDTD constraints structure of XML data

What elements can occurWhat elements can occur What attributes can/must an element haveWhat attributes can/must an element have What subelements can/must occur inside each element, What subelements can/must occur inside each element,

and how many times.and how many times. DTD does not constrain data typesDTD does not constrain data types

All values represented as strings in XMLAll values represented as strings in XML DTD syntaxDTD syntax

<!ELEMENT element (subelements-specification) ><!ELEMENT element (subelements-specification) > <!ATTLIST element (attributes) ><!ATTLIST element (attributes) >

Element Specification in DTDElement Specification in DTD

Subelements can be specified asSubelements can be specified as names of elements, ornames of elements, or #PCDATA (parsed character data), i.e., character strings#PCDATA (parsed character data), i.e., character strings EMPTY (no subelements) or ANY (anything can be a EMPTY (no subelements) or ANY (anything can be a

subelement)subelement) ExampleExample

<! ELEMENT depositor (customer_name account_number)><! ELEMENT depositor (customer_name account_number)> <! ELEMENT customer_name (#PCDATA)><! ELEMENT customer_name (#PCDATA)>

<! ELEMENT account_number (#PCDATA)><! ELEMENT account_number (#PCDATA)> Subelement specification may have regular expressionsSubelement specification may have regular expressions

<!ELEMENT bank ( ( account | customer | depositor)+)><!ELEMENT bank ( ( account | customer | depositor)+)> Notation: Notation:

“ “|” - alternatives|” - alternatives “ “+” - 1 or more occurrences+” - 1 or more occurrences “ “*” - 0 or more occurrences*” - 0 or more occurrences

Bank DTDBank DTD

<!DOCTYPE bank [<!DOCTYPE bank [<!ELEMENT bank ( ( account | customer | depositor)+)><!ELEMENT bank ( ( account | customer | depositor)+)><!ELEMENT account (account_number branch_name <!ELEMENT account (account_number branch_name balance)>balance)><! ELEMENT customer(customer_name customer_street <! ELEMENT customer(customer_name customer_street customer_city)>customer_city)><! ELEMENT depositor (customer_name <! ELEMENT depositor (customer_name account_number)>account_number)><! ELEMENT account_number (#PCDATA)><! ELEMENT account_number (#PCDATA)><! ELEMENT branch_name (#PCDATA)><! ELEMENT branch_name (#PCDATA)><! ELEMENT balance(#PCDATA)><! ELEMENT balance(#PCDATA)><! ELEMENT customer_name(#PCDATA)><! ELEMENT customer_name(#PCDATA)><! ELEMENT customer_street(#PCDATA)><! ELEMENT customer_street(#PCDATA)><! ELEMENT customer_city(#PCDATA)><! ELEMENT customer_city(#PCDATA)>

]>]>

Attribute Specification in DTDAttribute Specification in DTD Attribute specification : for each attribute Attribute specification : for each attribute

NameName Type of attribute Type of attribute

CDATACDATA ID (identifier) or IDREF (ID reference) or IDREFS (multiple ID (identifier) or IDREF (ID reference) or IDREFS (multiple

IDREFs) IDREFs) more on this later more on this later

Whether Whether mandatory (#REQUIRED)mandatory (#REQUIRED) has a default value (value), has a default value (value), or neither (#IMPLIED)or neither (#IMPLIED)

ExamplesExamples <!ATTLIST account acct-type CDATA “checking”><!ATTLIST account acct-type CDATA “checking”> <!ATTLIST customer<!ATTLIST customer

customer_id ID # REQUIREDcustomer_id ID # REQUIREDaccounts IDREFS # REQUIRED >accounts IDREFS # REQUIRED >

IDs and IDREFsIDs and IDREFs

An element can have at most one attribute of type IDAn element can have at most one attribute of type ID The ID attribute value of each element in an XML document The ID attribute value of each element in an XML document

must be distinctmust be distinct Thus the ID attribute value is an object identifierThus the ID attribute value is an object identifier

An attribute of type IDREF must contain the ID value of an An attribute of type IDREF must contain the ID value of an element in the same documentelement in the same document

An attribute of type IDREFS contains a set of (0 or more) ID An attribute of type IDREFS contains a set of (0 or more) ID values. Each ID value must contain the ID value of an values. Each ID value must contain the ID value of an element in the same documentelement in the same document

Bank DTD with AttributesBank DTD with Attributes

Bank DTD with ID and IDREF attribute types.Bank DTD with ID and IDREF attribute types. <!DOCTYPE bank-2[<!DOCTYPE bank-2[

<!ELEMENT account (branch, balance)><!ELEMENT account (branch, balance)> <!ATTLIST account<!ATTLIST account

account_number ID # REQUIREDaccount_number ID # REQUIRED owners IDREFS # REQUIRED>owners IDREFS # REQUIRED>

<!ELEMENT customer(customer_name, <!ELEMENT customer(customer_name, customer_street, customer_street,

customer_city)>customer_city)> <!ATTLIST customer<!ATTLIST customer

customer_id ID # REQUIREDcustomer_id ID # REQUIRED accounts IDREFS # REQUIRED>accounts IDREFS # REQUIRED>

… … declarations for branch, balance, customer_name, declarations for branch, balance, customer_name, customer_street and customer_city customer_street and customer_city]>]>

XML data with ID and IDREF attributesXML data with ID and IDREF attributes

<bank-2><bank-2><account account_number=“A-401” owners=“C100 <account account_number=“A-401” owners=“C100

C102”>C102”> <branch_name> Downtown </branch_name><branch_name> Downtown </branch_name> <balance> 500 </balance><balance> 500 </balance>

</account></account><customer customer_id=“C100” accounts=“A-401”><customer customer_id=“C100” accounts=“A-401”>

<customer_name>Joe <customer_name>Joe </customer_name></customer_name>

<customer_street> Monroe <customer_street> Monroe </customer_street></customer_street>

<customer_city> Madison</customer_city><customer_city> Madison</customer_city></customer></customer><customer customer_id=“C102” accounts=“A-401 A-<customer customer_id=“C102” accounts=“A-401 A-

402”>402”> <customer_name> Mary <customer_name> Mary

</customer_name></customer_name> <customer_street> Erin <customer_street> Erin

</customer_street></customer_street> <customer_city> Newark </customer_city><customer_city> Newark </customer_city>

</customer></customer></bank-2></bank-2>

Limitations of DTDsLimitations of DTDs

No typing of text elements and attributesNo typing of text elements and attributes All values are strings, no integers, reals, etc.All values are strings, no integers, reals, etc.

Difficult to specify unordered sets of subelementsDifficult to specify unordered sets of subelements Order is usually irrelevant in databases (unlike in the Order is usually irrelevant in databases (unlike in the

document-layout environment from which XML evolved)document-layout environment from which XML evolved) (A | B)* allows specification of an unordered set, but(A | B)* allows specification of an unordered set, but

Cannot ensure that each of A and B occurs only onceCannot ensure that each of A and B occurs only once IDs and IDREFs are untypedIDs and IDREFs are untyped

The The ownersowners attribute of an account may contain a attribute of an account may contain a reference to another account, which is meaninglessreference to another account, which is meaningless

ownersowners attribute should ideally be constrained to attribute should ideally be constrained to refer to customer elementsrefer to customer elements

Tree Model of XML DataTree Model of XML Data Query and transformation languages are based on a Query and transformation languages are based on a tree tree

modelmodel of XML data of XML data An XML document is modeled as a tree, with An XML document is modeled as a tree, with nodesnodes

corresponding to elements and attributescorresponding to elements and attributes Element nodes have child nodes, which can be Element nodes have child nodes, which can be

attributes or subelementsattributes or subelements Text in an element is modeled as a text node child of Text in an element is modeled as a text node child of

the elementthe element Children of a node are ordered according to their order Children of a node are ordered according to their order

in the XML documentin the XML document Element and attribute nodes (except for the root node) Element and attribute nodes (except for the root node)

have a single parent, which is an element nodehave a single parent, which is an element node The root node has a single child, which is the root The root node has a single child, which is the root

element of the documentelement of the document ExampleExample

XPathXPath

XPath is used to address (select) parts of documents usingXPath is used to address (select) parts of documents using path expressionspath expressions

A path expression is a sequence of steps separated by “/”A path expression is a sequence of steps separated by “/” Think of file names in a directory hierarchyThink of file names in a directory hierarchy

Result of path expression: set of values that along with Result of path expression: set of values that along with their containing elements/attributes match the specified their containing elements/attributes match the specified path path

E.g. E.g. /bank/customer/customer_name/bank/customer/customer_name evaluated on the evaluated on the bank data bank data we saw earlier returns we saw earlier returns

<customer_name>Hayes</customer_name><customer_name>Hayes</customer_name><customer_name>Johnson</customer_name><customer_name>Johnson</customer_name>

E.g. E.g. /bank/customer/customer_name/text( )/bank/customer/customer_name/text( ) returns the same names, but without the enclosing tagsreturns the same names, but without the enclosing tags

XPath (Cont.)XPath (Cont.) The initial “/” denotes root of the document (above the top-level The initial “/” denotes root of the document (above the top-level

tag)tag) Path expressions are evaluated left to rightPath expressions are evaluated left to right

Each step operates on the set of instances produced by the Each step operates on the set of instances produced by the previous stepprevious step

Selection predicates may follow any step in a path, in [ ]Selection predicates may follow any step in a path, in [ ] E.g. E.g. /bank/customer/account[balance > 400] /bank/customer/account[balance > 400]

returns account elements with a balance value greater than returns account elements with a balance value greater than 400400

/bank/customer/account[balance]/bank/customer/account[balance] returns account elements returns account elements containing a balance subelementcontaining a balance subelement

Attributes are accessed using “@”Attributes are accessed using “@” E.g. E.g. /bank/customer/account[balance > 400]/@account_number/bank/customer/account[balance > 400]/@account_number

returns the account numbers of accounts with balance > 400returns the account numbers of accounts with balance > 400 Here we assume account_number is an attributeHere we assume account_number is an attribute Otherwise Otherwise /bank/customer/account[balance > /bank/customer/account[balance >

400]/account_number400]/account_number IDREF attributes are not dereferenced automatically (more on IDREF attributes are not dereferenced automatically (more on

this later)this later)

Functions in XPathFunctions in XPath XPath provides several functionsXPath provides several functions

The function The function count()count() at the end of a path counts the number of at the end of a path counts the number of elements in the set generated by the pathelements in the set generated by the path

E.g. E.g. /bank/customer/[count(./account) > 1] /bank/customer/[count(./account) > 1] Returns customer with > 1 accountsReturns customer with > 1 accounts

Also function for testing position (1, 2, ..) of node w.r.t. siblingsAlso function for testing position (1, 2, ..) of node w.r.t. siblings Boolean connectives Boolean connectives andand and and oror and function and function not()not() can be used in can be used in

predicatespredicates IDREFs can be referenced using function IDREFs can be referenced using function id()id()

id()id() can also be applied to sets of references such as IDREFS can also be applied to sets of references such as IDREFS and even to strings containing multiple references separated and even to strings containing multiple references separated by blanksby blanks

E.g. E.g. /bank/customer/account/id(@owner) /bank/customer/account/id(@owner) returns all customers referred to from the owners attribute returns all customers referred to from the owners attribute

of account elements.of account elements.

More XPath ExampleMore XPath Example

Element AA with two ancestorsElement AA with two ancestors /*/*/AA/*/*/AA

First BB element of AA elementFirst BB element of AA element /AA/BB[1]/AA/BB[1]

All the CC elements of the BB elements which has an sub-All the CC elements of the BB elements which has an sub-element A with value ‘3’ element A with value ‘3’ /BB[A=‘3’]/CC/BB[A=‘3’]/CC

Any elements AA or elements CC of elements BBAny elements AA or elements CC of elements BB //AA | /BB/CC//AA | /BB/CC

Even More XPath ExampleEven More XPath Example

Select all sub-elements of elements BB of elements AASelect all sub-elements of elements BB of elements AA /BB/AA/*/BB/AA/* When you do not know the sub-elementsWhen you do not know the sub-elements Different from /BB/AADifferent from /BB/AA

Select all attributes named ‘aa’Select all attributes named ‘aa’ //@aa//@aa

Select all CITIES elements with an attribute named aaSelect all CITIES elements with an attribute named aa //CITIES[@aa]//CITIES[@aa]

Select all CITIES elements with an attribute named aa with value Select all CITIES elements with an attribute named aa with value ‘123’‘123’ //CITIES[@aa = ‘123’]//CITIES[@aa = ‘123’]