Well-designed XML Data - dc.uba.ar · Well-designed XML Data Marcelo Arenas and Leonid Libkin...

Well-designed XML Data

Marcelo Arenas and Leonid Libkin

University of Toronto

Outline

Part 1 - Database Normalization from the 1970s and 1980s.

Part 2 - Classical theory revisited: normalizing XML documents.

Part 3 - Classical theory re-done: new justifications for normalization.

Part 1: Classical Normalization

Design: decide how to represent the information in a particular data model.

• Even for simple application domains there is a large number of ways of representing the data of interest.

We have to design the schema of the database.

• Set of relations.

• Set of attributes for each relation.

• Set of data dependencies.

Designing a Database: An Example

Attributes: number, title, section, room.

Data dependency: every course number is associated with only one title.

Relational Schema:

R(number, title, section, room),

number → title

T(number, section, room),

S(number, title), number → title

GOOD alternative:

BAD alternative:

Problems with BAD: Update Anomaly

GB2481Database SystemsCSC434

GB2483Computer OrganizationCSC258

LP2661Computer OrganizationCSC258

roomsectiontitlenumber

Title of CSC258 is changed to Computer Organization I.

LP2661Computer OrganizationCSC258

GB2483Computer Organization I

CSC258

LP2661Computer Organization I

CSC258

Title of CSC258 is changed to Computer Organization I.The instance stores redundant information.

Deletion Anomaly

CSC258

CSC434 is not given in this term.

Deletion Anomaly

CSC258

Deletion Anomaly

CSC258

Additional effect: all the information about CSC434 was deleted.

Insertion Anomaly

CSC258

A new course is created: (CSC336, Numerical Methods)

Insertion Anomaly

CSC258

Insertion Anomaly

CSC258

??Numerical MethodsCSC336

CSC258

A new course is created: (CSC336, Numerical Methods)The instance stores attributes that are not directly related.

Avoiding Update Anomalies

Database SystemsCSC434

Computer Organization

CSC258

titlenumber

GB2481CSC434

GB2483CSC258

GB2582CSC258

LP2661CSC258

roomsectionnumber

Computer Organization

CSC258

titlenumber

GB2481CSC434

GB2483CSC258

GB2582CSC258

LP2661CSC258

roomsectionnumber

Computer Organization I

CSC258

titlenumber

GB2481CSC434

GB2483CSC258

GB2582CSC258

LP2661CSC258

roomsectionnumber

Title of CSC258 is changed to Computer Organization I.CSC434 is not given in this term.

The instance does not store redundant information.

CSC258

titlenumber

GB2481CSC434

GB2483CSC258

GB2582CSC258

LP2661CSC258

roomsectionnumber

CSC258

titlenumber

GB2483CSC258

GB2582CSC258

LP2661CSC258

roomsectionnumber

The title of CSC434 is not removed from the instance.

CSC258

titlenumber

GB2483CSC258

GB2582CSC258

LP2661CSC258

roomsectionnumber

Database SystemsCSC434 Numerical MethodsCSC336

CSC258

titlenumber

GB2483CSC258

GB2582CSC258

LP2661CSC258

roomsectionnumber

A new course is created: (CSC336, Numerical Methods)No information about sections has to be provided.Each relation stores attributes that are directly related.

Normalization Theory

Main idea: a normal form defines a condition that a well designed database should satisfy.

Normal form: syntactic condition on the database schema.• Defined for a class of data dependencies.

Main problems:

• How to test whether a database schema is in a particular normal form.

• How to transform a database schema into an equivalent one satisfying a particular normal form.

Normalization Theory Today

Normalization theory for relational databases was developed in the 70s and 80s.

Why do we need normalization theory today?• New data models have emerged: XML.

• XML documents can contain redundant information.

Redundant information in XML documents:• Can be discovered if the user provides semantic

information.

• Can be eliminated.

Part 2: XML and Normalization

</courses>

</course>

</taken_by>

</student>

<taken_by>

XML Document:

</courses>

</course>

</taken_by>

</student>

<taken_by>

XML Document:

</courses>

</course>

</taken_by>

</student>

<taken_by>

XML Document:

</courses>

</course>

</taken_by>

</student>

<taken_by>

XML Document:

</courses>

</course>

</taken_by>

</student>

<taken_by>

XML Document:

</courses>

</course>

</taken_by>

</student>

<taken_by>

XML Document:

#PCDATA⇒grade

#PCDATA⇒name

name, grade

⇒student

@sno⇒student

student*⇒taken_by

taken_by⇒course

@cno⇒course

course*⇒courses

</courses>

</course>

</taken_by>

</student>

<taken_by>

XML Document:

#PCDATA⇒grade

#PCDATA⇒name

name, grade

⇒student

@sno⇒student

student*⇒taken_by

taken_by⇒course

@cno⇒course

course*⇒courses

</courses>

</course>

XML Document:

#PCDATA⇒grade

#PCDATA⇒name

name, grade

⇒student

@sno⇒student

student*⇒taken_by

taken_by⇒course

@cno⇒course

course*⇒courses

</courses>

</course>

</taken_by>

</student>

<taken_by>

XML Document:

#PCDATA⇒grade

#PCDATA⇒name

name, grade

⇒student

@sno⇒student

student*⇒taken_by

taken_by⇒course

@cno⇒course

course*⇒courses

</courses>

</course>

</taken_by>

</student>

<taken_by>

XML Document:

#PCDATA⇒grade

#PCDATA⇒name

name, grade

⇒student

@sno⇒student

student*⇒taken_by

taken_by⇒course

@cno⇒course

course*⇒courses

</courses>

</course>

</taken_by>

</student>

<taken_by>

XML Document:

#PCDATA⇒grade

#PCDATA⇒name

name, grade

⇒student

@sno⇒student

student*⇒taken_by

taken_by⇒course

@cno⇒course

course*⇒courses

</courses>

</course>

</taken_by>

</student>

<taken_by>

XML Document:

#PCDATA⇒grade

#PCDATA⇒name

name, grade

⇒student

@sno⇒student

student*⇒taken_by

taken_by⇒course

@cno⇒course

course*⇒courses

</courses>

</course>

</taken_by>

</student>

<taken_by>

XML Document:

#PCDATA⇒grade

#PCDATA⇒name

name, grade

⇒student

@sno⇒student

student*⇒taken_by

taken_by⇒course

@cno⇒course

course*⇒courses

XML Databases

D : Σ : Two students with the same @sno value must have the same name.

name, grade

⇒student

@sno⇒student

student*⇒taken_by

taken_by⇒course

@cno⇒course

course*⇒courses

XML Schema: (D, Σ)

Redundancy in XML

courses

coursecourse info

@cno @cno taken_bytaken_by

student student

@snoname gradegrade name@sno

student

name@sno

“st1” “st1” “A+”“B+”

“CSC258” “CSC434”

“Fox”“Fox”

“st1” “Fox”

XML Database Normalization

DTD: Data dependency:

Two students with the same @sno value must have the same name.

name, grade

⇒student

@sno⇒student

student*⇒taken_by

taken_by⇒course

@cno⇒course

course*⇒courses

XML Database Normalization

, info* @sno is the identifier of info elements.

grade⇒student

@sno⇒student

student*⇒taken_by

taken_by

⇒course

@cno⇒course

course*⇒courses

name⇒info

@sno⇒info

Data dependency:

Two students with the same @sno value must have the same name.

A “Non-relational” Example

conf conf

title issueissue

article articlearticle

@yeartitle title @year

“ICDT”

author @yeartitleauthor“1999”

“1999”

“1999”“Dong” “2001”“Jarke”

“2001”

“. . .” “. . .” “. . .”

XNF: XML Normal Form

It eliminates two types of anomalies.

It was defined for XML functional dependencies:

DBLP.conf.@title → DBLP.confDBLP.conf.issue →

DBLP.conf.issue.article.@year

Problems to Address

Functional dependencies for XML.

Normal form for XML documents (XNF).

•Generalizes BCNF.

Algorithm for normalizing XML documents.

•Implication problem for functional dependencies.

Framework: Paths in DTDs

Paths(D): all paths in a DTD Dcourses.course courses.course.@cnocourses.course.student.namecourses.course.student.name.S

EPaths(D): all paths in a DTD D that end with an element, for example, courses.course.

We distinguish three kinds of elements: attributes (@), strings (S) and element types.

FDs are defined by means of a relational representation of XML documents.

Framework: XML Trees

courses

coursecourse

“cs100”@sno name grade @sno name grade

student student

“123” “456”

“Fox” “B+” “Smith” “A”S S S S

Tree Tuples

courses

course

@cno student

“cs100”

t(courses) = v0

t(courses.course) = v1

t(courses.course.@cno) = “cs100”t(courses.course.student) = v2

t(p) = ⊥, for the remaining paths

We consider tuples containing a minimal

amount of ⊥ values

Relational representation: tree tuples - mappings

t : Paths(D) → Vertices ∪ Strings ∪ {⊥}

A tree tuple represents an XML tree:

XML Tree: set of Tree Tuples

courses

coursecourse

student student

“123” “456”

“Fox” “B+” “Smith” “A”S S S S

courses

course

“cs100”

student

@sno name grade

“123”

“Fox” “B+”S S

@sno name grade

student

“456”

“Smith” “A”S S

course

Functional Dependencies for XML

Expressions of the form: X → Y

defined over a DTD D, where X, Y are finitenon-empty subsets of Paths(D).

XML tree T can be tested for satisfaction of X → Y if:

X ∪ Y ⊆ Paths(T) ⊆ Paths(D)

T |= X → Y if for every pair u, v of tree tuples in T:

u.X = v.X and u.X ≠ ⊥ implies u.Y = v.Y22

FD: Examples

University DTD: courses ⇒ course*course ⇒ @cno, student*student ⇒ @sno, name, grade

Two students with the same @sno value must have the same name:

courses.course.student.@sno → courses.course.student.name.S

Every student can have at most one grade in every course:

{ courses.course, courses.course.student.@sno } →

courses.course.student.grade.S

Implication Problem for FD

Given a DTD D and a set of functional dependencies Σ ∪ {ϕ}:

(D, Σ) |- ϕ (implies ϕ) if for any XML tree T conforming to D and satisfying Σ , it is the case that T |= ϕ

(D, Σ)+ = { ϕ | (D, Σ) |- ϕ }

Functional dependency ϕ is trivial if it is implied by the DTD alone.

Checking FD Satisfaction

courses

coursecourse

student

“123” “123”

“Fox” “B+” “Fox” “A+”S S S S

“cs225”

studentv1

courses

course

“cs100”@sno name grade

student

“123”

course

@sno name grade

“123”

“Fox” “A+”S S

“cs225”

student

{ courses.course, courses.course.student.@sno } → courses.course.student.grade.S

Checking FD Satisfaction

courses

course

student

“123” “123”

“Fox” “B+” “Fox” “A+”S S S S

studentv1

courses

course

“cs100”@sno name grade

student

“123”

@sno name grade

“123”

“Fox” “A+”S S

student

{ courses.course, courses.course.student.@sno } → courses.course.student.grade.S

XNF: XML Normal Form

XML specification: a DTD D and a set of functional dependencies Σ.

A Relational DB is in BCNF if for every non-trivial functional dependency X → Y in the specification, X is a key.

(D, Σ) is in XNF if:

For each non-trivial FD X → p.@l or X → p.S in (D, Σ)+, X → p is in (D, Σ)+.

Back to DBLP

DBLP is not in XNF:

DBLP.conf.issue → DBLP.conf.issue.article.@year ∈ (D,Σ)+

DBLP.conf.issue → DBLP.conf.issue.article ∉

(D,Σ)+

Proposed solution is in XNF.26

Normalization Algorithm

The algorithm applies two transformations until theschema is in XNF.

If there is an anomalous FD of the form:

DBLP.conf.issue → DBLP.conf.issue.article.@year

then apply the “DBLP example rule”.

Otherwise: choose a minimal anomalous FD and apply the “University example rule”.

Normalizing XML Documents

Remember:

DBLP.conf.issue[q] → DBLP.conf.issue.article.[p]@year

Normalizing XML Documents

Reasoning About FDs

Part 3: What was Missing? Justification!

What is a good database design?

• Well-known solutions: BCNF, 4NF, …

But what is it that makes a database design good?

• Elimination of update anomalies.

• Existence of algorithms that produce good designs: lossless decomposition, dependency preservation.

Previous work was specific for the relational model.

• Classical problems have to be revisited in the XML context.

Justification of Normal Forms

Problematic to evaluate XML normal forms.

• No XML update language has been standardized.

• No XML query language yet has the same “yardstick” status as relational algebra.

• We do not even know if implication of XML FDs is decidable!

We need a different approach.

• It must be based on some intrinsic characteristics of the data.

• It must be applicable to new data models.

• It must be independent of query/update/constraint issues.

Our approach is based on information theory. 30

Information Theory

Entropy measures the amount of information provided by a certain event.

Assume that an event can have n different outcomes with probabilities p1, …, pn.

Amount of information gained by knowing that event i occurred :Average amount of information gained (entropy) :

Entropy is maximal if each pi = 1/n :

log1pi

∑i = 1

p i log1pi

Entropy and Redundancies

Database schema: R(A,B,C), A → B

Instance I:

Pick a domain properly containing adom(I) :• Probability distribution: P(4) = 0 and P(a) = 1/5, a ≠ 4

• Entropy: log 5 ≈ 2.322

321CBA

Pick a domain properly containing adom(I) : {1, …, 6}

• Probability distribution: P(2) = 1 and P(a) = 0, a ≠ 2

• Entropy: log 1 = 0

{1, …, 6}

Entropy and Normal Forms

Let Σ be a set of FDs over a schema S.

Theorem (S,Σ) is in BCNF if and only if for every instance of (S,Σ) and for every domain properly containing adom(I), each position carries non-zero amount of information (entropy > 0).

A similar result holds for 4NF and MVDs.

This is a clean characterization of BCNF and 4NF, but the measure is not accurate enough ...

Problems with the Measure

The measure cannot distinguish between different types of data dependencies.

It cannot distinguish between different instances of the same schema:

entropy = 0

R(A,B,C), A → B

entropy = 0

Well-designed XML Data - dc.uba.ar · Well-designed XML Data Marcelo Arenas and Leonid Libkin...

Documents

Transcript of Well-designed XML Data - dc.uba.ar · Well-designed XML Data Marcelo Arenas and Leonid Libkin...

PCA18501 -XML & WEBSERVICES - WordPress.com...PCA18501 XML AND WEBSERVICES SRM/MCA Page 1 PCA18501 -XML & WEBSERVICES Unit-I Role of XML- XML language basics- XML revolution- XML technology

Introduction to XML History of XML Advantages of XML Syntax and structure of XML Rules of well formed XML document Components of XML XML Basics.

1 Inhomepages.inf.ed.ac.uk/libkin/papers/tcs-vienna.pdf · Lo w er Bounds for In v arian t Queries in Logics with Coun ting Leonid Libkin a; 1 Limso on W ong b a Dep artment of Computer

Writing Basic Software Applications Using AXI - dc.uba.ar · Lab Workbook Writing Basic Software Applications Using AXI xup@xilinx.com

Introduction to XSLT. What is XML? Design Goals of XML XML Format XML Declaration ElementsAttributes.

XML XML Web Applications 1. XML – XML is not…. 2. Basic XML Concepts 3. Defining XML Data Formats 4. Querying XML Data.

marenas/publications/jacm05.pdf · An Information-Theoretic Approach to Normal Forms for Relational and XML Data MARCELO ARENAS AND LEONID LIBKIN University of Toronto, Toronto, Ontario,

SDPL 20112: XML Basics1 2 Basics of XML and XML documents Survivor's Guide to XML, or XML for Computer Scientists / Dummies 2.1 XML and XML documents 2.2.

XML Introduction. Index Markup Language: SGML, HTML, XML An XML example Why is XML important XML introduction XML applications XML support.

XML Technologies XML Basics What is XML? Why use XML? How to write XML? 1XML Technologies - 2012 - David Raponi.

XML Technologies - KSIholubova/NPRG036/slajdy/XML_schemas.pdf · XML Technologies XML data relational XML DB native XML DB parsing validation SAX DOM DTD XML Schema RELAX NG Schematron

XML EXtensible Markup Language. Agenda Introduction to XML XML Rules XML Elements XML Attributes XML Validation XML Exercises XML Namespaces XML CDATA.

XML eXtensible Markup Language. Topics What is XML An XML example Why is XML important XML introduction XML applications XML support CSEB.

Data Integration and Exchangehomepages.inf.ed.ac.uk/libkin/teach/dataintegr15/rel...L. Libkin 25 Data Integration and Exchange Inclusion constraints •We expect every Title listed

The XML Standard Overview of our XML Standards Motivation: HTML vs XML XML 101: syntax, elements, attributes, DTDs, … XML 201: XML Schema, Namespaces.

XML Schema xml-xsd XML Schema - Bienvenue à TECFAtecfa.unige.ch/guides/te/files/xml-schema.pdf · XML Schema xml-xsd XML © Daniel. K. ... XML Schema - . xml-xsd-1-3 XML © Daniel.

1 Les technologies XML Cours 1.2 : Introduction au langage XML (XML, DTD, XML-Schema) (XML, DTD, XML-Schema) Janvier 2009.

Prepared by...Unit – II: XML Introduction to XML: What is XML, XML verses HTML, XML terminology, XML standards, XML syntax checking, The idea of mark-up, XML Structure, Organizing

Introduction to XML · Introduction to XML XML Document Type Deﬁnition, DTD XML Namespaces XML Schema XML Processors Other XML Standards What Is XML? I XML is ameta-markup languagethat

XML to XML through XML - Eindhoven University of Technology