Trends in Database Development: XML, .NET, WinFSTrends in Database Development: XML, .NET, WinFS

Alexander VaschilloMicrosoftAlexander VaschilloMicrosoft

Database developmentDatabase development

Relational model is working wellBenchmarksSecurityEnhancements

New modelsHierarchical (XML)Object

Ease of useNew uses

WinFS

Relational model is working wellBenchmarksSecurityEnhancements

New modelsHierarchical (XML)Object

Ease of useNew uses

WinFS

New Directions in SQL ServerNew Directions in SQL Server

XMLHierarchical, semi-structured data

Object oriented extensionsNew programming models.NET integration

ServerClient

New applicationsWinFS

XMLHierarchical, semi-structured data

Object oriented extensionsNew programming models.NET integration

ServerClient

New applicationsWinFS

Why new data modelsWhy new data models

Flat relational result is good to print reportsHierarchical result is ideal for Web PagesObject data model is for programming against

DatasetObjectspacesWeb Services

Flat relational result is good to print reportsHierarchical result is ideal for Web PagesObject data model is for programming against

DatasetObjectspacesWeb Services

Why XML?Why XML?

Presentation formatTransport formatPlatform independentText-based formatSchema with dataInternational standard not owned by any one company

Presentation formatTransport formatPlatform independentText-based formatSchema with dataInternational standard not owned by any one company

HTTP Access Via URLHTTP Access Via URLURL Queryhttp://server/vroot?sql=select+*+from+Customers+FOR+XML+Auto&root=root

XML Viewhttp://server/vroot/schema.xsd

/Customer[@ID='ALFKI']?params

Templatehttp://server/vroot/template.xml?params

URL Queryhttp://server/vroot?sql=select+*+from+Customers+FOR+XML+Auto&root=root

XML Viewhttp://server/vroot/schema.xsd

/Customer[@ID='ALFKI']?params

Templatehttp://server/vroot/template.xml?params

Loosely Coupled SystemsLoosely Coupled Systems

Scalable. Many to Many.Changes in Implementation do not break each other

Scalable. Many to Many.Changes in Implementation do not break each other

AppAppLogicLogic DataDataMappingMapping MappingMapping

Application SystemApplication System Data SystemData System

ObjectObject(XML)(XML)

Move data in a standardized format (XML)Move data in a standardized format (XML)

The Two WorldsThe Two Worlds

SQLSQLServerServer

RowSetRowSet

SQLSQL

Relational Relational worldworld

LanguageLanguage

Data Data storagestorage

Data Data outputoutput

XML XML worldworld

XMLXMLFilesFiles

XPathXPathXQueryXQuery

XML/XML/HTMLHTML

XMLXMLViewView

SQLXML SQLXML – –

Bringing Bringing worlds worlds

togethertogether

Three WorldsThree Worlds

SQLSQLServerServer

RowSetRowSet

SQLSQL

Relational Relational worldworld

XML XML worldworld

XMLXMLFilesFiles

XPathXPathXQueryXQuery

XML/XML/HTMLHTML

MapMapMemoryMemory

ObjectObject

C#,C++,C#,C++,VBVB

Object Object worldworld

MapMap

Different kinds of dataDifferent kinds of data

StructuredHighly regular, homogeneous structureRowsets, Comma delimited files

Semi-StructuredHeterogeneous structureSparse Occurrences of dataHTML and XML documents

UnstructuredDocuments/Content

StructuredHighly regular, homogeneous structureRowsets, Comma delimited files

Semi-StructuredHeterogeneous structureSparse Occurrences of dataHTML and XML documents

UnstructuredDocuments/Content

SQLXML From 10,000 FeetSQLXML From 10,000 FeetProvides a rich XML view of relational dataSemi-structured, hierarchical view of flat relational dataTwo-way view: query and updateMultiple access mechanisms (HTTP, ADO, ADO.NET, SOAP)Middle tier and Server sideXML: extensible, platform independent format for your data

Provides a rich XML view of relational dataSemi-structured, hierarchical view of flat relational dataTwo-way view: query and updateMultiple access mechanisms (HTTP, ADO, ADO.NET, SOAP)Middle tier and Server sideXML: extensible, platform independent format for your data

FOR XML QueryFOR XML Query

SQL Language ExtensionSELECT…FROM…WHERE…ORDER BY…FOR XML ( raw |

auto [, ELEMENTS] |nested [,

ELEMENTS] |explicit)

[, XMLData] [, BINARY base64])

SQL Language ExtensionSELECT…FROM…WHERE…ORDER BY…FOR XML ( raw |

auto [, ELEMENTS] |nested [,

ELEMENTS] |explicit)

[, XMLData] [, BINARY base64])

XML ViewsXML Views

Map between relational data and XMLDeclarativeNoninvasive

No changes to legacy data sourcesNo control over DB Server required

XML View is an XML SchemaXSD for SQLXML 2.0 and 3.0MSD for Yukon

Map between relational data and XMLDeclarativeNoninvasive

No changes to legacy data sourcesNo control over DB Server required

XML View is an XML SchemaXSD for SQLXML 2.0 and 3.0MSD for Yukon

XSD Mapping ExampleXSD Mapping Example<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:mapping-schema"> <xsd:element name="Customer" msdata:relation="Customers"> <xsd:complexType> <xsd:sequence> <xsd:element name="Order" msdata:relation="Orders"> <xsd:annotation><xsd:appinfo> <msdata:relationship

parent="Customers" parent-key="CustomerID" child="Orders" child-key="CustomerID" /> </xsd:appinfo></xsd:annotation> <xsd:complexType>

<xsd:attribute name="OrderDate" type="xsd:dateTime"/> </xsd:complexType> </xsd:element> </xsd:sequence> <xsd:attribute name="CustomerID" /> </xsd:complexType> </xsd:element></xsd:schema>

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:mapping-schema"> <xsd:element name="Customer" msdata:relation="Customers"> <xsd:complexType> <xsd:sequence> <xsd:element name="Order" msdata:relation="Orders"> <xsd:annotation><xsd:appinfo> <msdata:relationship

parent="Customers" parent-key="CustomerID" child="Orders" child-key="CustomerID" /> </xsd:appinfo></xsd:annotation> <xsd:complexType>

<xsd:attribute name="OrderDate" type="xsd:dateTime"/> </xsd:complexType> </xsd:element> </xsd:sequence> <xsd:attribute name="CustomerID" /> </xsd:complexType> </xsd:element></xsd:schema>

XPath/XQuery

Use XPath/XQuery to query SQL Database as if it was an XML fileEach query translates into a SQL statementXPath

/Customer/Order[@OrderID=‘10692’]

XQueryFor $i in sql:table('Customers', 'CustomerID') Return <Customer

ID = {$i/@CustomerID} Name = {$i/@ContactName}

/>

XML data typeNative SQL typeUse for column, variable or parameter

CREATE TABLE docs (id INT PRIMARY KEY, xDoc XML NOT NULL)

Store un-typed or typed XML instancesWell-formed and validation checksOptional XML Schema enforcementXML instances stored as LOB (2GB)

Efficient binary representation

XML data typeNative SQL typeUse for column, variable or parameter

CREATE TABLE docs (id INT PRIMARY KEY, xDoc XML NOT NULL)

Store un-typed or typed XML instancesWell-formed and validation checksOptional XML Schema enforcementXML instances stored as LOB (2GB)

Efficient binary representation

Native XML StoreXML Data TypeNative XML StoreXML Data Type

Native XML StoreXML IndexNative XML StoreXML Index

Create XML index on XML columnCREATE XML INDEX idx_1 ON docs (xDoc)

Creates indexes on tags, values & pathsSpeeds up queries

Entire query is optimizedSame industry leading cost based optimizer

Indexes are used as available

Create XML index on XML columnCREATE XML INDEX idx_1 ON docs (xDoc)

Creates indexes on tags, values & pathsSpeeds up queries

Entire query is optimizedSame industry leading cost based optimizer

Indexes are used as available

XML Schema SupportXML Schema Support

XML Schema (W3C standard)Rich mechanism for type definitions and validation constraintsCan be used to constrain XML documents

Benefits of typed data Guarantees shape of dataAllows storage and query optimizations

XML type systemStore XML schemas in system meta-data

XML Schema (W3C standard)Rich mechanism for type definitions and validation constraintsCan be used to constrain XML documents

Benefits of typed data Guarantees shape of dataAllows storage and query optimizations

XML type systemStore XML schemas in system meta-data

XQuery: query XML documents and dataStandards-based: W3C working draft

In document 123, return section heading of section 3 and laterSELECT id, xDoc::query('

for $s in /doc[@id = 123]//sec[@num >= 3]

return <topic>{data($s/heading)}</topic>')

FROM docs

XQuery: query XML documents and dataStandards-based: W3C working draft

In document 123, return section heading of section 3 and laterSELECT id, xDoc::query('

for $s in /doc[@id = 123]//sec[@num >= 3]

return <topic>{data($s/heading)}</topic>')

FROM docs

XML QueryXML Query

Insert, update, and delete XQuery extensionsXML sub-tree modification:

Add or delete XML sub-treesUpdate values

Add a new section after section 1:UPDATE docs SET xDoc::modify('insert<section num=''2''> <heading>Background</heading> </section>after /doc/section[@num=1]')

Insert, update, and delete XQuery extensionsXML sub-tree modification:

Add or delete XML sub-treesUpdate values

Add a new section after section 1:UPDATE docs SET xDoc::modify('insert<section num=''2''> <heading>Background</heading> </section>after /doc/section[@num=1]')

XML Data ModificationXML Data Modification

XML View: Unification ModelXML View: Unification Model

SQL Server “Yukon” XML data typeUse XQuery

Relational columnsUse SQL

XML View hides representationUse XQuery against any data

SQL Server “Yukon” XML data typeUse XQuery

Relational columnsUse SQL

XML View hides representationUse XQuery against any data

CustomerID

ContactName

Street City XML data type

XML ViewXML ViewXQuery andXQuery and

UpdatesUpdatesCustomer Table

Choice of XML TechnologyChoice of XML TechnologyNative XML Technology

Very simple way of storing XML dataXML schema is optionalDocument order is importantQuery and modify XML dataIndex XML data

XML View TechnologyXML-centric programming model over tablesSchema for XML data requiredOrder not importantBulk load XML data; decompose into tables

Native XML TechnologyVery simple way of storing XML dataXML schema is optionalDocument order is importantQuery and modify XML dataIndex XML data

XML View TechnologyXML-centric programming model over tablesSchema for XML data requiredOrder not importantBulk load XML data; decompose into tables

.NET Integration.NET Integration

Server side: SQLCLR.NET hosted inside the databaseWrite stored procedures in C#Use ADO programming model on the server the same way as on the client sideCreate UDTs

Client sideWeb ServicesDatasetObjectspaces

Server side: SQLCLR.NET hosted inside the databaseWrite stored procedures in C#Use ADO programming model on the server the same way as on the client sideCreate UDTs

Client sideWeb ServicesDatasetObjectspaces

SQLCLRSQLCLRComponent reuseMainstream development experienceFamiliar choice of programming languages and constructsLeverage existing libraries and componentsSeamless debugging and deploymentDeep integration with the engine

Component reuseMainstream development experienceFamiliar choice of programming languages and constructsLeverage existing libraries and componentsSeamless debugging and deploymentDeep integration with the engine

SQLCLR DevelopmentSQLCLR Development

VS .NET VS .NET ProjectProject

Assembly: “TaxLib.dll”

VB, C#, …VB, C#, … BuildBuild

SQL ServerSQL Server

SQL Data Definition: SQL Data Definition: create create assembly … assembly … create function … create function … create create procedure … procedure … create trigger … create trigger … create create type …type …

SQL Queries: SQL Queries: select select sum(sum(tax(sal,state)tax(sal,state) ) ) from Emp where county from Emp where county = ‘King’= ‘King’

Runtime hosted inside

SQL

SQL or SQLCLRSQL or SQLCLR

Why SQLSet-oriented queries Large data sets

Why CLRComputationally intensiveComplex behaviorsReusable components with rich behaviorsRich types (polygon)

Why SQLSet-oriented queries Large data sets

Why CLRComputationally intensiveComplex behaviorsReusable components with rich behaviorsRich types (polygon)

Design GuidelinesDesign GuidelinesT-SQL is best suited for data access

Relational programming modelStatic compilation modelOptimized for data access

SLQCLR is for procedural programming and computation

IL compiled to x86 code at runtime, easily outperforms interpreted T-SQLCompute-intensive business logic encapsulated as functions

Moving computation to where the data isData shipping cost goes awayServer CPU now used for user processing

T-SQL is best suited for data accessRelational programming modelStatic compilation modelOptimized for data access

SLQCLR is for procedural programming and computation

IL compiled to x86 code at runtime, easily outperforms interpreted T-SQLCompute-intensive business logic encapsulated as functions

Moving computation to where the data isData shipping cost goes awayServer CPU now used for user processing

Web Services OverviewWeb Services Overview

Natural client side programming modelTurn your existing Stored Procedures into web ServicesMessaging done according to SOAP 1.1 standardChoose how to model results

XMLObjectsDataset

Can run on database server or mid-tierIntegrated with Visual Studio

Natural client side programming modelTurn your existing Stored Procedures into web ServicesMessaging done according to SOAP 1.1 standardChoose how to model results

XMLObjectsDataset

Can run on database server or mid-tierIntegrated with Visual Studio

SOAP And Web ServicesSOAP And Web Services

WSDL file describing each template and stored proc exposedTool to choose which templates and stored procedures to expose

WSDL file describing each template and stored proc exposedTool to choose which templates and stored procedures to expose

SQLSQLServerServer

IIS/ISAPIIIS/ISAPI

MessageMessageSPSP

TemplateTemplate

ClientClient SOAPSOAP

MessageMessage

WSDLWSDL

Easy Programming ModelEasy Programming Model

SQLXML generates WSDL automaticallyVisual Studio.NET recognizes a DatasetRetrieve results of a Stored Procedure and load into a Dataset in 1 line of code!

Dim Service As New MyHost.MyWebService()Dim retval As IntegerDataSet ds = Service.GetCustomer(Name)

SQLXML generates WSDL automaticallyVisual Studio.NET recognizes a DatasetRetrieve results of a Stored Procedure and load into a Dataset in 1 line of code!

Dim Service As New MyHost.MyWebService()Dim retval As IntegerDataSet ds = Service.GetCustomer(Name)

Web Services – Decoupled ArchitectureWeb Services – Decoupled Architecture

SQL ServerSQL ServerSQLXMLSQLXMLApplicationApplicationMethod callMethod call SQL querySQL query

RowsetRowsetXmlReaderXmlReader

ClientClient Mid-TierMid-Tier ServerServer

User User viewview

DBA DBA viewview

Levels of AbstractionLevels of Abstraction

Abstract the data source – XML ViewAbstract the data access – HTTP queriesAbstract programming model – SQL Server Web Services

Abstract the data source – XML ViewAbstract the data access – HTTP queriesAbstract programming model – SQL Server Web Services

Data Model TransparencyData Model Transparency

XML Views – treat your relational data as if it was XML FileUse XML Query LanguagesPerform XML UpdatesNo need to be a DBA, learn SQL, or database programming APIs/logic

XML Views – treat your relational data as if it was XML FileUse XML Query LanguagesPerform XML UpdatesNo need to be a DBA, learn SQL, or database programming APIs/logic

Data Access TransparencyData Access Transparency

Access your data from any platformHTTP queries - platform independent protocolXML results – standard representation of dataUse SQL or XPath to query

Access your data from any platformHTTP queries - platform independent protocolXML results – standard representation of dataUse SQL or XPath to query

Programming Model TransparencyProgramming Model Transparency

Web servicesUse from any platformCall methods – get XML data returned

SQL Server stored procedure or XML Template is calledResults are transformed into XML form as needed

SQLCLR: programming model is the same on the server and on the client

Loosely coupled architecture

Web servicesUse from any platformCall methods – get XML data returned

SQL Server stored procedure or XML Template is calledResults are transformed into XML form as needed

SQLCLR: programming model is the same on the server and on the client

Loosely coupled architecture

WinFS: Structured Data StorageWinFS: Structured Data Storage

Files vs. DatabasesNTFS

Part of Operating SystemBackupWin32 APIsSimple

DatabaseOptimized for queryingReliabilitySecurityTransactions, multi-user, concurrency

Files vs. DatabasesNTFS

Part of Operating SystemBackupWin32 APIsSimple

DatabaseOptimized for queryingReliabilitySecurityTransactions, multi-user, concurrency

WinFSWinFS

System FilesExeDllSwap…

User FilesDocumentsPicturesMessages…

System FilesExeDllSwap…

User FilesDocumentsPicturesMessages…

User FilesUser FilesUnstructured data

Not really unstructured – proprietary structure

Data broken into filesOne level of granularity (HTML, Powerpoint)Easy manipulation?

Proprietary formatsNeed particular application to interpret filesNo Sharing (Import/Export)No relationships

Duplication of DataCompatibility of data (Emails, Contacts,…)

Unstructured dataNot really unstructured – proprietary structure

Data broken into filesOne level of granularity (HTML, Powerpoint)Easy manipulation?

Proprietary formatsNeed particular application to interpret filesNo Sharing (Import/Export)No relationships

Duplication of DataCompatibility of data (Emails, Contacts,…)

WinFSWinFSDatabase

Reliability, Concurrency, Speed, query optimization

Understanding schemasUniform SearchNew APIs

SQLObjects

Old APIsWill be supported

Old files still workWant to enable richer integration – provide translations mechanisms

DatabaseReliability, Concurrency, Speed, query optimization

Understanding schemasUniform SearchNew APIs

SQLObjects

Old APIsWill be supported

Old files still workWant to enable richer integration – provide translations mechanisms

WinFS SchemasWinFS Schemas

Unification on some levelBase schemas shipped with Windows

Play by the rules – all applications will be enabled with your data

Use extensions for your proprietary data

Convenient programming modelShell supports librariesNavigation (relationships)Integration (Email body is a document)

Unification on some levelBase schemas shipped with Windows

Play by the rules – all applications will be enabled with your data

Use extensions for your proprietary data

Convenient programming modelShell supports librariesNavigation (relationships)Integration (Email body is a document)

WinFS Data ModelWinFS Data ModelItems

Person, Document, Message, Meeting, etc.

RelationshipsAuthor, Attachment, Meeting participant

Nested typesAddress

ExtensionsProprietary dataMultityping

Inheritance

ItemsPerson, Document, Message, Meeting, etc.

RelationshipsAuthor, Attachment, Meeting participant

Nested typesAddress

ExtensionsProprietary dataMultityping

Inheritance

Audio Videos Images Games. . .

Principals LocationsCalendar EventsCoreMessage (Email)

The Windows SchemasThe Windows Schemas

WinFSTypesMeta BaseFile SyncShellSubscriptions. . .

System Tasks Explorer Config NaturalUI Programs

Services SecurityHelp Device. . .

System

User DataDocumentsAnnotationsMedia NotesPerson Tasks

Infrastructure

My favorite queryMy favorite query

What do I know about “John Smith”Documents by/about himEmails from himHis addressPhone calls from himAnnotations he added to my papersMeetings with him

What do I know about “John Smith”Documents by/about himEmails from himHis addressPhone calls from himAnnotations he added to my papersMeetings with him

Creating API for a SchemaCreating API for a SchemaCreate WinFS schema in XML formatSchema compiler generates API assemblyYou can add your own “helper” membersThe assemblies are installed into a WinFS store

WinFS types are registered as UDTs

Views and other database objects are created

Create WinFS schema in XML formatSchema compiler generates API assemblyYou can add your own “helper” membersThe assemblies are installed into a WinFS store

WinFS types are registered as UDTs

Views and other database objects are created

WinFSSchema

CLR Complier

Code for Helper

Members

WinFSSchemaCompiler

Code for Standard

APIAPI

Classes

WinFS API ExampleWinFS API Example

using (ItemContext ic = new ItemContext())

{ic.Open();

Contact c = (Contact) ic.FindItem( typeof(System.Storage.Contact.Person),

“DisplayName == ‘Bob Smith’”);

c.DisplayName = ‘Robert Smith’;c.BirthDate = ‘01/04/1982’;ic.Update();

}

using (ItemContext ic = new ItemContext())

{ic.Open();

Contact c = (Contact) ic.FindItem( typeof(System.Storage.Contact.Person),

“DisplayName == ‘Bob Smith’”);

c.DisplayName = ‘Robert Smith’;c.BirthDate = ‘01/04/1982’;ic.Update();

}

WinFS foldersWinFS folders

Every Item must be in at least one folderItem organizationLifetime managementOne file can be in multiple folders (reference counting)User can add custom fields to folders

Every Item must be in at least one folderItem organizationLifetime managementOne file can be in multiple folders (reference counting)User can add custom fields to folders

Body- Preference

WinFS Message Schema (Example)WinFS Message Schema (Example)

Message-Subject-Time sent-Type-Status

Contact-Name-Address-Email-Photo

Document-Title-Size-Type-

Document-Title-Size-Type-

Account-Name-Quota-Type-Server

Participant-DisplayName-Type-Address

Component

Database IntegrationDatabase Integration

XMLObject storageProgramming modelDevelopment environmentWebFile systemApplications

XMLObject storageProgramming modelDevelopment environmentWebFile systemApplications