1 CSL Workshop, October 13-14, 2005 From Model to Data Transformation – One approach Mapping...

1CSL Workshop, October 13-14, 2005

From Model to Data Transformation – From Model to Data Transformation – One approach Mapping One approach Mapping

TeleAtlas Data to EuroRoadSTeleAtlas Data to EuroRoadS

Anders Friis-Christensen, JRC


From Model to Data TransformationFrom Model to Data Transformation

Outline:

• Model– EuroRoadS subset

• XMI/GML– Visio -> XMI -> Rational / EA ->– XMI -> GML schema– TeleAtlas Road data (shape) -> GML (+schema)

• Model transformation– EuroRoads GML <-> TeleAtlas GML

• XSL• Data Transformation

– XSLT– Population of data

• Verification of data


Creating EuroRoadS subsetCreating EuroRoadS subset

• A subset of the network exchange model

• The network model is a common base for roads

• Including:– Attributes (modeled as abstract classes)– Geometry (GML data types)

• No topology

• No routing info


ER_IdentifiableObject

id : ER_ObjectId

(from euroRoads)

ER_RoadFeature(from euroRoads)

ER_ObjectId

permanentId : CharacterString...versionId : CharacterString...alternateId : CharacterString...

(from euroRoads)

<<DataType>>

ER_FerryLink

level : ER_FerryLinkLevel...formOfFerry : ER_FormOfFerry...

(from euroRoads)

ER_RoadLink

level : ER_RoadLinkLevel...formOfWay : ER_FormOfWay...nationalRoadClass : ER_NationaRoadClass...

(from euroRoads)

ER_ComplexRoadnetElement

formOfComplexElement : ER_FormOfComplexElement...

(from euroRoads)

ER_FerryLinkAttribute(from euroRoads)

ER_RoadNodeAttribute(from euroRoads)

ER_RoadLinkAttribute(from euroRoads)

ER_RoadnetElementAttribute(from euroRoads)

ER_ComplexRoadnetElementAttribute(from euroRoads)

ER_LinkAttribute(from euroRoads)

ER_FormOfComplexElement

ER_Interchange : CharacterStringER_Roundabout : CharacterStringER_AggregatedWay : CharacterString

(from euroRoads)

<<CodeList>>

ER_FormOfFerry

ER_ShipOrHovercraft : CharacterString...ER_Train : CharacterString...

(from euroRoads)

<<CodeList>>

ER_FormOfWay

ER_Motorway : CharacterString...ER_MultiCarriageway : CharacterString...ER_SingleCarriageway : CharacterString...ER_RoundaboutCirc le : CharacterString...ER_TrafficSquare : CharacterString...ER_EnclosedTrafficArea : CharacterString...ER_SlipRoad : CharacterString...ER_ServiceRoad : CharacterString...ER_EntranceOrExitCarPark : CharacterString...ER_EntranceOrExitServ ice : CharacterString...ER_UndefinedFormOfWay : CharacterString...

(from euroRoads)

<<CodeList>>

ER_NationaRoadClass

MainRoadS : CharacterStringFirstClass : CharacterStringSecondClass : CharacterStringThirdClass : CharacterStringFourthClass : CharacterStringSixthClass : CharacterStringSeventhClass : CharacterStringEighthClass : CharacterStringNinthClass : CharacterStringUndefinedNationalRoadClass : CharacterString

(from euroRoads)

<<Enumeration>> ER_FormOfNode

ER_Roundabout : CharacterString...ER_EnclosedTrafficArea : CharacterString...ER_PseudoNode : CharacterString...ER_GradeSeparatedCrossing : CharacterString...

(from euroRoads)

<<CodeList>>

ER_RoadLinkLevel

ER_RoadElement : CharacterString...ER_Road : CharacterString...

(from euroRoads)

<<CodeList>>

ER_RoadNodeLevel

ER_Junction : CharacterString...ER_Intersection : CharacterString...

(from euroRoads)

<<CodeList>>ER_FerryLinkLevel

ER_FerryConnection : CharacterString...ER_Ferry : CharacterString...

(from euroRoads)

<<CodeList>>

ER_SpeedLimit

speedLimit : Integer

(from euroRoads)

ER_RoadnetLink

validityFrom : DatevalidityTo : Date

(from euroRoads)GM_Curve

(from euroRoads)1 0..11

+curve

0..1GM_Point

(from euroRoads)

ER_RoadNode

level : ER_RoadNodeLevel...formOfNode : ER_FormOfNode...location : GM_Point

(from euroRoads)0..1 1

+point

0..1 1

StringAttribute

value : CharacterString...

(from euroRoads)

ER_RoadAttribute


(from euroRoads)ER_RoadnetElement

(from euroRoads)

0..n 0..1

+attributes

0..n 0..1


XMI (ver.1.0) UML1.1

transformation

EuroRoadsS model (Visio2003)

Clean up XMI

Load XMI in ArgoUML

Export XMI from ArgoUML

Import XMI in Rational Rose/

Enterprise Architect

Correct and modify model

Export to XMI 1.0 (from RR)

Apply ShaoeChange to

produce GML 2.1.2 or 3.1

GML ver.

Road data (TeleAtlas)

GML EuroRoads schema without ISO 19108 types

FME GML conversion

XSL: TeleAtlas - EuroRoads

Apply XSLT on data

Road data (TeleAtlas -> EuroRoads

schema)

2.1.2

? tools support ?

3.1

FME GML TeleAtlas schema

Model interchangeData interchange

GML EuroRoads schema


XMI Generation from VisioXMI Generation from Visio

• Use of Microsoft XMIExport.dll– Invoked by ESRI add-on or by Lars Vikstroms application

• Produces non-valid XMI supposedly based on UML 1.1– Several errors:

• Newline in empty tags, more than 400 errors (only in XML spy 2004, error in parser)

• Undefined references (associations ends and generalization)– Must be an error in the model, old references not deleted properly by

Visio

Newline in empty tag <Foundation.Core.Classifier.feature></Foundation.Core.Classifier.feature> not allowed. Changed 198 places.

Newline in empty tag <Foundation.Core.AssociationEnd.qualifier></Foundation.Core.AssociationEnd.qualifier> not allowed. Changed 192 places.

Newline in empty tag <Foundation.Core.Namespace.ownedElement></Foundation.Core.Namespace.ownedElement> not allowed. Changed 4 places.

Undefined references:

line 5901: <Foundation.Core.Classifier.associationEnd><Foundation.Core.AssociationEnd xmi.idref="UID92E0A85B-FFCA-4D3E-AFA3-8A94E5121744"/> <-<Foundation.Core.AssociationEnd xmi.idref="UIDFA3023F9-5631-4AF5-85C0-761190195B8C"/> <-</Foundation.Core.Classifier.associationEnd> deleted

line 6054:<Foundation.Core.Classifier.associationEnd><Foundation.Core.AssociationEnd xmi.idref="UID958794C3-5D75-4A43-89E3-F0B6E1546E5B"/><Foundation.Core.AssociationEnd xmi.idref="UID59BE8B25-EFB3-4BEF-8811-B438964F63A3"/> <-<Foundation.Core.AssociationEnd xmi.idref="UID657C99EC-7244-47BD-B84B-BBEEF3A69330"/> <-</Foundation.Core.Classifier.associationEnd>deleted

line 12659:<Foundation.Core.GeneralizableElement.specialization><Foundation.Core.Generalization xmi.idref="UID496EBF06-CABE-46F1-87C0-F66CDEAD17D8"/> <-<Foundation.Core.Generalization xmi.idref="UID7EA823B3-EB2F-4160-B6C8-F8518860E7F0"/></Foundation.Core.GeneralizableElement.specialization>deleted


XMI Import (Based on Visio export) XMI Import (Based on Visio export)

• Rational Rose– Does not read UML 1.1 XMI – Possible workarounds??

• Enterprise Architect– Based on OMG UML 1.1 XMI 1.0 dtd, success on import– However, several errors. E.g.:

• Generalizations not imported• Enumerations not imported• Association multiplicity not imported


XMI Import (Based on Visio export)XMI Import (Based on Visio export)

• ArgoUML 0.18:– Possible to import XMI– Errors:

• Multiplicities not imported

– Export UML 1.3 XMI 1.0 (without multiplicities)

• Rational Rose:– Import new XMI possible– No import of enumerations

• Enterprise Architect:– Same as RR

As a curiosity: ArgoUML cannot read its own import if it is saved as a project. The XMI parsing fails.


XMI – IssuesXMI – Issues

• Conclusion:– Microsoft Visio is not interoperable (choose another software for UML, if it is

foreseen that exchange of models is required)– Various vendors of XML editors/importers have errors in their parsers which

complicates the process of validating XMI– Tools do not support same versions of XMI– Tools interpret XMI in different ways– It is very difficult to validate the content of the XMI, i.e. if the model imported is

correct even though the XMI is validated.

• Report John Herring (communicated by Morten)– EA problem with RR XMI:

• It does not import data types <<DataType>> (see also the example EA diagram that was send to you

• It does not see realization ("inheritance" from interfaces and types) in the same way as Rose. It transforms those into a type of dependency



transformation


Clean up XMI

Load XMI in ArgoUML








GML ver.



FME GML conversion


Apply XSLT on data


schema)

2.1.2

? tools support ?

3.1





Prepare Sub-set Model in RR to support GMLPrepare Sub-set Model in RR to support GML

• Creation of GML schemas is done by ShapeChange tool (Interactive Instruments):– Converts XMI 1.0 to GML 2.1.2 and 3.1 (3.1.1?)– Other XMI than export from RR??

• Experiments from EA (Clemens)

• ShapeChange put different requirements on the UML model (and hence the resulting XMI)




id : ER_ObjectId

(from euroRoads)



formOfComplexElement : ER_FormOfComplexElement

(from euroRoads)







ER_SpeedLimit


(from euroRoads)

ER_RoadAttribute



(from euroRoads)0..n 0..1

+attributes

0..n

+roadnetElement

0..1

ER_RoadnetLink



(from euroRoads)

1 0..11

+curve

0..1GM_Point

(from euroRoads)

ER_RoadNode

level : ER_RoadNodeLevelformOfNode : ER_FormOfNodelocation : GM_Point


+point

0..1 1

StringAttribute

value : CharacterString

(from euroRoads)

ER_FerryLink

level : ER_FerryLinkLevelformOfFerry : ER_FormOfFerry

(from euroRoads)

ER_RoadLink

level : ER_RoadLinkLevelformOfWay : ER_FormOfWaynationalRoadClass : ER_NationaRoadClass

(from euroRoads)

Multiple inheritance:ER_RoadnetElement inherited directly from both ER_RoadFeature and ER_IdentifiableObject




id : ER_ObjectId

(from euroRoads)




(from euroRoads)







ER_SpeedLimit


(from euroRoads)

ER_RoadAttribute




+attributes

0..n

+roadnetElement

0..1

ER_RoadnetLink



(from euroRoads)

1 0..11

+curve

0..1GM_Point

(from euroRoads)

ER_RoadNode



+point

0..1 1

StringAttribute


(from euroRoads)

ER_FerryLink


(from euroRoads)

ER_RoadLink


(from euroRoads)

Two associations between ER_RoadAttribute and ER_RoadnetElement




id : ER_ObjectId

(from euroRoads)




(from euroRoads)







ER_SpeedLimit


(from euroRoads)

ER_RoadAttribute




+attributes

0..n

+roadnetElement

0..1

ER_RoadnetLink



(from euroRoads)

1 0..11

+curve

0..1GM_Point

(from euroRoads)

ER_RoadNode



+point

0..1 1

StringAttribute


(from euroRoads)

ER_FerryLink


(from euroRoads)

ER_RoadLink


(from euroRoads)

Added multiplicities toassociations and made them bidirectional




id : ER_ObjectId

(from euroRoads)




(from euroRoads)







ER_SpeedLimit


(from euroRoads)

ER_RoadAttribute




+attributes

0..n

+roadnetElement

0..1

ER_RoadnetLink



(from euroRoads)

1 0..11

+curve

0..1GM_Point

(from euroRoads)

ER_RoadNode



+point

0..1 1

StringAttribute


(from euroRoads)

ER_FerryLink


(from euroRoads)

ER_RoadLink


(from euroRoads)

Added a concrete class with a value of string datatype in order to store data




id : ER_ObjectId

(from euroRoads)




(from euroRoads)







ER_SpeedLimit


(from euroRoads)

ER_RoadAttribute




+attributes

0..n

+roadnetElement

0..1

ER_RoadnetLink



(from euroRoads)

1 0..11

+curve

0..1GM_Point

(from euroRoads)

ER_RoadNode



+point

0..1 1

StringAttribute


(from euroRoads)

ER_FerryLink


(from euroRoads)

ER_RoadLink


(from euroRoads)

Added a validityFrom and validityToInstead of TM_Period in order of simplicity and to support GML 2.1.2


Other UGAS requirementsOther UGAS requirements

• Additionally following rules needed to be applied the model– Model must be represented in a package

stereotyped with <<Application Schema>>– The XML namespace (a URI) and an abbreviation

must be present as tagged values

Resulting GMLResulting GML<?xml version="1.0" encoding="UTF-8"?><schema xmlns="http://www.w3.org/2001/XMLSchema" xmlns:cslWorkshop="http://www.jrc.it/inspire/cslWorkshop" xmlns:gml="http://www.opengis.net/gml" xmlns:sc="http://www.interactive-instruments.de/ShapeChange" xmlns:scXML="http://www.isotc211.org/scXML" xmlns:smXML="http://www.isotc211.org/smXML" xmlns:xlink="http://www.w3.org/1999/xlink" targetNamespace="http://www.jrc.it/inspire/cslWorkshop" elementFormDefault="qualified" version="0.1"> <import namespace="http://www.opengis.net/gml" schemaLocation="feature.xsd"/>  <element name="FeatureCollection" type="cslWorkshop:FeatureCollectionType" substitutionGroup="gml:_FeatureCollection"/> <complexType name="FeatureCollectionType"> <complexContent> <extension base="gml:AbstractFeatureCollectionType"/> </complexContent> </complexType>  <complexType name="ER_RoadnetLinkType" abstract="true"> <annotation> <documentation>Note: The attribute TM_Period in the original EuroRoadS model has been replaced by validityFrom and validityTo for simplicity reasons</documentation> </annotation> <complexContent> <extension base="cslWorkshop:ER_RoadnetElementType"> <sequence> <element name="validityFrom" type="date" minOccurs="0"/> <element name="validityTo" type="date" minOccurs="0"/> <element name="curve" type="gml:LineStringPropertyType" minOccurs="0"/> </sequence> </extension> </complexContent> </complexType> <complexType name="ER_RoadnetLinkPropertyType"> <sequence> <element ref="cslWorkshop:ER_RoadnetLink" minOccurs="0"/> </sequence> <attributeGroup ref="gml:AssociationAttributeGroup"/> </complexType> <complexType name="ER_RoadnetLinkPropertyByValueType"> <sequence> <element ref="cslWorkshop:ER_RoadnetLink"/> </sequence> </complexType> <element name="ER_RoadnetLink" type="cslWorkshop:ER_RoadnetLinkType" abstract="true" substitutionGroup="cslWorkshop:ER_RoadnetElement"/> <complexType name="ER_IdentifiableObjectType" abstract="true"> <complexContent>

<element name="FeatureCollection" type="cslWorkshop:FeatureCollectionType"

substitutionGroup="gml:_FeatureCollection"/><complexType name="FeatureCollectionType"> <complexContent> <extension base="gml:AbstractFeatureCollectionType"/> </complexContent></complexType>


Resulting GMLResulting GML

• Because of the many classes in the model, the GML is complex

• I did not work further with GML 3.1 schemas– Complex model – 3.1 schemas not valid (according several parsers),

experience in producing 3.1.1?– Tools support (3.1 and 3.1.1)?– Any experiences??



transformation


Clean up XMI

Load XMI in ArgoUML








GML ver.



FME GML conversion


Apply XSLT on data


schema)

2.1.2

? tools support ?

3.1





Data transformationData transformation

• Produce GML from TeleAtlas data– FME GML– Create schema option– No support for 3.1 or 3.1.1??

• TeleAtlas based on GDF– Should be no problems when mapping attributes– Except that the attributes are named: ID, FEATTYP, FT, F_JNCTID,

F_JNCTTYP, T_JNCTID, T_JNCTTYP, PJ, METERS, FRC, NETCLASS, NETBCLASS, NET2CLASS, NAME, NAMELC, SOL, NAMETYP, CHARGE, ROUTENUM, RTETYP, RTEDIR, RTEDIRVD, PROCSTAT, FOW, SLIPRD, FREEWAY, BACKRD, TOLLRD, RDCOND, STUBBLE, PRIVATERD, CONSTATUS, ONEWAY, F_BP, T_BP, F_ELEV, T_ELEV, KPH, MINUTES, POSACCUR, CARRIAGE, LANES


XSLXSL


id : ER_ObjectId

(from euroRoads)




(from euroRoads)







ER_SpeedLimit


(from euroRoads)

ER_RoadAttribute




+attributes

0..n

+roadnetElement

0..1

ER_RoadnetLink



(from euroRoads)

1 0..11

+curve

0..1GM_Point

(from euroRoads)

ER_RoadNode



+point

0..1 1

StringAttribute


(from euroRoads)

ER_FerryLink


(from euroRoads)

ER_RoadLink


(from euroRoads)

We only map data to ER_RoadLink

FeatureCollection properties (e.g.

bbox) are mapped

Name (TA) mapped to

StringAttribute

KPH (TA) mapped to speedLimit

lineStringProperty (TA) mapped to

GM_CurveAlways ER_RoadElement

FOW (TA) mapped to FormOfWay using lookup

table

FRC (TA) mapped to NationalRoadClass using lookup table

ER_ObjectId

permanentId : CharacterStringversionId : CharacterStringalternateId : CharacterString

(from euroRoads)

<<DataType>>

FID (TA) mapped to ID:permanentID (should

actually have been ID (TA) as FID is FME

generated)


Result in GaiaResult in Gaia


Resulting GMLResulting GML

• Validates to EuroRoadS submodel GML schema

• Complex attribute types not possible to import via FME or ArcGIS– Requires flat structure

• After short! trial, no success in publishing via GeoServer WFS


Approach(1)Approach(1)

• Pragmatic solution• Feasibility of approach

– Does it satisfy requirements: • It is automatic. • Creating XSL files can be complicated, especially if the semantics of

attributes to be transformed vary too much• Time-consuming (costly)• XSLT is a standardized methodology, GML is supported by the important

software producers • We shall later see similar approach in a service architecture (Snowflake,

Finnish Geodetic Ins. etc..)

• Would best support:

• s

Common conceptual model

Local conceptual model

Logical/physical model

Local conceptual model

Logical/physical model

Derivation/specialization

Implementation

- Automatically (preferably no manual intervention)- Easy set-up- Minimum cost- Support from vendors/technologies- Support a SOA


ApproachApproach

• Requires that the two models that need to be mapped do not differentiate too much:– Attribute types, feature types, etc.– No (immediate) generalization mechanisms

• i.e., the approach is scale dependent– Aggregation (attributes, features) is complicated

• Requires in-depth knowledge of both models– From domain to low level spatio-temporal aspects of models

• How to test that a mapping occurs error free– Validation of data?– Constraints?

• Other possibilities:– Materialized views and stored procedures for mapping

• Create DB schema based on ‘complex’ UML or GML schemas?

1 CSL Workshop, October 13-14, 2005 From Model to Data Transformation – One approach Mapping...

Documents

Transcript of 1 CSL Workshop, October 13-14, 2005 From Model to Data Transformation – One approach Mapping...