Slide 1Eurostat Directorate B – Statistical methods and tools; dissemination Towards implementation of SDMX – 9/11 January 2007

SDMX Open Data Interchange(SODI)

Towards Implementation of SDMX 9-11 January 2007

Slide 2Eurostat Directorate B – Statistical methods and tools; dissemination Towards implementation of SDMX – 9/11 January 2007

Eurostat and the European Statistical System

• Network of national institutions and agencies• Co-ordinated by Eurostat

– European Commission Directorate General– European Union Statistical Authority

• Tasked with providing a high quality statistical information service

• 1000 data providers to Eurostat• More than 100 committees, working groups and

expert groups

Slide 3Eurostat Directorate B – Statistical methods and tools; dissemination Towards implementation of SDMX – 9/11 January 2007

Harmonisation & methodology coordination

• EU Regulations and Agreements• Code lists, nomenclatures• Reporting tools, services and standards

– National raw data– Aggregated European data

• Latest technologies– Internet, Web services, XML– SDMX

Slide 4Eurostat Directorate B – Statistical methods and tools; dissemination Towards implementation of SDMX – 9/11 January 2007

SODI objectives

• Implement a data sharing approach for the Principal European Economic Indicators (PEEI)– a set of 19 key short term indicators aimed at improving timeliness of

European economics statistics

SDMX is not only an informatics project...

...SODI is Eurostat’s vehicle for driving forward the whole range of technical, organisational, statistical, legal, and political developments involved in implementing SDMX in the European Union

Slide 5Eurostat Directorate B – Statistical methods and tools; dissemination Towards implementation of SDMX – 9/11 January 2007

Data sharing in SODI

NSI

Collection Production Dissemination

Collection DisseminationProduction

shared data

Eurostat

Slide 6Eurostat Directorate B – Statistical methods and tools; dissemination Towards implementation of SDMX – 9/11 January 2007

The SODI Workflow

Receiveddata in

SDMX-ML

Eurostat"Pull"

RequestorReferenceData Base /Shared Data

InternetConnection

to NSIs

Extract(and merge)SDMX-ML

XSL forSDMX-ML

RSS-Feed

Webservice

Dissemination

Slide 7Eurostat Directorate B – Statistical methods and tools; dissemination Towards implementation of SDMX – 9/11 January 2007

The SODI Workflow

Receiveddata in

SDMX-ML

Eurostat"Pull"

RequestorReferenceData Base /Shared Data

InternetConnection

to NSIs

Extract(and merge)SDMX-ML

XSL forSDMX-ML

RSS-Feed

Webservice

Dissemination

http://sodi.istat.it

Slide 8Eurostat Directorate B – Statistical methods and tools; dissemination Towards implementation of SDMX – 9/11 January 2007

The SODI Workflow

Receiveddata in

SDMX-ML

Eurostat"Pull"

RequestorReferenceData Base /Shared Data

InternetConnection

to NSIs

Extract(and merge)SDMX-ML

XSL forSDMX-ML

RSS-Feed

Webservice

Dissemination

<item><title>IND-PROD monthly 1</title><link>http://sodi.istat.it/feeds/sodi_rss2_feed2.xml</link><guid>urn:uuid:11</guid><description>IND-PROD monthly 1</description><pubDate>Tue, 07 Sep 2006 12:16:00 GMT</pubDate>

Slide 9Eurostat Directorate B – Statistical methods and tools; dissemination Towards implementation of SDMX – 9/11 January 2007

The SODI Workflow

Receiveddata in

SDMX-ML

Eurostat"Pull"

RequestorReferenceData Base /Shared Data

InternetConnection

to NSIs

Extract(and merge)SDMX-ML

XSL forSDMX-ML

RSS-Feed

Webservice

Dissemination

…

<Query><query:DataWhere><query:And><query:DataProvider>ISTAT</query:DataProvider><query:Dataflow>STSIND_ORD_M</query:Dataflow><query:Dimension name="FREQUENCY">M</query:Dimension>…<query:Time><query:StartTime>1990-01</query:StartTime><query:EndTime>2006-01</query:EndTime></query:Time>…

Slide 10Eurostat Directorate B – Statistical methods and tools; dissemination Towards implementation of SDMX – 9/11 January 2007

The SODI Workflow

Receiveddata in

SDMX-ML

Eurostat"Pull"

RequestorReferenceData Base /Shared Data

InternetConnection

to NSIs

Extract(and merge)SDMX-ML

XSL forSDMX-ML

RSS-Feed

Webservice

Dissemination

Slide 11Eurostat Directorate B – Statistical methods and tools; dissemination Towards implementation of SDMX – 9/11 January 2007

Reference Metadata for SODI

• To enable end users and metadata producers to access, analyse and reuse statistical metadata originated from multiple sources

• Standardise the set of concepts for collecting, processing, and disseminating statistical metadata

• SDMX protocols for ensuring the interoperability of messages, independently on the respective IT platforms.

• Tasks:1. Architecture analysis; design, development and testing of a MSD

(Metadata Structure Definitions) tool.2. Analysis of metadata transfer from national and international

sources; toolkit for conversion and transfer of reference metadata.3. Loading metadata into Eurostat's database and formulating queries

for extracting the information.4. Module for dispatching SDMX-compliant metadata to Eurostat’s

website.

Slide 12Eurostat Directorate B – Statistical methods and tools; dissemination Towards implementation of SDMX – 9/11 January 2007

SODI Reference Metadata: System Architecture

Slide 13Eurostat Directorate B – Statistical methods and tools; dissemination Towards implementation of SDMX – 9/11 January 2007

SDMX implementation in the European Statistical System

• Implementation strategy including short, medium and long term actions

• Aimed at making SDMX a workable tool for statisticians

• To be developed along the technical and content lines of work

• Mostly financed by IDABC, the European Commission program on interoperability

Slide 14Eurostat Directorate B – Statistical methods and tools; dissemination Towards implementation of SDMX – 9/11 January 2007

Lines of work

• Main tools– SDMX registry– SDMX converter

• Content oriented work– DSD development: agriculture, demography, education,

waste, …– transition from GESMES to SDMX-ML– SDMX reference metadata– harmonisation of code lists

Slide 15Eurostat Directorate B – Statistical methods and tools; dissemination Towards implementation of SDMX – 9/11 January 2007

Conclusions

• Strong commitment of Eurostat to SDMX implementation

• SODI as driving project

• Continuity with the past

• Feedback to SDMX standard development