27-28/10/2011

Overall framework :The ESS instruments - a review

J.M. Museux – Eurostat Methodology and Research Unit jean-marc.museux@ec.europa.eu

27-28 October 2011 ESSnet workshop

Joint strategy : how we produce EU statistics

The development of data warehouses and HUB mode/technology for enabling vertical integration of processes and the development of multipurpose statistics

The development of interoperability of systems through the reuse of IT tools and sharing of resources (SOA, distributed architecture, data and metadata exchange standards (SDMX), metadata repositories, etc.).

The development of metadata systems, methodological and statistical standards and common IT tools aiming at industrialisation of production processes.

Innovation in methods for producing statistics like reuse existing data sources, data matching and linking, mixed modes approach, visualisation and macro validation … aiming at efficiency gains and quality improvements.

Better documentation of processes (process indicators and metadata) as a back bone for monitoring efficiency-quality trade off and quality assurance.

27-28 October 2011 ESSnet workshop

Legal framework : from stove pipe legislation to cross cutting framework legislation

Methodological developments– Best practices – guidelines – rules– Common tools

Modernisation of IT infrastructure (SOA, web services, distributed, use of standardised IT solutions)

Standardisation and industrialisation– Common description and semantic– Common data exchange standard– Common metadata– Common tools

ESS strategy

27-28 October 2011 ESSnet workshop

ESS architectures

International standards

Common developments

/ infrastructures

27-28 October 2011 ESSnet workshop

ESSnets

A collaborative network of several ESS organisations aimed at providing results beneficial to the whole ESS

26 ESSnet projects since 2008 - a portal www.essnet-portal.eu

ESSnet functions: review - coordination - planning

ESSnet mapping according to their contribution to innovation process

27-28 October 2011 ESSnet workshop

ESSnets typology

1. ESSnet research consolidation and incorporation of research knowledge and testing of prototype

tools Ex. SDC I - ISAD

2. ESSnet pole of expertise specialisation in process steps – cover wider range of activities (best practices

identification, guidelines, tool development, dissemination of knowledge-know how and training

Ex SDC II, DI, SAE, Mixed mode approach

3. ESSnet infrastructure facilitation to incorporation of international standards, recommendations, cases

studies, taylorisation, improvement of standards, setting up of common infrastructure

Ex. EGR, SDMX I & II, MEETS Datawarehouse, Geostat, CORA & CORE, Decentralised access to EU micro data

27-28 October 2011 ESSnet workshop

ESSnets typology

1. ESSnet standardisation contribute to knowledge and process standardisation (concept,

tools, guidelines, ….) Ex SDC harmonisation, MEETS Memobust, Preparing

Standardisation

2. ESSnet strategy : support to strategic action like Sponsorship on Standardisation

3. ESSnet piloting domain dependent, testing in common, practical implementation

27-28 October 2011 ESSnet workshop

ESSnet – contribution to a reference architecture

CORE

DI

ESSnet

SDMX

DW

Consistency

MemobustSDC Harm

Stand Prep

DecentralisedAccess

EGR

27-28 October 2011 ESSnet workshop

Contribution to ESS business process

Administrative Data

SDC

SAE

DI

Profiling

Net SILC

ICTLinking

Culture

PHGeostat

27-28 October 2011 ESSnet workshop

Common tool development

Tools developed in common for implementing ESS guidelines

Need for specific governance and development strategy– Open source– Modular – web service – portability (JAVA)– ESS Steering Group – User group

Best practice– Demetra+ :

• Tool for implementing ESS seasonal adjustment guidelines• Demetra+ user group – prototype development• Java toolkit (API, GUI, Data cruncher to be run in batch, library

of core functions, structured objects)• Re-used for Ecotrim, NAPS developments

– Argus+ : sustainability of T-Argus – proof of concept – Metadata and SDMX “suite”

27-28 October 2011 ESSnet workshop

Eurostat modernisation - VIPs

Strategic project for Eurostat and ESS modernisation Joint venture between business area units (relevance) and

horizontal units (methodology, IT) 9 projects (still on going) being launched in 2010, 2011 and 2012

1. Data matching

2. Secure infrastructure for sharing confidential data

3. Data validation (vertical integration)

4. New Eurostat dissemination chain

5. National account production system

6. Geospatial statistics & tools

7. HUB for ICT statistics

8. PILLAR approach for household data collection

9. Administrative sources for social statistics

10. Master in Official statistics

ESTAT process /

infrastructure

ESS focus

27-28 October 2011 ESSnet workshop

Eurostat modernisation - VIPs

Common characteristics– Cross domains development– ESTAT focus - IT or process oriented ESS focus– Prepare for higher (MS- Eurostat) integration

Need to maintain global coherence– Planning – yearly work programme– Spin off of ESS(net) project– ESS relevance - coordination

27-28 October 2011 ESSnet workshop

Geo

VIP, ESSnet – elements of coherence

NAPS

Data Matching

Pillar

CORE

DI

SICON

ESSnet

SDMX

VIP

New Diss

DW

Consistency

Memobust SDC Harm

Stand Prep

Data Validation

DecentralisedAccess

DW

Common tools

Demetra+

Argus+

EGR

Admin

ICT HUB

27-28 October 2011 ESSnet workshop

Strategic level – sponsorship

Sponsorship on standardisation

Sponsorship group : general directors or deputies of NSOs (NL,DE, HU, IT, FR, LV), co chaired by CBS and Eurostat IT director to build mechanism and steer competence on key challenge for the ESS over a relatively short period (2 years)

Sponsorship is expected to – define the scope and the business case for ESS standardisation;

– review the state of the art, identify barriers and proposed remedies

– to set up an action plan and mechanism for standardisation in the ESS

– liaise with external initiatives and actively involve stakeholders.

27-28 October 2011 ESSnet workshop

Strategic level – initial reflection

Standardisation is a process:– Set up, agree on standards

– Maintain and manage coherence

– Support use and implementation of standards

Key elements– Standards (rules, guidelines, handbook, tools) are agreed by consensus

– Mixed (bottom up and top down approach)

– ESSnet and standardisation

• All Essnets contains the seeds for standardisation

• ESSnet lack of focus, ESSnet follow up

• How to make it happen ?

27-28 October 2011 ESSnet workshop

Conclusions

Many initiatives on going – bottom up approach initially

Hard to maintain coherence between different layers, scope

Need for an Enterprise Architecture approach– Assembling the different piece of the jigsaw– Defining ESS common architecture elements (principles,

common target, ….) allowing distributed and coherent developments and overall alignment

– Tool for strategic planning and for driving change