Luisa FranconiIntegration, Quality, Research and Production Networks Development DepartmentUnit on microdata accessISTAT

Essnet on Common Tools and Harmonised Methodology for SDC in the ESS

ESSnet workshop, Rome, 3-4 December 2012

Essnet Workshop Rome, 3-4 December 2012

REASONS for an Essnet on SDC harmonisation

– EU regulations made to harmonise surveys

– When releasing microdata different MSs have different requirements/strategies: jeopardize utility

– Allow some flexibility in the choice of SDC methods while maintaining utility

– AIMS– Produce microdata from multiple countries useful for

the final users: achieve COMPARABILITY

– Compare strategies for tabular data protection and include the EU dimension in the process

– Study sustainability of software at EU level

Essnet Workshop Rome, 3-4 December 2012

Essnet on SDC harmonisation “ESSnet on common tools and harmonised methodology for SDC in the ESS”

– Ended April 2012; Partners:

– CBS, Istat, Destatis, University of Wien, SCB

– WP 1: Harmonisation of microdata release at EU level: case study (Istat)

– WP 2: Case studies on tabular data (Destatis)

– WP 3: Future directions of SDC software tools (Statistics Netherlands)

Essnet Workshop Rome, 3-4 December 2012

WP1- Microdata dissemination frameworkOriginal microdata risk assessment

Apply SDC methods to reduce risk while maintaining some utility comparability

Evaluate utility define benchmarking statistics

Original microdata

Disclosure risk

Utility

SDC methods

Anonimized microdata

Essnet Workshop Rome, 3-4 December 2012

WP1- Microdata dissemination framework The actors and their objectivesIntroduce bounded utility into SDC paradigmby means of BENCHMARKING STATISTICS

Case study: Structure of Earning Survey1. Data producers

a) Maintain what is already published by MS and EUb) Be coherent : relationship amongst variables c) Avoid identification – SDC paradigm

2. Data Users/researchers a) care of how data is used in scientific research

Gender pay gap, Linear models

Different methods can be used by MS BUT maintain the benchmarking statistics

Essnet Workshop Rome, 3-4 December 2012

WP1- Microdata dissemination framework

WP 1: Case Study: Structure of Earning Survey Release of microdata in multiple countries

• Task 1-1: Choice of benchmarking statistics: MS constraints Meeting at LAMAS WG

• Task 1-2: User needs: definition of benchmerking statistics• Task 1-3: Definition of methodologies

– We have modified already proposed anonymisation methods for past SES microdata in order to satisfy the chosen benchmarking statistics

• Task 1-4: Implementation and reporting: different methods implemented routines are available on the web site together with documentation

• Task 1-5: Final report– Application of different methods to microdata from

partners in the project and analysis of the results obtained. Satisfying results were obtained.

– Report on governance structure and workflow of the process, mapped to GSBPM

Essnet Workshop Rome, 3-4 December 2012

WP1- Microdata dissemination framework

Governance structure : organisation of the

implementation process In order to study viability of the proposed framework we

address issue related to:

Architecture of the framework: SDC for microdata mapped to the GSBPM (flows of data and metadata)

We provide an analysis of different options for the organisational architecture

We suggest actors in the ESS and their role the SDC decision process

More info available on the project web site:

http://neon.vb.cbs.nl/casc/..%5Ccasc%5CESSNet2index.htm

Essnet Workshop Rome, 3-4 December 2012

WP2: Case Studies on Tabular Data

– Suggest methodological framework for harmonization of SDC methods for economic data based on cell suppression

– Study alternatives of perturbation based SDC

– Give recommendation for future work

– Census 2011: first evaluation of individual countries

Essnet Workshop Rome, 3-4 December 2012

WP2 Results• Cell suppression: Case StudiesDevelopment of a generic table model for secondary

disclosure risks to enhance practical implementation of cell suppression methodology

• Test report on application of the generic table model for SBS and FATS tables under the respective table release concepts of Destatis, Istat, CBS and SCB

• Conclusion:– Applicable linked table solutions to SBS and FATS – Generic table model useful for communication between

data managers and SDC experts– Long term vision:

• Specialized tools to collect table model information as standard meta-data along the production process

Essnet Workshop Rome, 3-4 December 2012

WP2 ResultsCell suppression: Software Issues of the Case Studies• At the MS level:

– All instances of the case studies can be handled by the now available linked-tables approach of -ARGUS

– SAS wrapper packages for -ARGUS available now at SCB and Destatis

• At the EU level– Explore the complexity of the special EU-MS level table

model• Extension of -ARGUS to support application of the

hypercube method in the context of this table model (No need for the CIF interface to the hypercube method anymore!!!)

• Conclusions for future work:• Implement special facilities available in the Destatis

SAS wrapper package to handle complex table models (in particular the EU-MS table model)

Essnet Workshop Rome, 3-4 December 2012

WP2 ResultsStudy and demonstrate stochastic perturbation methods• Main complexity of tabular SDC: Generating the

table model for secondary disclosure risk– Integration with one task of DwB project

• Develop methodology based on stochastic noise to protect tabular data „on the fly“ without need to set up table models

• Method must be integrated into „standard“ tabulation packages

– Conclusion:• Ideal for harmonizing SDC between EU and MS

– if adopted by all MS, and if IT solutions available• Useable for SDC of EU level tables only? (e.g. if MS

stick to cell suppression)– Not clear. More research needed

• Implementation in -ARGUS recommended

Essnet Workshop Rome, 3-4 December 2012

WP2 Results• Census SDC Workshop in April 2012 Need for knowledge of the SDC of Census

Data No consensus yet, so: Workshop has been organised to learn from

each other 7 invited papers, 4 shorter papers and a panel

discussion Report/conclusions available Several conclusions presented by Eurostat at

the Census Working Group Meeting

Essnet Workshop Rome, 3-4 December 2012

WP 3 - Future directions of SDC software tools

• AIM: sustainability of the toolset

• Results:• Extensive description of the current μ-

ARGUS and τ-ARGUS• Description of sdcTables• A blueprint of the new μ-ARGUS and

τ-ARGUS in Open Source• Analysis of Open Solvers• Description of the SAS interface tool

Essnet Workshop Rome, 3-4 December 2012

WP 3 - Future directions of SDC software tools

• A project is planned to rewrite τ-ARGUS as described in WP3.

• Open Source• Platform Independent• Also Open solvers

• OS Cell-suppression is being developed in Data without Boundaries project.

• OS rounder is an open issue, but current version still available.

Essnet Workshop Rome, 3-4 December 2012

Dissemination

• papers and a poster at NTTS2011

• presentations at the LAMAS WG and SDC Expert group

• papers at UNECE-Eurostat worksession

• papers at PSD2012 – Palermo

• All info at project web page