CORE Final Meeting – 11 January 2012 1

CORE Demo Scenario

Diego Zardetto,Istat & CBS CORE team

CORE Final Meeting – 11 January 2012 2

Demo Scenario

• Involves 3 typical processing steps

performed by NSIs for sample surveys:

Sample Allocation

Sample Selection

Estimation

• It has been used as empirical test-bed

during the whole implementation cycle

of the CORE environment

CORE Final Meeting – 11 January 2012 3

Rationale for the Scenario

• Minimality: very easy workflow (no conditionals, nor cycles), can be run without a Workflow Engine

• Appropriateness: addresses

heterogeneity issues

heterogeneity is precisely what CORE

must be able to get rid of

CORE Final Meeting – 11 January 2012 4

Spreading Heterogeneity over the Scenario

• The Scenario incorporates both:

Data Heterogeneity: Via data exchanged

by CORE services belonging to the scenario

process

Technological Heterogeneity: Via IT tools

implementing scenario services

– A batch job based on a SAS script

– Two full-fledged R-based systems

CORE Final Meeting – 11 January 2012 5

The Scenario at a

glanceSTART

MAUSS-R

AL

LO

CA

TIO

N

SAS SCRIPT

SELECTION

STOP

ReGeneseesSystem

ESTIMATION

CORE Final Meeting – 11 January 2012 6

Sample Allocation

Service• Overall Goal: determine the

minimum number of units to be sampled inside each stratum, when lower bounds are imposed on the expected level of precision of the estimates the survey has to deliver

• IT tool: Istat MAUSS-R system implemented in R and Java

• CORA tag: “Statistics”

START

MAUSS-R

AL

LO

CA

TIO

N

CORE Final Meeting – 11 January 2012 7

Sample Selection Service

• Goal: draw a stratified random sample of units from the sampling frame, according to the previously computed optimal allocation

• IT tool: a simple SAS script to be executed in batch mode

• CORA tag: “Population”

SAS SCRIPT

SE

LE

CT

ION

CORE Final Meeting – 11 January 2012 8

• Goal: compute the estimates the survey has to provide (typically for different subpopulations of interest) along with the corresponding confidence intervals

• IT tool: Istat ReGenesees System R-based

• CORA tag: “Statistics”

Estimates and Errors Service

STOP

ReGeneseesSystem

ES

TIM

AT

ION

From the Scenario to the

Demo

CORE Final Meeting – 11 January 2012 9

Allocation(MAUSS-R)

Selection(SAS Script)

Estimation(ReGenesees)

bethel_out

stratif errors

xml

bethel_out sample sample

xml

estimatesframe

Runtime Process Engine

COREtransformations

COREtransformations

Allocation(MAUSS-R)

Selection(SAS Script)

Estimation(ReGenesees)

bethel_out

stratif errors

xml

bethel_out sample sample

xml

estimatesframe

Java/Webserver

COREtransformations

COREtransformations

ISTAT

Allocation(MAUSS-R)

Selection(SAS Script)

Estimation(ReGenesees)

bethel_out

stratif errors

xml

bethel_out sample sample

xml

estimatesframe

Bonita/Windows

COREtransformations

COREtransformations

CBS

ISTAT

• What we are going to see:

A set of GUIs for process, services

and data design

A set of GUIs for process execution

CORE Final Meeting – 11 January 2012 13

Demo Details: Istat

Istat Demo back-end

• What lies “behind” the GUIs

Integration API for CSV-CORE

transformations

Core Repository

Data Flow Control System

CORE Final Meeting – 11 January 2012 14

• A process run executed via

Bonita workflow engine

CORE Final Meeting – 11 January 2012 15

Demo Details: CBS