Backstage of Population Census in Italy: fieldwork and paradata … · 2013. 8. 7. · (212 UCCs)...

Backstage of Population Census in Italy:

fieldwork and paradata to manage data

quality collection

The role of Istat in Tuscany

Bianca Maria Martelli, Linda Porciani, Alessandro Valentini

5th Conference of European Survey Research Association

Ljubljana, Slovenia, July 15-19, 2013

Summary

� 15th Population and Housing Census in Italy

� Territorial organization

� Managing data quality in the case of Tuscany

�Theoretical approach

� Application� Application

• Field work

• Paradata indicators

• Post census ad hoc survey

� Discussion

Backstage of Population Census in Italy: fieldwork and paradata to manage data quality collection. The role of Istat in Tuscany

B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia, July 15-19, 2013

Main innovations of Census

� From a totally door-to-door to assisted census (by LAU2

Population Register)

� Use of other administrative registers to reduce under-coverage

(fiscal registers, ‘permits-to-stay’ , etc…)

� Mail-out of census forms� Mail-out of census forms

� Multi-way modalities to answer the questionnaires (paper and

web based)

� Sample strategies in order to reduce response burden (short

and long forms)

� Supervision of the process via on line system SGR (Survey

Recording System) SGR



1

NUTS0 - Italy NUTS2 - Region

- ISTAT

- Statistical Office

of the Italian

Ministry of

Interior

Regional Offices

of Census (URCs)

Territorial organization

Responsibility to manage the data collection process

in its own territory

Responsibility to manage the data collection process

in its own territory

NUTS3 - Provinces LAU2 - Municipalities

Statistical Offices

of Prefectures

(UPCs)

Municipal Offices

of census (UCCs)



1

Territorial organization: a focus on Tuscany

Tuscany:

Regional Office of

Istat URC

10 Provinces UPCs 287 Municipalities

(212 UCCs)



2

Managing Data Quality (DQ) in the case of Tuscany

Top-down approach

Researcher selects

DQ attributes

According Wang and Strong (1996), the methodological

framework adopted is the following

Field workIntuitive

Top-down approach

How data quality

weakens during

data collection process

Bottom up approach

Investigation on

actors’ opinions

about DQ attributes

Paradata tool

Ad hoc survey

on census

operators

Theoretical

Empirical



3

Definition of a matrix quality measures/actions developed by URC

Intuitive approach: field work

Quality measures



4

Field-work actions

Training

Scheduled: between July and October 2011[Census day: October, 9th 2011]

Subjects: UPCs, UCCs, Enumerators

(around 5,000 persons)

SupportingIn-distance: by phone (91%) and by e-mail (9%)



5

SupportingIn-distance: by phone (91%) and by e-mail (9%)

In-presence: additional training sessions

MonitoringInspections: checks to UCCs by URC staff

Interventions: on critical cases

Response rate by nationalities

Field-work actions: an intervention

The “elusive population” of foreigners in Prato[15% of the total population in 2011]

1. Official actions promoted by local

institutions (meeting URC, UCC, head of local community);

2. mass media communication (press conference, internet pages by chineselanguage)

1. Official actions promoted by local

institutions (meeting URC, UCC, head of local community);

2. mass media communication (press conference, internet pages by chineselanguage)

1. Informal actions (direct mailing from UCC)

1. Informal actions (direct mailing from UCC)



6

Theoretical approach: paradata tool

1. Dashboard indicators

Real time indicators -> to realize an

immediate set of interventions to

solve general or local criticisms



7

2. Roadmap indicators

Ex post indicators -> to improve DQ

in incoming surveys

(reengineering of the process)

Paradata tool 1.Dashboard indicators

Indicator Description Target

value (%)

QUES.LAC Percentage ratio between number of questionnaires filled in and number of questionnaires in the population registry lists (LAC) at 31.12.2010

≥ 97 ≤ 103

QUES.CA Percentage ratio between number of questionnaires processed and number of questionnaires enabled to the comparison (CA) between census and population registers.

100

Dashboard indicators consist of 6 indicators derived from SGR data

population registers.

DIARIO.LIFA Percentage ratio between processed rows and total rows of potential undercoverage derived from administrative archives (LIFA)

100

DIARIO.NETTO Percentage ratio between rows processed and total number of rows excluding (NETTO) rows of potential undercover derived from LIFA

100

CFR01.EDI Percentage ratio between number of surveyed building (EDI) and number of surveyed buildings in 2001 census

≥ 100 ≤ 150

CFR01.ABI Percentage ratio between number of surveyed unoccupied dwellings (ABI) and number of surveyed unoccupied dwellings in 2001 census

≥ 85 ≤ 115



8

Indicator DIARIO.LIFA

(tested undercoverage)

An application to LAU2

0 – 20,000 inhabitants

Paradata tool 1.Dashboard indicators

Problem

Indicator CFR01.ABI (unoccupied dwellings)(tested undercoverage)

Problem

Localized criticism

Action

Targeted intervention on specific LAU2

Problem

Widespread criticism due to lacking information

Action

Additional training phase to all network census actors



9

Paradata tool 2. Roadmap indicators

97.9

94.9

97.6

95.095

96

97

98

99

100

%

Italy

Tuscany

33.431.7

22.624.5

31.7 31.4

20

25

30

35

40

%

Italy

Tuscany

Standardization process and rate of postal

delivery (%). Italy and TuscanyQuestionnaires by channel of answer (%).

Italy and Tuscany

90

91

92

93

94

95

Standardization

process

Rate of postal

delivery

%

12.3 12.4

0

5

10

15

20

Web Municipal data

collection

centers

Post offices Enumerators

NO need of specific territorial intervention Need of deeper analysis



10

∑=

+=4

1 i

o (WR)logit ii xββ


Where:

WR = Quota of web respondent (by Municipality)

X1 = Number of members per households



11

X1 = Number of members per households

[effect of class >2.29; ref. category ≤2.29]

X2 = Quota of foreigners


X3 = Ageing index


X4 = Demographic dimension of Municipality

[effect of class > 5,573 ; ref. category ≤5,573]


+- 0

Demographic

dimension

Quota of

foreignersAgeing indexHousehold

dimension

Results:

Main insights:

1. Use of web is linked to demographic characteristics of population;

2. To increase the quota of web users it is necessary to support specific

subsets of population: ‘traditional families’ and ‘old population’;

3. A particular care has to be dedicated to small LAU2, with more traditional

habits

dimensionforeignersdimension



12

Empirical approach: ad hoc survey on operators

At the end of Census operations an ad hoc survey (Ivalcens, ‘inquiry to evaluate the census’) was submitted to qualified census operators

The survey was realized via a web technique. The rate of response was 94.6% for national level (100% in Tuscany)

Main results of Ivalcens



13

Main results of Ivalcens

� Scarce interest for the internet channel (one of the lowest evaluation in Italy)� Criticisms about data collection of unoccupied dwellings (problem already emerged during the field work and paradata analysis);� Low appreciation of mail-out (despite the good quality of road-map indicators)� Limited appreciation of the (institutional) context

Discussion

The key factor for DQ should be represented by a complete integration between the central organism of planning and the peripheral offices devoted to supervise the application of rules (URCs in our case).

From our experience, the set of instruments introduced is a From our experience, the set of instruments introduced is a valid tool to:� easily point out the criticalities through simple indicators� prevent bad practices in data collection and help best ones

� monitor the field work and, finally� assess data and process quality



14

Thanks for your attention!

Bianca Maria Martelli, [email protected]

Linda Porciani, [email protected]

Alessandro Valentini, [email protected]



15

Backstage of Population Census in Italy: fieldwork and paradata … · 2013. 8. 7. · (212 UCCs)...

Documents

Transcript of Backstage of Population Census in Italy: fieldwork and paradata … · 2013. 8. 7. · (212 UCCs)...