The Design of the Online Questionnaire

of the Italian Population Census

Leonardo Tininini

Antonino Virgillito

NTTS 2013

Brussels

March 5, 2013

The Design of the Online Questionnaire of the Italian Population Census,- Tininini & Virgillito - Brussels, March 5, 2013

2

Outline

• The QPOP Online Questionnaire: the Requirements

• The Design of QPOP

• The Questionnaire Graph

• The Online Questionnaire Usage Statistics

• Conclusions and Future Work

The Design of the Online Questionnaire of the Italian Population Census,- Tininini & Virgillito - Brussels, March 5, 2013

3

The QPOP online questionnaire: the requirements

• To be used by both citizens and operators (by SSO)

• Tight integration with the Census Management System,

particularly the Census - Register comparison

(Poster Session Mar 7, 12:00-13:30)

• Reproducing the 3 paper questionnaries

– short / long form for households + form for collective dwellings

• Each in 3 languages (Italian, German and Slovanian)

• Very stringent requirements of robustness, scalability, data

quality

...therefore

almost impossible re-using or adapting an off-the-shelf product

The Design of the Online Questionnaire of the Italian Population Census,- Tininini & Virgillito - Brussels, March 5, 2013

4

The Design of QPOP

• Write more metadata (and less programming code)

– question templates (graphical rendering but also processing)

– question routing (achieved by Questionnaire Graph)

• Assisted coding of open (textual) questions

– search engine on highest educational qualification (6,000 items)

– dictionary pre-processing, similarity string comparison + result ranking

• Robustness, scalability and data quality

– open source frameworks: Struts2, Spring and Hibernate

– extensive use of caching to limit DB access

– quality checks on both client and server side

The Design of the Online Questionnaire of the Italian Population Census,- Tininini & Virgillito - Brussels, March 5, 2013

5

The Questionnaire Graph (QG)

• Formal description of question routing by Directed Acyclic Graphs

– nodes = questionnaire fragments (questions, but not only)

– labeled edges = go from node 1 to node 2 if the “label” is true

• Simple syntax, easy to be evaluated, yet powerful

– quantification, counting, composition mechanisms...

• Used on both client (Javascript) and server (Java) side

– enable/disable questions, compilation checks

• Each node is associated with a “state” (filled-in, unreachable, ...)

– each node state changes continuously during compilation as it

depends on both the QG and the answers already given

– changes in the answers triggers the recomputation of node states

The Design of the Online Questionnaire of the Italian Population Census,- Tininini & Virgillito - Brussels, March 5, 2013

6

From the questionnaire to the QG

The Design of the Online Questionnaire of the Italian Population Census,- Tininini & Virgillito - Brussels, March 5, 2013

7

Node states and state recomputation

The Design of the Online Questionnaire of the Italian Population Census,- Tininini & Virgillito - Brussels, March 5, 2013

8

Node states and state recomputation in action (1)

The Design of the Online Questionnaire of the Italian Population Census,- Tininini & Virgillito - Brussels, March 5, 2013

9

Node states and state recomputation in action (2)

The Design of the Online Questionnaire of the Italian Population Census,- Tininini & Virgillito - Brussels, March 5, 2013

10

Node states and state recomputation in action (3)

The Design of the Online Questionnaire of the Italian Population Census,- Tininini & Virgillito - Brussels, March 5, 2013

11

QPOP Usage statistics (1)

Usage % for the possible means of returning

the Census questionnaire

33,4%

31,7%

22,6%

12,3%

Online quest.

Municip. c.c.

Post offices

Enumerators

0,00%

1,00%

2,00%

3,00%

4,00%

5,00%

6,00%

7,00%

8,00%

9,00%

10,00%

00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23

Preferred means (33.4%) of returning the

Census questionnaire:

~ 8.5 million household questionnaires

corresponding to:

~ 20 million individuals

~ 30 “partial saves” (on average) for each

completed questionnaire

Majority of access during

office hours

• agreement with the

Public Administrations

• PCs made available by

Municipality Offices

Overall hourly distribution of online questionnaire completion

The Design of the Online Questionnaire of the Italian Population Census,- Tininini & Virgillito - Brussels, March 5, 2013

12

QPOP Usage statistics (2)

0 200.000 400.000 600.000 800.000 1.000.000 1.200.000 1.400.000

Week 10

Week 09

Week 08

Week 07

Week 06

Week 05

Week 04

Week 03

Week 02

Week 01

Number of completed online questionnaires in the first 10 weeks

Maximum load was measured during

the first week

• up to 500/minute questionnaires

completed

• up to 20,000/minute partial saves

(and up to 500/second)

Anomalous peak during the 6th

week

• misinterpretation of a date in the

first page of the form?

• starting date for the enumerators’

activity?

• ....?

The Design of the Online Questionnaire of the Italian Population Census,- Tininini & Virgillito - Brussels, March 5, 2013

13

Conclusion and future works

• Good design practices resulted in:

– High percentage of online-returned questionnaires

– Good stability of the system, even during peak periods

– Easy re-use...

• ...from QPOP to the online questionnaire for the Industry and

Services Census (Sep 2012 – Feb 2013)

– 3 different questionnaires (much more complex than QPOP)

– ~ 70% compiled by end users (+ 30%, also online, by operators)

• The future

– The (paperless) rolling Censuses (first pilot in 2014)

– Graphical User Interface to facilitate Questionnaire Graph Design