Modernization of Statistical Information systems Global initiatives · 2015. 1. 30. ·...

18
1 Modernization of Statistical f Information systems Global initiatives Eric Hermouet, Statistics Division, ESCAP Presentation Why modernization? What is modernization? What is modernization? How: Global initiatives and collaboration mechanisms Main outputs of those mechanisms

Transcript of Modernization of Statistical Information systems Global initiatives · 2015. 1. 30. ·...

  • 1

    Modernization of Statistical fInformation systemsGlobal initiatives

    Eric Hermouet, Statistics Division, ESCAP

    1

    Presentation

    Why modernization? What is modernization? What is modernization? How: Global initiatives and collaboration mechanisms Main outputs of those mechanisms

  • 2

    Modernization

    Modernization of statistical information systems Industrialization of statistics Industrialization of statistics Modernization of statistical production and services

    The data deluge

    The internet hadThe internet had 1800 exabytes of data in 2011

    Exabyte=1018

    Or 1 million terabytesy

  • 3

    The data deluge

    50 000 exabytes by 202050 000 exabytes by 2020

    Even if only 0.1% of these data is useful, that leave millions of terabytes for possible statistical purposes

    New data providers

    Google – Real-time price indices– Real-time price indices– Public data explorer– First point of reference for the “data generation”

    Facebook Telecom companies

    – 4 billion mobile phones in Asia and the Pacificp

    How can statistical offices access and use those data sources?

  • 4

    Changing user expectations

    Expectation of data available at a faster rate– Real-time – Real-time

    Ability to customize datasets– Linking coherently datasets across domains– With a high degree of detail

    Data presentation addressing different target groups– Governments– General public

    IT progress

    IT developments offers new solutions– In processing huge amounts of data– In processing huge amounts of data– In accessing new sources of data

    In a more efficient way– In terms of speed– In terms of financial costs

  • 5

    So what is modernization?

    Common generic processes Common tools Common tools Common methodologies Recognizing that all statistics are produced in a similar way

    – No special domains

    Increased flexibility to adapty– To access new data sources– To generate new statistical products

  • 6

    A shared production environment

    From To

    Custom design Use of information bank andCustom design Use of information bank and modular design

    Built solutions Assembled solutionsDesign for a collection mode Source independent design

    “designed-in” quality Working with existing data and varying level of qualityvarying level of quality

    Survey cycles with clear start and end points

    Continuous approach of on-going collection, processing, and release

    A shared production environment

    From To

    Direct data collection Tapping into existing dataDirect data collection supplemented with data from administrative source

    Tapping into existing data, using direct data collection to link sources and bridge gaps

    Individually crafted data structure

    Use of agreed standard approaches

    Management of data Management of data, metadata, and paradata

    A large field workforce Smaller field workforce with specialized interviewing skills

  • 7

    A shared production environment

    From To

    Spending effort and resources More emphasis on specifyingSpending effort and resources on collection and processing

    More emphasis on specifying needs, design, and analysis

    Understanding user needs and designing collection instruments

    Understanding data characteristics and negotiating solutions that bridges gaps between existing data and user requirementsuser requirements

    What is modernization and why is it needed?

    Questions so far ?

    14

  • 8

    Global initiatives & collaboration mechanisms

    15

    Global initiatives & collaboration mechanisms

    High-Level Group for the Modernization of Statistical Production and Services (HLG)Production and Services (HLG)

    Formerly HLG-BAS Objectives

    – To promote common standards, models, tools and methods to support the modernization of official statistics;

    – To drive new developments in the production, d d f ff lorganization and products of official statistics, ensuring

    effective coordination and information sharing within official statistics, and with relevant external bodies;

  • 9

    HLG

    Gosse van der Veen (Netherlands) - Chairman Brian Pink (Australia) Brian Pink (Australia) Eduardo Sojo Garza-Aldape (Mexico) Enrico Giovannini (Italy) Mr Park Hyungsoo (Republic of Korea) Irena Križman (Slovenia) Katherine Wallman (United States)( ) Walter Radermacher (Eurostat) Martine Durand (OECD) Lidia Bratanova (UNECE)

    HLG reference documents

    Strategic vision– http://www1 unece org/stat/platform/display/hlgbas/S– http://www1.unece.org/stat/platform/display/hlgbas/S

    trategic+Vision Strategy to implement the vision of the HLG

    – http://www1.unece.org/stat/platform/display/hlgbas/HLG+Strategy

  • 10

    Technical groups - MSIS

    Management of Statistical Information Systems (MSIS)– Objectives: j

    • To provide, through regular meetings and other means such as the MSIS Wiki, a forum for exchange of experiences and good practices among information systems managers from national and international statistical organizations.

    • To contribute to the coordination of activities of different national and international organizations in the area of statistical information systems.y

    • To facilitate and encourage implementation of international standards and recommendations in the field of statistical computing among national and international statistical organizations.

    Technical groups - MSIS

    Annual meetings jointly organized by UNECE, Eurostat, OECD. And ESCAP for this 2013 MSISOECD. And ESCAP for this 2013 MSIS– MSIS meetings consider issues related to information

    technology governance and management, system architecture, accessibility and usability.

    – First meeting in 2000– Secretariat: UNECE

    O ll UN d l Open to all UN countries and international organizations Reports to the Conference of European Statisticians

  • 11

    Technical Group - METIS

    Steering Group on Statistical Metadata – METIS– Objectives:

    P t th i l t ti f t d t t b d l i • Promote the implementation of metadata systems by developing advocacy targeting the senior management level and subject-matter staff of NSOs;

    • Oversee the maintenance of the Common Metadata Framework (CMF) directing it towards a practical guide serving national statistical offices;

    • Facilitate collection, discussion and dissemination of best practices in the field of statistical metadata

    – Established in 1990Established in 1990– Work sessions every 2-3 years– Workshops in-between– Organized with Eurostat / OECD– Open to all UN member countries and international organizations

    Technical Groups - SAB

    Sharing Advisory Board– Objectives – Objectives

    • To promote harmonization of business and information systems architectures;

    • To support collaboration for the development of statistical software

    • To provide guidelines and tools to assess new statistical f l dsoftware tools and components

    • To assist in the improvement of the technical statistical infrastructure of countries both within and outside the UNECE region as required.

  • 12

    Technical groups - others

    – Statistical Data editing– Statistical Data collection– Statistical Data collection

    Not directly overseen by HLG – Statistical Network– SDMX Expert Group– Statistical Open Standards Group– Working group on quality in statistics g g p q y– ….

    Main outputs

    Emerging “statistical industry” standards– GSBPM– GSBPM– GSIM– SDMX– DDI

  • 13

    GSBPM

    Generic Statistical Business Process Model– To define and describe statistical processes in a coherent – To define and describe statistical processes in a coherent

    way– To standardize process terminology– To compare and benchmark processes within and between

    organisations– To identify synergies between processes– To inform decisions on systems architectures and

    organisation of resources

    GSBPM

    Applicability– All activities undertaken by producers of official statistics – All activities undertaken by producers of official statistics

    which result in data outputs– National and international statistical organizations– Independent of data source, can be used for:

    • Surveys / censuses• Administrative sources / register-based statisticsg• Mixed sources

  • 14

    GSBPM

    Not a linear model Sub-processes do not have to be followed in a strict order Sub-processes do not have to be followed in a strict order It is a matrix, through which there are many possible paths,

    including iterative loops within and between phases Some iterations of a regular process may skip certain sub-

    processes

  • 15

    GSBPM

    Examples of use– Harmonizing statistical computing systems – Harmonizing statistical computing systems – Facilitating sharing of statistical software– Framework for process quality management– Structure for storage of documents – Measuring operational costs

  • 16

    GSIM

    GSIM contains objects which specify information about the real world – 'information objects‘ – Examples include data and

    metadata (such as classifications) as well as the rules and parameters needed for production processes to run (for example, data editing rules) editing rules).

    GSIM identifies around 150 information objects, which are grouped into four top-level groups

    GSIM

    Generic Statistical Information Model To describe data and metadata objects and flows within the To describe data and metadata objects and flows within the

    statistical business process Implementation of GSIM, in combination with GSBPM, allow

    – Creating an environment to prepare for reuse and sharing of methods, components and processes;

    – Implementing rule based process control, thus minimizing h h dhuman intervention in the production process;

    – Economies of scale through development of common tools by the community of statistical organizations.

  • 17

    SDMX and DDi

    Standard Data and Metadata eXchange– UNSD/DFID project in Cambodia Laos Thailand Viet – UNSD/DFID project in Cambodia, Laos, Thailand, Viet

    Nam Data Documentation Initiatives

    – World Bank Microdata Management Toolkit

    SDMX and DDI

  • 18

    What’s next?

    Launch of “Plug and Play” project – February 2013– To create a common statistical production architecture To create a common statistical production architecture – To create a standardized architecture for statistical

    production solutions, including processes, information and systems, coherent with the Generic Statistical Business Process Model (GSBPM) and the Generic Statistical Information Model (GSIM),

    – To enable and advance the sharing of production processes t th d i tor components, thus reducing costs.

    – To provide the basis for a central inventory or repository with life cycle management of sharable production processes and components.