Data Management & Data lifecycle

• Survey Conception• Data System Architecture• Data collection

management• Data Analysis &

Dissemination

Data Management & Data Lifecycle

Type of info per usage

Global reports

Project report

Outreach material

Indicators (Focus)

Registration X X X X

IDP profiling X X X XProtection Incident monitoring X X X XProtection situation monitoring X X X XPopulation movement monitoring X X X X

Sectoral assessment X XPartners activities monitoring X XAbsorbtion capacity evaluation X X X

Introduction

Data Management & Data Lifecycle

From Data.. to InformationIntroduction

Operation Data Manager should be involved in all the steps of a “Data Lifecycle”.

SurveyConception

DataSystem

Architecture

DataCollection

management

Data Analysis &

Dissemination

Any break of this cycle ends with the failure of the system :• A data collection form that is ill-

designed either because it does not satisfy operational information requirements or is flawed from a technical standpoint

• A well designed survey with a poorly designed and therefore poorly maintained database

• A structurally well designed database with no data, as data collection cycles have not been integrated/respected

• A well populated database without implemented reports and queries and therefore no output

Data Management & Data Lifecycle

Before the Form…Survey Conception

• Avoid reinventing the wheel – check what has been designed and piloted before

• Consultation with all stakeholders – avoid duplication of efforts and assessment fatigue of beneficiaries

• Layers of data collection• Collect Simple base reference data first

• Embark on detailed info based on samples defined from the base reference

• Data collection frequency should vary according to how frequently the phenomena being tracked or measured changes

Data Management & Data Lifecycle

Good practices for Data Collection Forms

1. Questionnaires used in survey research should be clear and well presented.

2. Think about the form of the questions, 3. Keep the survey as short as possible.4. Make definitions of data elements

consistent with standard definitions and analytic conventions

5. Plan clearly how answers will be analyzed.

6. Test the survey for “understandability” and respondent effort through focus groups

Survey Conception

Data Management & Data Lifecycle

Data model

• Data models are the key for interoperability (i.e easy data exchange with partners)

• Implementing partners should not have to draft and decide on a core data model; it should be the same everywhere and just adapted locally where necessary; support (guidelines) need to be there

• Importance of a common referential

Data System Architecture

Beneficiaryregistration

Site / community

Assessment

Activity monitoring

Site

Who’s doing what where?

Multi sectoral assessment:-Health-Education-Water

Bio DataVulnerabilityNeeds

Demographics

Project activities descriptionPerformance Indicators

Base indicatorsDelivered Assistance

Infrastructure InventoryOrganization

Data Management & Data Lifecycle

System architectureData System Architecture

• Building an Interface for data collection:• Mobile • Offline desktop• Web/Server based• OCR* ready form (can be scanned)

• Integration of external data source (ETL**)• Offering analysis capacity (OLAP*** and

Stats)* Mechanical or electronic translation of scanned images of handwritten, typewritten or printed text into machine-encoded text

** Extract, transform, and load (ETL) is a process in database usage that involves Extracting data from outside sources, Transforming it to fit operational needs (which can include quality levels), Loading it into the end target (database or data warehouse)*** An OLAP (Online analytical processing) cube is a data structure that allows fast analysis of data.

Data Management & Data Lifecycle

Reports are part of the data system

Queries and tools to extract data from the databases need to be designed along with the database

Must give abilities for reporting officers to

- Set up queries and reports without high level IT knowledge

- To be clear on the standard indicators these queries should be based on

Data System Architecture

Data Management & Data Lifecycle

Data collection strategies

• Direct coordination with partners•ex : Somali protection cluster

• Establishment of a « data collection project »•ex : UNOPS Goma

• Specific Contract with a dedicated partner•Ex: CartONG in Uganda

Data collection management

Data Management & Data Lifecycle

Implementation matrix

UNHCRDirect Government

Implementing partner Project

Dedicated partner

Registration X X

IDP profiling X X XProtection Incident monitoring X XProtection situation monitoring X XPopulation movement monitoring X X X

Sectoral assessment X XPartners activities monitoring XAbsorbtion capacity evaluation X X X

Avoid conflict of interest

Data collection management

Data Management & Data Lifecycle

PDF reports and maps

• Targets mostly local partners and decision makers

• Can be disseminated through• mailing list (cf Somali protection)• Google group (cf Goma Update)• Website (cf ReliefWeb)

Data Dissemination

Data Management & Data Lifecycle

GeoPortal and Open Data API

GeoPortal: • is a tool to ensure institutional memory

and “Master Data” management• Can be a tool for desk officers to visualize

a situation and use map extracts in their reporting

Data API:• Can be used for global dissemination: cf

Worldbank Data API or Google public data

• Offers material for data journalism (e.g. computer assisted reporting on data through journalists)

Data Dissemination

Data Management & Data Lifecycle

Data, Law & License

For all data sets that do not fall under the “Guidelines for the Regulation of Computerized Personal Data Files” (for instance protection data) ….

…. The “Open database license” (ODBL) can give a legal frame to all our data collection activities

http://www.opendatacommons.org/licenses/odbl/1.0

Data Dissemination

http://www.unhcr.org/refworld/pdfid/3ddcafaac.pdf

Data Management & Data Lifecycle

Providing support for the 4 phases of the process

Conclusion

4 specific types of expertise that are difficult to combine in one profile:

•Statistician/Analyst: Creating a questionnaire and compiling analyzing the resulting statistics

• IS Architect: Building the information system

• Manager: Managing the stakeholder consultation process during the design phase, the collection in the field and dissemination of results

• Data journalist: Developing sound and sexy reports

Need to find where are the gap among the “Operation Data Management” officers network

Need to define the training & support need for each of those specific domains