The FOODSECURE Information System: where are we?

David Verhoog (LEI), Robert M'barek(JRC-IPTS)

7 October 2013, Addis Ababa

• Introduction DMS (David Verhoog)

• Internal project data communication: MetaBase (David Verhoog)

• Data to end-users: DataM (Robert M’barek)

Outline:

Introduction DMS

Internal data management

MetaBase

•Data storage (primary data + WP outputs)

•Data up/download

•Meta-information

•Data harmonisation

•Data visualisation

Data dissemination DataM

•Data comparison and advanced analysis

•Reporting and data dissemination

Navigator

Selected dataset

WP deliverables

ClassificationsMapping

FoodSecure partners

Stakeholders and public

Reports prepared by FoodSecure

partners

• Prototype DMS (MetaBase, DataM) is work in process• Prototype MetaBase is in place, is used and can be

reached through the FoodSecure website• In order to show the data in the FoodSecure website,

web-services in XML, SOAP and JSON were build• Data from different partners already loaded• Possibility to download big files directly• Upload function for sending in data

State of play

Structure of MetaBase

Already provided by WUR-LEI:• Eurostat data

• FAO data

Received:• WP2: indicators FNS (K.U. Leuven)

• WP3: Access database on innovations and typologies (ZEF)

• WP4: Large amount of datasets uploaded (Uniroma3)

• WP5: Stakeholder data based on WDI-database (WUR-LEI)

• WP7: AgMIP data entered as an example of possibilities offered and

FAO International Nutritive factors (WUR-LEI)

Data In MetaBase

Expected:• WP3: D3.3 datasets from FAO/CGIAR (global crop yields)• WP6: Database for short term forecasting• WP7: Data from modellers

• Input data (indicators, drivers)• Model results (scenarios)• Mappings of codes between models

• WP8, 9, 10, 11 qualitative and quantitative data, including policy shocks, scenario outcomes, summary tables on micro-level datasets on institutions, case study material, ground proofing work.

Data in MetaBase (continued)

• Data should be delivered in database format (different classification columns and one data value column).

• MS-Access or SQL database file or csv file in database format.

• Contain metadata (source, date, description of content, status).

• Consistency between database and meta information (e.g. codebook) is very much appreciated.

• Mappings between codes with other data sources

Data requirements

• Coded conform the code in the classification column in the database

• Explained (e.g. country codes in database are in ISO 3166-1 alpha-3)

• Supplied with a label and (if meaningful) a description

Requirements for classifications

• Time consuming to develop web-services

• Data not in right format, inconsistencies between dataset and codebook

• WP3 data in MS-Access tool, download complete tool, no option to download separate tables

Problems encountered

• Metadata in web-services• Download function in web-tool for whole files (e.g. Access,

Word, Excel, Pdf)• Classifications and metadata in web-tool• Make DataM visible through the FoodSecure website• Establish the link between MetaBase and DataM• Add more datasets and concordances from partners• Documentation on how to use the web-tool

Still to be done

Over to Robert: DataM

Introduction DMS

Internal data management

MetaBase

•Data storage (primary data + WP outputs)

•Data up/download

•Meta-information

•Data harmonisation

•Data visualisation

Data dissemination DataM

•Data comparison and advanced analysis

•Reporting and data dissemination

Navigator

Selected dataset

WP deliverables

ClassificationsMapping

FoodSecure partners

Stakeholders and public

Reports prepared by FoodSecure

partners

General Vision

14

DataM Processing/ Mapping

Predefined Reports Queries Factsheets

Link to FoodSecurewebsite

Data in DataM:• ESTAT• FAO• Other

New Datasets:• Africa…

FoodSecureProject data(METABASE)

Model Results

… ….

… Etc.

Subtopics

DataM back-office application

DataM: http://www.datamweb.com/datam/ (work in progress)

EC available or upon purchasing a licence

Public availability

DataM Web vision

15

Data dissemination with DataM web-portal

Existing portal featured by European Commission (JRC-IPTS)

Most cost-efficient way to disseminate and visualize data Goes beyond project life-time Fulfills the promised features Synergy with other datasets – more attractive for user Flexible concept to accommodate numeric data

Countries: world (group and individual countries) Attributes: Production, Consumption (Domestic Use,

Food Use, Biofuel use, Feed Use, Other use), Stocks (beginning, ending, variation), Import, Export, Price (Producer, Future, Real, Nominal, Index), Employment and Other Macro-economic indicators

Commodities: Crops and Livestock, Food Products (e.g. sugar, oils and fats etc.), Biofuels

Other commodities to be included: Fish and aquatic organisms, Forest and Wood Products, Other relevant Manufacturing Products, Waste, Energy

Datasets

17

Already available Partially available To be includedAgricultural Outlook databases (data projections)Eurostat Supply Balance Sheets; Eurostat Crop production and Land use; Eurostat Livestock and Meat; Eurostat Milk and Dairy Products; FAO Food Balance SheetFAO Food SupplyFAO TradeFAO ProductionFAO PopulationFAO ResourcesPrices: Spot and futures (Wall Street Journal, London, Paris), EUROSTAT Prices, OilwordPrices, UN FAO (Consumer Price, Food Price Index, International Prices, National Prices, PriceStat), World BankEuropean Labour Force Survey (EU-LFS)World Development IndicatorsOECD PSEAgrosam Model DatabaseOther Model Databases

ComextComtrade

FAO Food Security IndicatorsFood aid (World food program and FAO)

18

Remark: currently FS website shows also ESTAT, OECD, FAO,WB data – most of it and more will be available in DataM web.However, main idea is to show PROJECT results.

19

Predefined ReportsDynamic reports already created containing graphs and/or tables that will enable to user to browse the data on a specific issue by country/commodity/year.

Examplesto be shown online.

20

Predefined Reports – cont.

21

Predefined Reports – cont.

Predefined Reports – cont.

23

Queries – ad-hoc visualisations of the data present in the harmonised datasets.

24

Queries – ad-hoc visualisations of the data present in the harmonised datasets.

Factsheets – static reports containing both graphics and information on a specific issue. Available for download in pdf format.

More information on DataM:

ftp://ftp.jrc.es/pub/EURdoc/JRC81668.pdf

25

• LEI team:• David Verhoog (WP13 leader)• Frans Godeschalk (content)• Wietse Dol (MetaBase and web services)• Eugene Westerhof (web services)• Barbara van der Hout (website)• John Doornbos (website)

• IPTS team:• Robert M’barek – Action leader• Fabien Santini – Business• Arnaldo Caivano – IT• Cornelia Suta – Contents• Tevezia Ronzon - Contents• JRC-IPTS-DATAM-WORKGROUP@ec.europa.eu

26

Contacts

• Comments on data delivery and data requirements• What is interesting for the public• What data can be made public? Property rights?

Difference between external and internal use.• Suggestions for improvement Missing functionalities?

• Feedback!!

Points for discussion:

Thank you for your attention