Data and Data Requirements
17
Project number: 283465 Data and Data Requirements Wouter Los University of Amsterdam
description
Data and Data Requirements. Wouter Los University of Amsterdam. Environmental Science. Earth as a single complex and coupled system. ESFRI Environmental Research Infrastructures. Gas (CO2 etc ) fluxes ∂ (concentration). Radar interference data. Areal and s atellite observation. - PowerPoint PPT Presentation
Transcript of Data and Data Requirements
Project number: 283465
Data and Data Requirements
Wouter LosUniversity of Amsterdam
Project number: 283465
Environmental Science
oceanic and atmospheric
processes
long-term development of
the climate system
Biological processes
biodiversity
development of the cryosphere and lithosphere
Earth as a single complex and coupled system
23/10/12 W. Los - ENVRI @ EUDAT 2
Project number: 283465
ESFRI Environmental Research Infrastructures
• Tropospheric research aircraft
COPAL
• Upgrade of incoherent SCATter facility
EISCAT-3D
• Multidisciplinary seafloor observatory
EMSO
• Plate observing system
EPOS
• Global ocean observing infrastructure
EURO-ARGO
• Aircraft for global observing system
IAGOS
• Integrated carbon observation system
ICOS
• Biodiversity and ecosystem research infra
LIFEWATCH
• Svalbard arctic Earth observing system
SIOS
23/10/2012 W. Losi - ENVRI @ EUDAT 3
Project number: 283465
23/10//12 -ENVRI @ EUDAT 4
Project number: 283465
29/03/12 Pasquale Pagano - ENVRI @ EGI CF 2012 6
Radar interference dataGas (CO2 etc) fluxes
∂ (concentration)
Areal andsatelliteobservation
Species data, distributions, abundance, biomass, etc.
Observations, sensor data,collection data, DNA, etcMarine
sensors
Currents, salinity,deposition, etc
Platetectonics
Seismic data,satellite data,sensors, etc
Project number: 283465
Goal
Enable multidisciplinary scientists to access and study data from multiple domains for “system level” research
by providing solutions and guidelines for the RIs common needs
Multiple data producersMultiple data consumers
723/10/12 W. Los- ENVRI @ EUDAT
Project number: 283465
8
Approach
discover data which are heterogeneous in format, content, and metadata description
harmonise, integrate and analyse data across domains and RIs Pr
omot
e Ac
cess
ibili
tyPreserve Specificity
23/10/12 W. Los - ENVRI @ EUDAT
PROVIDE SOFTWARE TOOLS TO
Project number: 283465
Data infrastructure requirements
Integrated data discovery across various catalogues(Near) real-time data handlingFederation over infrastructures/servicesPersistent identifiers mechanismMetadata definition and assignmentAttribution / crediting author/ownershipQuality control of dataProvenance and preservationArchiving vs. regeneration of data and/or results (processed data)Single sign-on, delegated authorisationRunning complex modelsData staging or moving computation to data
23/10/12 W. Los- ENVRI @ EUDAT 9
Project number: 283465
First steps - priority areas
Integrated data discovery across various centres / catalogues
(near) Real-time data handling
Federation over existing (national or international) infrastructures / services
23/10/12 W. Los - ENVRI @ EUDAT 10
Project number: 283465
1: Integrated data discovery
Integrated data discovery across various centres / catalogues
The challenge of being able to easily discover data which are heterogeneous (in format, content, and metadata description) and which are stored at different placesENVRI partners ESA, CNR, UvA and CSC are tackling thisDoes EUDAT see a role to contribute?
Project number: 283465
Geospatial Data Services
Geospatial Repositories
Data Discovery
Data Access Data Process
OGCOpenSearch
Linked Open DataCatalogue Services
OGCWCS
THREDDS
OGCWPS
WPS 52N
P1 P2 P..
WPS Hadoop
Hadoop Cluster
HDFS
Data Pub. /Vis.OGC
WMS, WFS
GeoServer
gCub
e Da
ta
stag
ing
by courtesy of P. Pagano
Project number: 283465
2: (Near) Real Time Data Handling
(near) Real-time data handlingThe challenge(s) of being able to handle real-time data Challenges include:
collecting, storing and cataloguing data as it arrives in real-time from sensorsprocessing data into derived data products in real-time analysing data in real-time
It was suggested that EUDAT might take this up
Project number: 283465
3: Federation
Federation over existing (national and international infrastructures / services
The challenge of bringing together existing infrastructure components / services as contributions to the construction of an Research InfrastructureThe challenge of bringing about interoperability (syntactic and semantic) between separately owned and operated facilities that each contribute to the Research InfrastructureIs EUDAT also interested?
Project number: 283465
Data infrastructure requirements
Integrated data discovery across various catalogues(Near) real-time data handlingFederation over infrastructures/servicesPersistent identifiers mechanismMetadata definition and assignmentAttribution / crediting author/ownershipQuality control of dataProvenance and preservationArchiving vs. regeneration of data and/or results (processed data)Single sign-on, delegated authorisationRunning complex modelsData staging or moving computation to data
23/10/12 W. Los- ENVRI @ EUDAT 15
Project number: 283465
Need a Common Data Infrastructure
Managing the growing amount of dataImproving interoperability between infrastructures and across disciplines Promoting collaboration and clarifying roles and responsibilities
23/10/12 W. Los- ENVRI @ EUDAT 16
EUDAT contributions to the ENVRI consortium is welcome!
Project number: 283465
Thank you
http://envri.eu/
23/10/2012 W. Los – ENVRI @ EUDAT 17
