Marine Data & Information Management – Lessons learnt from...
Transcript of Marine Data & Information Management – Lessons learnt from...
1
SCOR/IGBP Meeting onData Management
for Marine Research Projects
Foresight Centre, Liverpool, UK8-10 December 2003
Marine Data & Information Management– Lessons learnt from JGOFS
Bernard Avril(JGOFS International Project Office)
Presentation Outline
PreambuleIntentions and Disclaimer
Science & Data from Joint Global Ocean Flux Study (1)-(3)What is JGOFS?
JGOFS Data Management System – Initial Plans (1)-(2)What were the initial objective and proposed set-up?
JGOFS Data Management – Final Report, 2003 (1)-(4)What were the actual working set-up and achievements?
JGOFS International Data Collection (1)-(2)Contents and benefits for the users
JGOFS Data Management – Lessons LearntRecommendations of improvements for the benefits of all parties
Data & Information Management – some ideas (1)-(3)Objective and subjective evaluations
Idealised D&IM framework in marine projects (1)-(2)An idealised vision or a feasible set-up for the future?
Conclusion
2
Preambule
Intentions and Disclaimer
This presentation is ...... written with the overall idea that the highest quality inD&IM should help and support the PIs, project sponsors, funding agencies and end-users, to achieve and/or benefit from the best science, today and tomorrow.
This presentation is not ...... always (or ever) the “official” viewpoint from the JGOFS community, if such thing can be defined...
... Yet it is a viewpoint, and it could be sometimes useful, and discussed...
... So, please, do not hesitate to interrupt!
• global carbon survey• regional process studies• long-term time-series• data management• synthesis & modelling
Scientific Goal• oceanic biogeochemical processes of carbon & associated
biogenic elements, and response to climate change.
Science & Data of Joint Global Ocean Flux Study (1)
→ 20+ countries→ 15-year project→ SCOR + IGBP
Strategies
3
12.5
25
37.5
0
Annual primary production (mol-C m-2 a-1) based onSeaWiFS data and model. Falkowski et al., IMCS, Rutgers University, USA.
Global map of the average annual exchange
CO2 flux (mol-C m-2 a-1) across the sea surface. Takahashi et al.,. 1999
Global map of seafloor respiration based on benthic O2 fluxes (mol-O m-2 a-1). Jahnke R., 1996. GBC © AGU
Particulate organic carbon export(mol-C m-2 a-1). Schlitzer, AWI, 2001.
Global composite image of surface Chlorophyll a concentration (mg m-3) estimated from SeaWiFS data (NASA / GFSC + ORBIMAGE)
± 1 GtC/yr
D. Karl
Science & Data of Joint Global Ocean Flux Study (2)
(evolving) extensive & intensive, marine biogeochemistry studies,
design of field projects, hypothesis testing and diagnostic & prognostic models
data management (& information) system
Unprecedented, high spatial & temporal resolutions of complex, multidisciplinary ocean data (20+ physical, chemical, biological and sedimentological core parameters)
D. Karl
Science & Data of Joint Global Ocean Flux Study (3)
4
JGOFS Data Management System – Initial Plans (1)
(rewritten after JGOFS Science Plan, 1990 and JGOFS Implementation Plan, 1992)
JGOFS Data Management System:
“To provide all interested scientists with complete and convenient access to the international JGOFS dataset.”
JGOFS policy regarding data submission, access and exchange:
- encourage free & open communication of findings, including raw data, - access without restriction for any interested user, provided that data originator is contacted for permission of further data use,- national (mandatory) submission by project investigators of their data to the project data system in a timely manner (cf. core parameter protocols),- each national project committee should endorse this policy.
JGOFS Data Management System – Initial Plans (2)
(rewritten after JGOFS Science Plan, 1990 and JGOFS Implementation Plan, 1992)
National Project Data Centres (with a data coordinator):
- flexible, online procedures of submission, exchange, retrieval & work- online national project database, including data inventory & metadata- (extra-)national exchange modes and formats for data project centres,- arrange training in using the procedures,- interact with the project DMTT as needed.
Data Management Group (now DMTT) continuously:- co-ordinate the national activities, as needed,- monitor data management compliance, and- reassess periodically needs & performance.
Core Project Office (now IPO) also for Data Management:Goal: Help the investigators to locate and access project data.- project planning & execution, data management, synthesis & modeling,- project resource center on national and regional activities
5
JGOFS Data Management – Final Report, 2003 (1)
JGOFS Data Management Task Team (cf. ToR, 1997):
• initiate, encourage and develop national data management,• develop, implement timely, national data exchange
mechanisms, and international data integration • compile & dissemintate an integrated, international data
and cruise inventory,• monitor international acceptance of, compliance with,
policy (& adapt it),• liaise with other (inter-)national initiatives & data centres,• provide IPO with advice regarding data management, and• report to SSC and implement its recommendations.
Data Management Task Team (DMTT, 1993-2003)
JGOFS Data Management – Final Report, 2003 (2)Australia
Germany
India
Japan
United States
France
United Kingdom
Canada
from within their NODC (CA, IN, JP, UK), or with a funded position in an oceanographic institution (AU, FR, GE, US).
In other countries, data mostly with individual PIs (e.g., IT, PK, SP) and rarely managed nationally (NL, NO).
6
Data Management Task Team (DMTT)
DMTT Modellers
Task Teams &Synthesis Groups
SSCIPO
PANGAEA
IOC IODEMDM
NODCs
CEOS/GCMD
DIFs
JGOFS Data Management – Final Report, 2003 (3)
National Contact Points
(NCP)North AtlanticIndian Ocean
Southern OceanNorth Pacific
Equatorial Pacific
Regional Synthesis andModelling Groups
Remote SensingGlobal Synthesis and ModellingPhotosynthesis MeasurementsPaleo-JGOFS (w/ PAGES)Continental Margin (w/ LOICZ)Task Teams
SICAP-CO2 IOCCG
Global Synthesis and ModellingOcean Carbon Modelling (w/ GAIM)Global Synthesis Activities
JGOFS Data Management – Final Report, 2003 (4)
Data Management Task Team summarized achievements:
- high profile for biogeochemical data management in community and outside,
- compile and publish national cruise and data inventories, with added value,
- produce a unique, multinational, biogeochemical data product (DVD vol. 1),
- contributed to WDC-A, NASA’s GCMD, WDC-MARE, and others.
- & multiple CDs, websites, publications (books, special issues, special reports, meeting reports, notes)
7
first large scale, international, discrete data collection in global oceanic biogeochemistry, from ~1000 cruises~80-96% JGOFS-funded science data available (for DMTT-represented countries),~85% datasets/cruises from DMTT-represented
countries,according to JGOFS cruise inventories, ~80% of all JGOFS data are on DVD vol. 1,rough estimate: ~5% [2-8%] dedicated to data management... + ~5-yr half-time IPO supportfair representation of the national JGOFS data managersʹ efforts,
JGOFS International Data Collection (1)
JGOFS International Data Collection.Volume 1: Discrete Datasets DVD
JGOFS International Data Collection. Vol. 2: Integrated
Datasets DVD with WDC-MARE, funded by AWI and
BMBF (Germany).
JGOFS International Data Collection (2)
About 40’000 JGOFS data entries already in PANGAEA (www.pangaea.de/)
8
JGOFS Data Management – Lessons Learnt
(rewritten after JGOFS International Data Collection. Vol. 1 – readme fileand presentation during the 3rd IGBP Congress)
Lessons learned from JGOFS: Recommendations for future Data and Information Management (to ensure the rapid dissemination of data and its long-term preservation and accessibility)
Establish a coherent, credible, semi-distributed and scalable, end-to-end D&IM plan with:clear D&IM objectives, activities and timeline,policy (e.g., delivery and exchange standards), widely agreed upon by the scientists and sponsors, and supported by funding agencies to insure compliance,clearly identified core project science and associated core parameters, to be revised regularly,an International Project Data Centre designed to establish guidelines, provide advice, and facilitate exchange of knowledge and expertise,experienced, full-time national data managers / coordinators,increase overall value of scientific research and derived outputs...
Issues / Questions to be addressed about Data & Information Management, through few keywords
describing the overall D&IM system
Data & Information Management – some ideas (1)
Flexibility
Credibility
Adaptability
Support
Focus
Productivity
Manageability
Integration
9
Data & Information Management – some ideas (2)
Focus: definition of the project scope (business plan ?)(e.g., identified project (inter)-national activities, core parameters, expected data volume and data management & product requirements)
Credibility: Coherent & credible data management framework(“An attractive carrot is worth ten big sticks”)
Support: full and adequate support of the PIs, project managers, project sponsors and other end-users..., and reciprocally!
Flexibility: levels of granularity, of stability and of efficiency(e.g., DM organised by PIs, projects, nations, parameters, communities?)
Issues / Questions to be addressed about Data & Information Management, through few keywords describing the overall D&IM system
Data & Information Management – some ideas (3)Issues / Questions to be addressed about Data & Information
Management, through few keywords describing the overall D&IM system
Adaptability: roles of already existing N(O)DCs?(e.g., long-term archival / on-demand, multi-disciplinary, multi-strategic
project support)
Integration: added-value for all integrated activities and data product, for all parties
(e.g., no duplication of works, standards, easier synthesis and modelling)
Manageability: Objective tools to review, evaluate and improve the D&IM efficiency and achievements
(e.g., data delivery, data usability, capacity building)
Productivity: create highly visible, international deliverables to maintain the D&IM dynamics
10
Schematic diagram representing the interactions between the project partners andcollaborative bodies, and the main fluxes of data & information.
Idealised D&IM framework in marine projects (1)After Avril & Dittert (2003)
Each element represents a
function rather than an office
• a range of services and products, taking full advantage of best practices, standards or innovative approaches,
• pro-active, ”bottom-up” strategies addressing all needs and requirements,• establish & support experienced, full-time national or regional data
managers, through existing infrastructures, capacity building, new international synergies,
• fair ”top-down” strategies to insure compliance, better integration, • policy fully agreed upon and widely disseminated at the (inter)-national
levels, for delivery, quality-control, referencing, exchange, integration, dissemination, preservation..., to facilitate interoperability,
• project knowledge resource centre (DIMO), a Data Management Committee (DMC, with data managers, observationalists and modellers) and a set of Data Evaluation Units (DEUs), where needed.
Idealised D&IM framework in marine projects (2)(rewritten after Avril & Dittert (2003) and other texts)
+ There is a large number of marine data policies already available. A rough compilation of unofficial comments from data managers is also available.
11
Conclusion(rewritten after H. Ducklow, Final JGOFS OSC, May 2003)
JGOFS was created as a visionary programme, especially through its union of biology, chemistry and physics, (and sedimentology) and its collaborative science works including more than 20 nations, in a global, marine biogeochemistry (V. Vernadsky, “The Biosphere”, 1926) project. This is a major achievement of JGOFS.
And behind all those science works and approaches, there is also a revolution in the way data and information have been managed! ... and it is not finished!
JGOFS experienced some drastic changes in the way oceanography is done and managed. Now, “things” are changing even faster: increased overall commitment and focus for data management, information management and even knowledge management; and the international collaboration and standards are better promoted and accepted... Yet sometimes, it works pretty differently than anticipated on this side of the world... Within the early plans, “great” ideas and principles were presented but they were not always realistic, and were not applicable on into solutions pratical enough, or sometimes with a commitment inadequate for the task.