OER End-to-end Data Management

Sharon Mesick, IPT Chairsharon.mesick@noaa.gov

June 2011

1

OER Data Management - Outline

• Introduction to OER

• Data Management Approach

• Implementation

• Challenges

• Back up slides and notes

2

• The President’s Panel Report established the guidance for a National Ocean Exploration Program (2000)

• Ocean Exploration defined as

“discovery through disciplined diverse observations and recording of the findings” • Data Management identified as one of five critical program components

• NOAA’s Ocean Exploration Program formed• NOAA OE assumes national leadership role (2001)

• Ocean Exploration and Research (OER) created with passage of PL 111-11 (2008)

• Unites NOAA’s OE and Undersea Research Programs

The Frontier Report

3

OE Data Management Drivers

• Frontier report recommendations for data management• Design and implement a data management information processing system

• Document exploration results, ensuring standardization, interoperability and stewardship

• Develop a Geographic Information System (GIS) for exploration data

• Facilitate multidisciplinary data management for an Exploration Flagship

• Widely disseminate exploration results

• Celebrating OE’s 10th anniversary in 2011, all data management recommendations have been addressed

• End-to-end system operations

• Robust, standards-driven documentation program

• Operational, online GIS tools provides access to geospatial database

• Multidisciplinary support to NOAA Ship Okeanos Explorer

• Exploration results disseminated to the public via Digital Atlas

4

Integrated Product Team for Data Management

• NOAA’s OE Program formed an interdisciplinary data management team• Define Ocean Exploration programmatic data management requirements• Perform gap analysis between requirements and available tools & standards• Implement data management solutions

• Publications• Data Management Strategy for the Ocean Exploration Program (April 30, 2002)• Ocean Exploration Program: Metadata Guidelines (November 25, 2002)

• Awards• DOC Bronze medal awarded

“For developing a unique end-to-end data management system to organize, archive and disseminate ocean exploration data…”

5

OER IPT Teams / Members / Activities

TEAM MEMBERS PRIMARY ACTIVITIES

Program Management NCDDC (lead), OER, NNDC, Library, NOS, OMAO, UNH-CCOM, URI, Research Scientists, other extramural partners

Coordinate team, Data management planning and execution

Data Throughput NCDDC (lead), OER, NNDC, OMAO, UNOLS, UNH-CCOM, URI

Develop, operate and maintain end-to-end data management systems

Data Dissemination NCDDC (lead), NOS, NNDC, Library, extramural partners

Ensure access to scientific data and information

• Data management is a cooperative venture between OER and partners• OER Program Plan applied to individual collection activities

• OER offers data management services to partners• Ensures compliance with overarching programmatic goals

6

Post-collection processes

&archive prep

Post-collection processes

&archive prep

Post-collection processes

&archive prep

Post-collection processes

&archive prep

OER / Extramural Data Management Workflow

1 2 3

FundedProposal

Cruise Plan

Data Management

Plan

On-scene data collection &

documentation

Post-collection processes

&archive prep

4

Central dissemination of distributed

information

7

Post-collection processes

&archive prep

Post-collection processes

&archive prep

Post-collection processes

&archive prep

Post-collection processes

&archive prep

OER / Okeanos Data Management Workflow

1 2 3

FundedProposal

Cruise Plan

Data Management

Plan

On-scene data collection &

documentation

Post-collection processes

&archive prep

4

Central dissemination of distributed

information

Okeanos Explorer

8

OER Comparative Data Management Models

ACTIVITY AO AO SPECIFIC EX EX SPECIFIC

Proposal X Peer-review N/A

Cruise Plan X OER science and operations teams; extramural partners

X OER science team, OER & OMAO operations teams & principle scientists

Data management plan

X OER DM Team / Extramural partnerspart of cruise planning

X OER DM Team / Operations Team – identify and plan for individual cruise objectives

During cruise X On-scene X Remote

Post collection X Reach-back to PIs, facilitate agreed-to data management timelines and objectives

X Operate and maintain semi-automated systems; QA DM objectives

Dissemination X Digital Atlas X Okeanos Atlas9

Data Management Plan Overview

TOC DESCRIPTION

Introduction Purpose, authority, scope & points of contact

Actionable Plan

• Data collection activities Instruments, data types, formats, archive target

• Metadata Standards, granularity, keywords

• OER product requirements Web coverage, data products, timeline

• Data dissemination plans Access points, timeline

• Other considerations Intellectual property rights, protected resources, other

• Archive submission plan What data, which archive, how and when

OER DM Guidance OER DM policy and granted authority

OER data submission plan Detailed guidance from each NNDC and NCL

Example documents

10

E2E Data Management System

Systems:• CIMS• VIMS

• MERMAid

Archives• NGDC• NODC

Library• VDMS

NCDDC• GIS DB

Okeanos Atlas

DIGITAL ATLAS

External Sources

USER COMMUNITIES

1 3

4

5

11

2

OER Digital Atlas explore.noaa.gov/DigitalAtlas

Where in the World is Ocean Exploration?

Central Access Point: to distributed exploration data and information

- Data Archives - Data Resources - Outreach- Geophysical - GIS DataBase - Education Resources- Oceanographic - GIS Tools - Public websites- Multimedia - Metadata - Publications 12

OER Okeanos Atlas explore.noaa.gov/OkeanosAtlas

13

IPT / OER Data Management Challenges

• Need for overarching NOAA policy guiding data management for

research grants

– Programmatic policy can be usurped by various mean unless NOAA has an

irrefutable, standard approach

• Need for evolutionary policy toward data documentation standards

– Programs addressing the NSDI objectives, particularly those with methods and

systems in place, need time to evolve

• Need for NOAA-wide approach to preserving video data collections

– Must be recognized as essential to preserve due to large investment

– Various instruments, formats, no standards or guidance

– What does ‘independently understandable’ mean for video?

14

IPT / OER Data Management Challenges

• Partners developing interest in managing data

– UNOLS & NOAA R2R, CRCP, NMSP, other programs, Federal partners

– Need to develop cooperative methods to leverage common practices

• Which partner is the authoritative source of information?

• Each need to show return on investment

• ISO metadata methods offer potential solution

– Enables parent child relationships (e.g., cruise, datasets, products)

• Children ‘point to’ parents

– Enhanced “Responsible Party” enables detailed description of each partner’s

role; also can “tag” record with program names; other options

15