Status of IOOS Data Management and Communications Subsystem

Implementation Plan

Steve Hankin

Chair, IOOS Data and Communications Steering Committee

The Data Management and Communications Steering Committee (DMAC-SC) was appointed in May 2002, as recommended at the March 2002 National Meeting

• DMAC-SC is supported by four Expert Teams

Data Transport Data Discovery and MetadataData Applications Data Archival

and two Outreach Teams Data Facilities Management Products and Services

• Expert Teams are evaluating available technologies and making recommendations in the form of White Papers;

• Outreach Teams are assembling Community Issues Lists and establishing feedback and review networks

• The Plan will be circulated widely for review

Implementation Plan - The Process

IOOS Data and Communications Steering Committee

Committee Members

• Steve Hankin (Chair), NOAA Pacific Marine Environmental Laboratory• Lowell Bahner, NOAA Chesapeake Bay Office• Landry Bernard, NOAA National Data Buoy Center• Peter Cornillon, Univ. of Rhode Island (Data Transport Team Lead)• Fred Grassle, Rutgers Institute of Marine and Coastal Sciences• Chuck Hakkarinen, EPRI• David Legler, U.S. CLIVAR (Data Applications Team Lead)• John Lever, Naval Oceanographic Office• Phil Mundy, Exxon Valdez Oil Spill Trustee Council, Gulf of Alaska

Ecosystem Monitoring & Research (GEM) (User Outreach Team Lead)• Worth Nowlin, Texas A&M University• Susan Starke, NOAA Nat’l Coastal Data Development Center (Data

Discovery/Metadata Team Lead)• Steven J. Worley, UCAR/NCAR (Data Archive Team Lead)

IOOS Data and Communications Steering Committee

Team Members

DATA TRANSPORT

Peter Cornillon (Lead), URISteve Collins, NOAADonald Denbo, NOAAAllan DoyleJames Gallagher, URIDan Holloway, URITony Lavoi, NOAAKen McDonald, NASAReagan Moore, SDSCEben OldmixonRichard Owens, NOAAPhoebe Zhang, Rutgers U.

DATA DISCOVERY/METADATA

Susan Starke (Lead), NOAAAnne Ball, NOAAJulie Bosch, NOAAJohn Caron, UCARCheryl Demers, NOAADonald Denbo, NOAADan Holloway, URILola Olson, NASAKaren Stocks, UCSD

IOOS Data and Communications Steering Committee

Team Members

DATA APPLICATIONS

David Legler (Lead), USCLIVARRuss Beard, NOAAJames Cummings, NavyDaphne Fautin, U. KansasMargarita Gregg, NOAACraig Kelly, NavyBernie Kilonsky, SOESTJohn Lever, Navy

DATA ARCHIVE

Steve Worley (Lead), UCARDon Collins, NOAABob Cushman, CDIACMark Fornwall, Maui Res. & Tech. CenterSteven Hale, EPAJohn Hughes, MELAlex Koyrz, CDIACSyd Levitus, NOAAChris Lynnes, NASASteven Rutz, NOAAKurt Schnebele, NOAAGeorge Sharman, NOAAGus Shumbera, NOAA

IOOS Data and Communications Steering Committee

Team Members

DATA FACILITIES MGMNT

Landry Bernard (Lead), NOAA

William Birkemeier, USACE

Lee Dantzler, NOAA

Steve Hankin, NOAA

J. Edward Johnson, Navy

Joe Stinus, NOAA

USER OUTREACH

Phil Mundy (Lead), Exxon Valdez Trusteeship Council

Phil Bogden, GoMOOSCarol Dorsey, AL Dept. of HealthDave Eslinger, NOAAMark Luther, U. South FLMike McCann, MBARIRoy Mendelssohn, NOAAMalcolm Spaulding, URIJack Tamul, NavySuzanne Van Cooten, NOAA

Jun Oct Dec

First Meeting of DMAC Steering Committee.Teams and Outreach groups defined.

Workgroups and Teams present White Papers

Draft SummaryPlan Delivered

Public Review of Plan Begins

Draft Detailed Plan Delivered

Timeline for Plan Development

Mar

Airlieworkshop

Sep Jan

http://www.ocean.us.net/dacsc/index.jsp

The Vision

See “The Plan” paper …. Highlights only here

• IOOS data will be available through broad range of current and future computer applications, largely unencumbered by traditional barriers of data format, location, and size

• Users will be able to browse and compare any of the data through a standard Web browser

• For data suppliers, small and large, the effort to make data available will be minimized

The Vision (Continued)

• DMACS will be a “free market” of ocean science information -- easy to compare official IOOS products with outside results

• DMACS will engage the private sector in the development of value-added products for special user groups -- commercial fishermen, recreational sailors, divers, etc.

• Outreach mechanisms will ensure that the needs of users are recognized and acted upon.

Community Building Issues

The greatest challenges are in the area of community outreach and organizational behavior rather than in technology!

Challenges for– All levels of management– Technical staff (the implementers)– Users (the definers of “success”)

Community Building Issues

Required to succeed:The perceived benefits of participation must

exceed the costs!

– Ensure very low barriers to participation for both data suppliers and users

– Follow an evolutionary path– Sustain a program of vigorous outreach

Metadata management and search

Complete, consistent, searchable metadata– Metadata must be inseparable from data– Create/adopt standardized, shared

vocabularies (not as easy as it sounds!)– Supply training and tools to data providers– How to make a human-dominated (flawed)

approach more automated?– “Web portal” is just one way to do searches– How to do highly effective search in distributed

systems?

Components of DMAC System

Data Transport

“middleware” is the foundation

– Networking infrastructure for connecting heterogeneous clients and servers

– Data providers must translate from “native” formats and data bases (DCS must provide tools). Applications adapted to read from middleware.

Data Transport (Continued)

OPeNDAP (DODS) is an IOOS “pilot” activity– Discipline-neutral, low entry barrier– Can immediately begin building DCS– Must develop the “ocean data model”

(standards) in parallel

Parallel work on blending data push/data pull

Archival

Irreplaceable observations, data products of lasting value and associated metadata preserved in perpetuity– Accessible using DCS standards– Acquired at Archive Center using DCS standards

Archival

Issues– How to phase in new techniques? (“don’t look

back”?)– How to partition responsibilities among existing

facilities, new operational product centers and new facilities?

– Relationship between archive centers and “delayed-mode (climate) data assembly centers”

User Outreach, Products and Applications

User needs will drive how IOOS evolves– DCS already has an incipient IOOS User

Outreach Committee (Phil Mundy)– Most feedback focuses on needs for products –

not something that DCS controls– A mechanism for sharing the User Outreach

Committee among all components of IOOS?

Administration and Operations

The Plan will convene a WG to define the DCS Administration policies working closely with Ocean.US

• Day to day DCS operations• Monitoring system performance / correcting problems• Liaison with Product, Archival facilities and Observing

Subsystem• Liaison with regional systems• “help desk” • Assures compatibility with related national and international

data systems

Technical Issues• Scalable metadata management

• Version tracking

• Data mining

• Data volumes generated by satellite (and sonar) measurements

Blending “data push” and “data pull”

Comprehensive geo-spatial data model

Aggregation of distributed data to achieve seamless access between assembly centers and archives.

The End

http://tmap.pmel.noaa.gov/~hankin/OCEAN.US/USGOOS_VII