Post on 18-Dec-2015
NOAA Earth Information Service
(NEIS)
OverviewGSD Technical Seminar
February 28th, 2012
ESRL Global Systems Division
NEIS Agenda
IntroductionEvaluation and FindingsConceptActivities and Next StepsDemonstration
ESRL Global Systems Division
NEIS Background
GSD Director Discretionary Funds (DDF) Proposal
NEIS Requirement• Fast access to any data, any time, right now, on any platform.
NEIS Group Objectives• Build upon NEIS work in the past
• Assess GSD capabilities pertinent to the concept of NEIS.
• Evaluate existing, developmental technology, and solutions relevant to concept.
• Evaluate envisioned needs of primary users.
• Provide recommendation of how to best use GSD resources to contribute to a National capability.
ESRL Global Systems Division
NEIS Team
Research Team
Jebb Stewart - LeadChris MacDermaid Woody RobertsDarien Davis Jeff SmithMike Romberg Paul Schultz
Development Team
Julien Lynge Jeff SmithChris MacDermaid Jebb StewartRandy Pierce
ESRL Global Systems Division
NEIS History
January 18, 2007, Roberts, et al. A OneNOAA Concept Prototype for Data Viewing, 23rd IIPS.
March 10, 2009, Hansen, et al. Earth Information Services, GSD Technical Review on GFE and Other Activities.
ESRL Global Systems Division
NEIS Evaluation
Interviewed/Discussed concepts with variety of groups across NOAA Line Offices. More than 15 individuals.
Questions Asked:How are people using your data?How are you using NOAA data yourself?What tools/systems/services/standards are you using?What related recommendations/lessons learned exist? What obstacles or roadblocks have you encountered with using/disseminating NOAA data?
Listened for items to help facilitate projects or help NOAA data become more interoperable.
ESRL Global Systems Division
Interviews
Person Role/Responsibility
Michele Jacobi, NOS ORR/ERMA. Just finished OAR detail with GoogleOceans, part of DWH ERMA response team
Karen Sender NMFS inPort
Katie Fisher NOS Center for Operational Oceanographic Products and ServicesHarmful Algae Bloom POC
Dan Pisut NESDIS NVL
Chris Ortiz Space Weather Center
Charley Alexander IOOS
Tiffany Vance PMELHPCC
Glen Rutledge NOMADS
Ted Haberman NGDC
Tony Lavoi CMSP, CSC, NOAA GIS
Tim Haverland NMFS GIS group
Cecelia DeLuca, OAR OAR NESII
Steve Hankin PMEL
John McDonough OER
Mike Little NASACathy Smith OAR/PSD
Jeff de La Beaujardiere NESDIS/OSD/TPIO also DMIT
NEIS System Evaluation
ESRL Global Systems Division
xkcd.com
Part of the evaluation looked at existing systems and standards including:THREDDS LAS ERDDAPGoogle Earth NASA World Wind GeoPortalMERMAid REVERB iRODSGrads Data Server More….
Lot of Great Technology, which can be leveraged, no ‘one’ solution.
NEIS Committee andGroup Involvement
ESRL Global Systems Division
GSD is aware of many ongoing activities
We have active membership in a variety of committees and groups including:OGC Standards CommitteesEDMCDMITUAFNextGen/NNEW programFIM/WRF ModelingGIS Committee
GSD Central Facility has vast experience in working with data and standards
NEIS Findings
• Existing metadata systems contain stale or obsolete information and are not useful Search for common term like wind returns hundreds of results. Expertise required to understand what metadata you are looking for.
• Many “Stovepipes” but no ability to combine/integrate data from different disciplines (Ecological, Meteorological, Oceanographic, Fisheries, etc...)
• Data sets are large and getting bigger. Future data sets will be difficult if not impossible to move around network to consumer applications.
• Cross data source data processing service does not exist. Some data processing services exist but usually only for data available from that particular service.
• Feedback (broken links, data usage, etc…) to data providers and consumers is non-existent.
Many more…
ESRL Global Systems Division
11
Biological
ChemicalPhysical
• Biological, Chemical, and Physical data is all interrelated
• NEIS allows data integration in 4D time and space, providing a scientific tool for everything NOAA does professionally and the world NOAA affects.
Carbon TrackerFIM
Data Integration
ESRL Global Systems Division
12ESRL Global Systems Division
NOAA Vision - 2020
Ability to discover, integrate, understand all NOAA data (past, present, and future) from homepage.
TerraViz
13ESRL Global Systems Division
NOAA Vision - 2020
• Most of NOAA's environmental data will be accessible via web services and will be described by good metadata
• A central and intelligent search engine, that has harvested the metadata of NOAA's datasets, will exist to find data
• NEIS server side analytics/processing will enable subsetting, aggregating, diff'ing, and other operations to be performed on data
• Data will be efficiently streamed to a variety of platforms and clients, including mobile devices
• TerraViz will be a general purpose client running on multiple platforms capable of searching for NEIS data and visualizing it in novel ways
• Framework is extensible, other groups can add knowledge and services.
NEIS – Concept Components
Build new Graphical User Interface system integrating and leveraging new and emerging technologies to meet NEIS goal ‘any data, any
location, any platform, now’
• Perform processing within cloud environment and with high speed connectivity to data sources, taking advantage of large processing power within clouds.
• Send graphics and server side processed/rendered/streamed data to GUI, improving bandwidth utilization.
• Take advantage of fast networking to make remote requests and processing appear like local application.
• Similar to how the new Amazon Silk Browser works.
Looking to the Future
ESRL Global Systems Division
Improve existing Metadata services making information useful.
• Improve searching by understanding what data means and providing improved filtering capability similar to how airline or hotel type search engines work.
• Incentivize/encourage people to use proper Metadata.
• Create a Metadata Dashboard to:
oGather information from users on relevance, ratings, usage patterns, search key words used (ie crowd sourcing).
This information can be used to determine what users are searching for, how they are finding data, what data they are not finding, etc...
oProvide feedback on broken links, service uptime
oProvide feedback on adherence to standards (Services, Metadata).
oProvide feedback to users of similar or related data sets.
Looking to the Future – Part 2
ESRL Global Systems Division
ESRL Global Systems Division
ESRL Global Systems Division
Impacts
• NOAA data ready for action. Services model facilitates agile
response to events. Services can be combined or reused quickly
• Any data available through NEIS system can be operated on or combined with other data. Integrated standardized formats and access.
• New and Existing systems have access to wide variety of NOAA data. Any new data added, easy incorporated with minimal to no changes required.
ESRL Global Systems Division
NEIS Use Case – Deepwater Horizon
• There was no way to pull disparate data sets into one display in order to track the impacts of the oil spill
oOcean currents
oEcosystems
oWeather
oLocation of booms, ships, and other response equipment
• Many GIS developer labor hours spent working on converting and formatting data to make it available.
Are We Prepared?
What would happen if another hurricane or other disaster occurred?
With a framework such as this:
• Improved searching capability would allow those in need to quickly and easily find pertinent data to the situation.
• All data available through framework would be readily accessible to be integrated across 4D (time and space).
• New data that is properly documented could be added to services and become immediately available to the framework of services. Spin up time for new data reduced dramatically.
• Because framework exists as a collection of services, any changes or modification can be completed quickly without taking entire system down.
ESRL Global Systems Division
Demonstration
ESRL Global Systems Division
Prototype Demonstration Background
24ESRL Global Systems Division
What is TerraViz?• A system developed from the ground up by GSD based on Unity3D.• There are no licensing restrictions in distributing the TerraViz executable.• Unity3D is a well supported commercial game and data visualization engine
with 750,000 active users • Unity/C# is designed for 3D development, making it easy to create multiple
views, create 3D meshes representing environmental objects, etc.
• Develop once, then will run under Windows, Mac, web browsers, iPhones, iPads, Android devices, and even game systems (Wii, Xbox 360, etc.)
• We can modify the user interface and add any feature we need.• TerraViz Handles millions of polygons and performs quickly
TerraViz is one possible visualization tool built to develop, test, and showcase NEIS framework.
25
Google Earth, works great for certain applications and static products, however:
• Handles ~10,000 polygons before slowing to a crawl.• Closed system (Google Earth source code not available) so we can’t add needed
features.• Google Earth plug-in works on Windows and Mac 32-bit browsers only (won’t run
on iPhones, iPads, or Android devices.)• Developing GE plug-in user interfaces in browser-dependent Javascript is
tedious.• Very Limited integrated data discovery or browsing, most uses require a
separate application.• Limited animation/data interaction support.• Limited support for non-static data (ie Time Series)
Why not Google Earth