HYDROLOGIC METADATA CATALOG AND SEMANTIC SEARCH SERVICES IN CUAHSI HIS
CUAHSI - Consortium of Universities for the Advancement of Hydrologic Science, Inc Building...
-
Upload
bruce-shaw -
Category
Documents
-
view
218 -
download
0
Transcript of CUAHSI - Consortium of Universities for the Advancement of Hydrologic Science, Inc Building...
CUAHSI - Consortium of Universities for the Advancement of Hydrologic Science, Inc
Building Community for Hydrologic Science
CUAHSI Hydrologic Information System (HIS) Institutions
Supporting CUAHSI using Hydrologic Information Systems
Building on the strengths of the experts and institutionswe have to support the hydrologic science community
CUAHSI- HIS Goal
Support innovation and advancement in the hydrologic sciences by providing better access to the hydrologic data and information systems technology needed to formulate and test new hydrologic science research hypotheses.
Hydrologic Information System
Analysis, Modeling, Hypothesis testing
Hydrologic Database
Relation to the Science Plan
• Interfaces between hydrologic processes
• Process behavior across spatial scales
• Ecology – hydrology• Hydrologic prediction• Hydrologic science
and water resources management
Integration
Science Plan Goals:
What does the community need?
• CUAHSI Survey– How do you use hydrologic information in your
research?– What kind of infrastructure and services could
CUAHSI provide that would help you?– What new research questions could you then
address?
How do you use hydrologic information?
• “To provide a context in which to interpret my own field data, in a regional, and/or a historical context” – John Hermance, Brown University
• “I am interested in how water moves through mountain blocks, which requires that fluxes be defined at the input boundaries” – John Wilson, New Mexico Tech
Study area
Regional context
How do you use hydrologic information?
• “The type of hydrologic data I use relate to site characterization and solute transport at either actual remediation sites or at one of the well-known experimental field sites (Borden, Cape Cod)” – Albert Valocchi, University of Illinois
How do you use hydrologic information?
• “I use the following types of data for research and teaching: stream discharge and stage from USGS gaging stations, rainfall from NOAA, water levels in wells from USGS and state surveys, chemical concentrations in stream sediments from NURE chemical atlas, chemical concentrations in wells from local sources, radon gas concentrations in air from local sources, ET data, various photographs, satellite images and maps” – Larry Murdoch, Clemson University
How can CUAHSI Infrastructure and Services help you?
• “When we need to compile data for calculations, we have to consult a broad range of web servers with interfaces of varying quality to extract these data. If HIS could provide some sort of front end that would at least make it possible to quickly identify links to the various online databases, that in itself would be a great service” – Carolyn Ruppel, Georgia Tech
Development of hydrologic data portal
How can CUAHSI Infrastructure and Services help you?
• “The HIS effort should also work to evaluate and promote an appropriate data format to minimize reformatting efforts. Development of simple generic data management software tools would seem helpful” – David Brown, California State University, Chico
Development of hydrologic data standards and tools
How can CUAHSI Infrastructure and Services help you?
• “Need to be able to go to the top of the each database column, click on it and a window pops up and describes the data, explains the units, and provides an explanation of the data. Most databases don’t do this” – Brenda Ekwurzel, University of Arizona
Development of hydrologic metadata
New Questions….Scaling
• “The greatest opportunity to me is the ability to examine scaling effects” – Robyn Hannigan, Arkansas State University
• “Integration of data across systems with varying scales, formats and frequency” – Michael Barber, Washington State University
• “How does a single measurement relate to the large scale properties of the medium?” -- Ty Ferré, University of Arizona
Survey Results: Diversity
• The survey yielded a wide spectrum of interests in variables, processes, space and time scales– Processes: hydrologic, above and below ground, bio-
geochemical, flow and transport
– Variables: all quantities affected by hydrological processes
– Spatial scales: micro/pore to continental
– Temporal scales : seconds to years/decadal
Survey Results: Commonality
Better access to a large volume and variety of high quality hydrologic data through the internet
Access to visualization tools and data analysis software to inspect and assess data prior to download
Standardized datasets for hydrologic stores and fluxes across the United States that can be used as benchmark datasets for both individual and community model development
A Personal Vision
• Coupling hydrologic elements in the landscape
• Integration of geospatial and temporal water resources information
• Integration of information and modeling
Water Balance in the Landscape
Atmospheric Water
Soil Water Surface Water
Groundwater
Precipitation
Evaporation
Runoff
Recharge
Baseflow
Atmospheric Water
Soil Water Surface Water
Groundwater
Vertical and Horizontal Water Balance
/t= P - E - T - R
Soil Moisture MicrowaveAMSR, SMOS, HYDROS*
P Precipitation MicrowaveTRMM/TMI, SSM/I, GPM
T Transpiration/NDVI Visible/NIRMODIS, AVHRR, GLI, VCL
R Runoff/River Level LaserHYDRASAT*, TOPEX
E Evaporation/Surface Humidity Infrared/MicrowaveAIRS/AMSU
Tq
AIRS/AMSU
P
R
Rn RadiationShortwave GOESLongwave AIRS/AMSU
* Planning Phase
The land surfacewater and energybudgets are linkedvia evapotranspiration
H, G Surface TemperatureAIRS, AVHRR, MODIS
CLOSING THE TERRESTRIAL WATER BUDGET USING REMOTE SENSING
Rn + H + LE + G = 0Water BalanceEnergy Balance
Clouds GOESWater Vapor (LE)AIRS/AMSU
Groundwater flux
TE
Water table
W/t = E + T - P - div QAtmospheric Water Balance
Decoupling and connecting hydrologic systems
Source: http://www.acnatsci.org/research/anserc/mercurybiogeochemistry.htm
Space-Time Scales and Hydrologic Processes
Hydrologic InformationContinuum
Past
Present
Future
Time
Space
Point ContinuousSurface
AreaLine
Hydrologic Data in Space and Time
20 Two-digit Water Resource Regions
18 in Continental US + Alaska + Hawaii-Guam-Puerto Rico
Average area = 450,000 km2
6-digit Basins
349 basins in the continental US Average Area is 23,000 km2
Gage at outlet for 6-digit basins
A goal of the National Streamflow Information Program
8-digit HUC Subbasins
2156 units in the continental USAverage area = 3700 km2
Coupling Spatial and Temporal Data
1 hour Nexrad Rainfallon the Guadalupe Basin
October 13, 2001
EDNA Catchments
Average area = 7.5 km2
EDNA = Elevation Derivatives forNational Application
Intersection of Nexrad and Catchments
Atmospheric Water
Soil Water Surface Water
Surface Water Balance
Modflow Groundwater Cells
Average area = 2.6 km2
Modflow Model of theEdwards-Trinity
aquifer
Intersection of Nexrad and Modflow Cells
Atmospheric Water
Groundwater
Groundwater Balance
Intersection of Nexrad, Catchments and Modflow
Atmospheric Water
Soil Water Surface Water
Groundwater
IdeasScience Plan
Data
(time series and geospatial, in-situ and remotely sensed, physical, chemical and
biological hydrologic data sets)
Information Technology
(improved data storage, access, visualization and modeling tools)
Information Synthesis(data synthesis and standardization,
prototype system development, hypothesis testing)
Area of Emphasis in CUAHSI HIS
Hydrologic Data Access Center
Hydrologic Scientists
Hydrologic InformationScience Center
Hydrologic InformationTechnology Program
HIS Program Components
Hydrologic Data Access Center
• Assessment of User Requirements
• Data identification and interpretation
• Data archiving, query and provision
• Hydrologic Internet Data Distribution
The mission of the HDAC is to facilitate the CUAHSI hydrologic researchers with access to the data they need, and to provide information tools to analyze the data.
Program Elements:
HDAC Operation
Objectives of HDAC
Serve the user community by providing the following services :
• Online access to data (pull or push) + analysis and viz
• Permit submission of data
• Conduct PR and disseminate HIS information (newsletter)solicit feedback, be the “window” to user community
• Generator of “added-value” data products
Program elements of HDAC
• Query and access data elsewhereUSGS, NCDC, IRI, NOAA, etc
• Provide access to added-value data within HDAC conceptNEXRAD interpreted, ArcHydro, Benchmark Datasets, etc
• Allow data being stored within HDAC repositoryInstitutions, individuals, research groups looking fora place to make their newly generated data public
• Create “Site” through which all access is ensured, data,HIS (CUAHSI) mission, HIS news, Tech-support, drop-box,=> visibility, PR-activities
Central HDAC“one-stop-shop”Web-portal: query, retrieve, analyze, download
Data Center
UCAR
Data Center
IRI
Data Center
USGS
Data Center
NCDC
1) Data elsewhere
Tasks to tackle
Q. How do you get server access? A. Need to talk to data centers, development of protocols difficult and time consuming task => early start
Q. Once you are there how do you query?A. Need common data description, mapping between various
descriptions => Hydrologic Metadata catalogue
Q. How do you retrieve and in what format?A. Need in/free/out format converters for data sets residing elsewhere.
Central HDAC
Back-upMirror siteSoftware +Tech Support
Sub-HDAC
Paleo-FloodData
Sub-HDAC
Bench-DataCollection
Sub-HDAC
WR ModelData
Sub-HDAC
SatelliteData
Sub-HDAC
NEXRADData
2) New internal Data
Advantages of distributed approach
Q. Why distributed? A. Idea: build on the knowledge and resources of individual
institutions who have experience in particular areas.
Q. Doesn’t this mean more resources needed?A. Actually less, because no BIG facility is needed. Needed
staff is easier to find at location. Systems are already setupand can easily be expanded to accommodate small extra.
Q. Doesn’t this mean impossible communication ?A. No. Internet makes it possible. Data remains at sub-centers only Metadata information is passed on. Data is retrieved via URL => sub-center task is manageable.
Data Collectors wanting to give up dataHDAC becomes physical data center
CUAHSI-Observatories
Data sets generated through federal funds
3) New external Data
• Closely parallels a similar service provided to the atmospheric science community by Unidata
• Real-time streaming of data describing the hydrologic condition of the nation
Hydrologic Internet Data Distribution
Unidata Internet Data Distribution (IDD)with Multiple Sources (Injecting 17 Gigabytes per Day)
Using LDM software for instant data relaying, ~160 institutions cooperate to acquire a wide range of real-time, global, atmospheric & oceanic observations, model outputs, remotely sensed images..., in a coordinated community effort.
Source
LDM
Source
Source
LDM LDM
LDMLDM
LDM LDM
LDM
LDM
Internet
Tasks to tackle
Q. How to deposit new data? A. Web-portal has Metadata-editor, easy fill out forms online electronic submission and automatic storage in RDBMS
Q. How about QA/QC?A. Controlled through collector (show proof) or HDAC
Q. How do you store and retrieve in what format?A. Storage can be any format (least amount of space => binary) Just like any other data : Retrieval in any desired format
Q. What about the ownership?A. Needs to be worked out, donations only? Tit for tat? $$$?
Governance of HDAC
Advisory Committee (possibly all HIS partner Universities)External User CommitteePart-Time Acting Director
Budget
Staff for 1 + 5 centers6 senior personnel 12 grad students, programmers, tecs
Hardware reqs12 high end PC3 low end serverstorage + backup + mirror-sites
Hydrologic Information Technology Program
• Hydrologic Information Toolkit
• Data Model for Hydrologic Science
• Hydrologic Data Assimilation System
• Community Hydrology Modeling System
Program Elements:
Hydrologic Data Assimilation
Hydrologic Data Models
Feature
Waterbody
HydroIDHydroCodeFTypeNameAreaSqKmJunctionID
HydroPoint
HydroIDHydroCodeFTypeNameJunctionID
Watershed
HydroIDHydroCodeDrainIDAreaSqKmJunctionIDNextDownID
ComplexEdgeFeature
EdgeType
Flowline
Shoreline
HydroEdge
HydroIDHydroCodeReachCodeNameLengthKmLengthDownFlowDirFTypeEdgeTypeEnabled
SimpleJunctionFeature
1HydroJunction
HydroIDHydroCodeNextDownIDLengthDownDrainAreaFTypeEnabledAncillaryRole
*
1
*
HydroNetwork
*
HydroJunction
HydroIDHydroCodeNextDownIDLengthDownDrainAreaFTypeEnabledAncillaryRole
HydroJunction
HydroIDHydroCodeNextDownIDLengthDownDrainAreaFTypeEnabledAncillaryRole
• Standardized frameworks for integrating hydrologic data
• Provide support for new HIS developments
• Encourage the creation of a larger group of “Hydroinformaticists”, e.g. not only the collaborationbetween Hydrologists and Computer Scientists but the education of a new “species”.
• Respond to HDAC/HISC needs by soliciting RFPs
Objectives of HITP
• Build Hydrologic Metadata CatalogueServes as a bridge for the HDAC centers, data submissionsstandards, extremely important, HIGH PRIORITY!
• Data Assimilation and Visualization Toolscreates added value products through assimilation, analysis
• Community ModelSet of models to be used by community, provision of benchmark data sets, also high value for post-BS education
Program elements of HITP
Iowa Drexel
Florida
AlabamaOklahoma
Columbia
Illinois
Utah State
Washington
Texas-Austin
Cal-Berkeley
Virginia Tech
South Carolina
HITP
MetadataModel
HITP
DataAssimilation
HITP
DataAna/Viz
HITP
CommunityModel
HITP
TerrainModel
HITP
Land/Atmos
Possible HITP Focal Points
HITP
Data Library
Governance of HITP
The HISC Advisory Committee will guide the efforts and identify additional future elements of the HITP program.
Budget
Staff for 4 programs4-6 senior personnel 8 grad students, programmers, techs
Hardware reqs8 high end PCsoftware licences
Hydrologic Information Science Center
• Advanced prototypes• Metadata standards
(Hydrologic XML)• Info science initiatives• Computational
initiatives• Benchmark datasets• Support services
Program Elements:
This is the driver of intellectual development integrating information and hydrologic science
Benchmark Datasets
• Standardized data for verifying hydrologic models
• Datasets for fluxes at hydrologic interfaces– Precipitation
– Evaporation
– Recharge
– Streamflow
Hydrologic Observing System
• An integrated system of data and models that permits hydrologic analysis at any scale of space and time at any location
• Includes paleoflood records and data reconstruction in past centuries
• Link the science agenda of CUAHSI to specificinformation sources and derived data products.
• Facilitate the development of methodologies for the creation,assessment, and use of information (hydrologic or other) ina variety of hydrologic contexts.
Objectives of HISC
• Act as steering (brainstorming) venue to move HIS to thenext level, identify future areas of activity.
• Organize workshops and meetings both within HIS and community participation.
• Work out guidelines and criteria that can be used to formulate RFP for the creation and location of HDACand HISC.
Program elements of HISC
Governance
Advisory Committee (HIS Partner Universities)Part-Time Chairperson (current HIS chairperson?)
Budget
Location
The center may be a virtual one during the first two years.
Timeline
• Distributed approach allows for a quick setup as many institutions are ready to go based on existing research efforts• Scope is limited to a few crucial topics that have been identified as “relatively easy to implement” (results after 2 yrs.)• Long term issues or projects will be identified and, if possible, started otherwise be identified as future scope• Hard- and Software reqs are limited in scope and easily (quick) obtainable
Implementation
Two crucial issues:a) Timeline (in view of two year initial project duration)b) Staffing (where to get qualified personnel quickly?)
Staffing
• Distributed approach allows for a quick staffing because expertise is available in sufficient numbers at respective locations • Because scope is limited numbers needed are small and therefore manageable• The involvement of academic senior personnel and graduate students (plus a few post-docs) and very few full-time staff results in small personnel costs without long term commitments
Two crucial issues:a) Timeline (in view of two year initial project duration)b) Staffing (where to get qualified personnel quickly?)
Implementation
Hydrologic Data Access Center
Hydrologic Scientists
Hydrologic InformationScience Center
Hydrologic InformationTechnology Program
Keys for Success of CUAHSI HIS
InnovationDiscipline
Responsiveness Participation