Important ESDIS 2009 tasks review

Important ESDIS 2009 tasks review

Kent Yang, Mike Folk

The HDF Group

April 1st, 2010

04/20/23 1Annual briefing to ESDIS

Outline

• HDF-EOS Website• HDF OPeNDAP support• HDF-EOS5 Augmentation tool


HDF-EOS Web Sitehttp://hdfeos.org


Main page: http://hdfeos.org

• Reorganize the overall structure

• Update the main page

04/20/23 404/20/23 4Annual briefing to ESDIS

http://hdfeos.org/

Help: http://www.hdfeos.org/help

04/20/23 504/20/23 5Annual briefing to ESDIS

http://www.hdfeos.org/help

Examples

• Added a new “Examples” section

- C, Fortran, IDL and MatLab examples to access HDF-EOS2 and HDF-EOS5 Grid and Swath files

- Examples on how to use tools such as HEG, GrADS, NCL, Using IDV via OPeNDAP to access HDF-EOS 2 and HDF-EOS5 data


Software

• Evaluated 35 tools and libraries

• Added 11 tools and libraries

• Added detailed descriptions

November 3-5, 2009 7HDF/HDF-EOS Workshop XIII04/20/23 7Annual briefing to ESDIS

November 3-5, 2009 HDF/HDF-EOS Workshop XIII 8

Forum


HDF-EOS website statistics

Current Period: Nov. 1st 2009 – Feb. 28th 2010

Previous Period: July 4th 2009 – Oct. 31st 2009


Definitions

Visits – The total number of visits to the website.Pageviews – The total number of pages viewed on the website. Repeated views of a single page are counted.

Pages/Visit – The average number of page views per visitor.Bounce Rate – The percentage of single-page visits. (Total number of visits viewing one page) / (Total number of visits)

Avg. Time on Site – The average visit duration.

% New visits – The percentage of visits by people who had never visited the website before.

Organic – Web traffic which comes from unpaid listing at search engines or directories.


HDF-EOS Website Statistics Comparison

Jul Aug SepOct


Month by Month Visits Comparison


Traffic Sources

- Visitors who typed in the URL directly

-Visitors who clicked a link from another website

- Visitors who used a search engine


Pageviews


Top Pages Viewed


Current work


Current Work

• Collect sample files for NASA HDF4/HDF-EOS2 products

• Provide special HDF-EOS description for each sample file

• Examine the access of each sample file with widely used tools

• Provide the codes and the plots generated from the tools for each sample file


Tools to access the HDF data

• NCL

• MatLab

• IDL


04/20/23 24

Datapool Example


HDF OPeNDAP Work Update


HDF5 –OPeNDAP Handler

• Four official releases from OPeNDAP website• The latest release date is February 2010• Add the support for HDF-EOS5 Aura swath files• Aura MLS, OMI, HIRDLS files are served via

OPeNDAP• URL:

http://opendap.org/download/hdf5_handler.html


http://opendap.org/download/hdf5_handler.html

http://hdfeos.org/software/ncl_opendap_mls.php

Access Aura MLS data via OPeNDAP


http://hdfeos.org/software/ncl_opendap_mls.php

HDF4 OPeNDAP Handler


Phase 1 work review

• Time: April 2009 – October 2009• Enhanced the original HDF4-OPeNDAP handler• Some HDF-EOS2 data can be served via

OPeNDAP• Provided a version to GES DISC


Phase 2 work

• Time: November 2009 – July 2010• Goal: Enhance the HDF4-OPeNDAP handler so

that more NASA HDF-EOS2 and HDF4 data can be served via OPeNDAP


Phase 2 work

• Comprehensive examinations of all NASA HDF-EOS2 data products

- NASA Data pool in the previous slides• Come up with the re-design based on the

examinations• Re-implement the HDF-EOS2 module of HDF4-

OPeNDAP handler• Add the support for handling some NASA pure

HDF4 data


Phase 2 - Current Status

• We finished the examination of NASA HDF-EOS2 products

• We finished the design note of HDF-EOS2 module of HDF4-OPeNDAP handler

• We finished implementing 90% features of the HDF-EOS2 module

• We provided a customized version for GES DISC to test


Phase 2 – Current Work Highlights

• Focus on making the data accessible by Java-based OPeNDAP clients

• Strictly Follow CF/COARD conventions• Handle different HDF-EOS2 cases


Phase 2 work plan

• Time: Now to July 14th, 2010• Finish all features for the HDF-EOS2 module• Handle Hybrid HDF-EOS2 files• Handle some NASA HDF4 files


HDF-EOS5 augmentation tool


HDF-EOS5 and NetCDF-4

• Enabling NetCDF4 to access HDF-EOS5 data Conversion Augmentation

• One file can be used for both EOS5 and NetCDF-4.

• Note that EOS5 users are not affected at all.

HDF5

HDF-EOS5HDF-EOS5

file HDF5

NetCDF4NetCDF-4

file

AugmentedHDF-EOS5

file

Augmentation


Goals

• Augmented file must be readable through NetCDF-4 API as well as HDF-EOS5 API

• How? Read info in the HDF-EOS5 part.

Transform it to conform to NetCDF-4 model. .

Append the new info.

+ =


Past Work Review

Grid Swath

OMI L2GOMI L3GTESMLSHIRDLS

OMIMLSHIRDLSTES

• Not all Aura HDF-EOS5 files use projection for storing dimension values.

• For example, OMI L2G grid files store lat/lon in data fields.

• TES grid files use both ways.

Collected and Examined different HDF-EOS5 data


Past Work Review

• To follow the netCDF4 data model, we need users to provide the information about the dimension fields.

• MLS and HIRDLS data are vertical profilers. The dimension fields are Pressure and Time.

• We need similar information for other instruments.

Provided a customized version for HIRDLS team


Future Work

• Handle other cases:

OMI L2G,future HIRDLS/MLS grid• Design a general input file for insufficient

information• Continue testing with NetCDF-4 snapshots• Consistency checker

If one modifies dim info in an augmented file, file becomes inconsistent.

• Testing, Documentation


Thank You!


Acknowledgements

This work was supported by the Cooperative Agreement with the National Aeronautics and Space Administration (NASA) under NASA grant NNX06AC83A and NNX08A077A.

Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of NASA.


Questions/comments?


Backup slides


Role of HDF4-OPeNDAP handler

OPeNDAP Clients OPeNDAP HDF-EOS2

HDF4-OPeNDAP handler


Important ESDIS 2009 tasks review

Documents

Transcript of Important ESDIS 2009 tasks review