Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to...
-
Upload
caren-stanley -
Category
Documents
-
view
214 -
download
0
Transcript of Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to...
![Page 1: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/1.jpg)
Page 1
HDF-EOS Status, Related Tools and Issues
Abe Taaheri, Raytheon IIS2009 THG Briefing to ESDIS
![Page 2: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/2.jpg)
Page 2
TOOLKIT / HDF-EOS Support
• Maintenance support is funded on EMD Evolution through May 22, 2009
- Bridge contract until EODDIS Evolution & Development (EED) award in 2010
• Current support HDF-EOS 2 & 5 SDP/MTD Toolkit HDF-EOS plug-in for HDFView HDF-EOS2 to GeoTIFF Converter (HEG) Other Toolkit/HDF-EOS related software Toolkit/HDF-EOS/HEG helpdesk
![Page 3: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/3.jpg)
Page 3
ECS Support of HDF-EOS
• Both HDF4 and HDF5 based flavors of HDF-EOS are part of ECS baseline.
• OS’s: Solaris (9, 10), Irix6.5 (64, n32), Linux (including 64-bit Opteron and Itanium),
Mac OS X (Power PC and Mac Intel), Windows XP(MTD Toolkit, HDF-EOS2 & 5, HEG, HDFView)
• added full Cygwin support for next release
• Compilers: Fortran 77/90 & g77/pgf90 , C, C++, gcc, g++,Intel Fortran (SGI Altix), gfortran (Mac
Intel)
![Page 4: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/4.jpg)
Page 4
Downloadable Software
• SDP/MTD Toolkit (Science Data Processing Tools)
• HDF-EOS2 (HDF4 based)
• HDF-EOS5 (HDF5 based)
• heconvert (converts HDF4 - based Grid/Point/Swath structures to HDF-EOS 5 equivalents)
![Page 5: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/5.jpg)
Page 5
Downloadable Applications
• HEG
- GeoTIFF conversion
- Spatial subsetting, Field Subsetting
- reprojection
- stitching, sub-sampling and more ….
• DumpMetadata (Dump HDF-EOS metadata)
• HDFView Plug-in, Java-based browser (HDF4, HDF5, HDF-EOS 2 and 5 access)
• HE5View* (HDF5 - based browser)
• EOSView* (HDF4 - based browser)* Dropped support for OS updates on July 2006
![Page 6: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/6.jpg)
Page 6
Software Releases
Toolkit/HDF-EOS:
• Last release – March 2008– TOOLKIT 5.2.15 – HDF-EOS 2.15, HDF- EOS5.1.11
with HDF4.2r3, HDF5-1.6.7, Szip2.1
• Next release – May 2009– TOOLKIT 5.2.16 – HDF-EOS 2.16, HDF-EOS5.1.12
with new releases of HDF4 (2r4), 5 (1.8.3)
![Page 7: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/7.jpg)
Page 7
Software Releases (Cont.)
HEG (HDF-EOS2 to GeoTIFF Converter):
• Last release – August 2008
– Version 2.9
• Next release – May 2009
– Version 2.10
– Already have released Datapool version to the DAACs (Feb. 2009)
![Page 8: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/8.jpg)
Page 8
Software Releases (Cont.)
HDF-EOS Plug-in for HDFView:
• Last release – September 2008
– Version 2.4
• Next release – June 2009
– For HDFView Version 2.5
![Page 9: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/9.jpg)
Page 9
Software Releases (Cont.)
HECONVERT (HDF-EOS2 to HDF-EOS5 Converter):
• Last releases
– 2001, 2004 Version ?
– Oct. 2008 Version 2.0
DumpMetadata (ASCII Dump of HDF-EOS metadata)
• Last release
–Dec. 2008 Version 1.0
![Page 10: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/10.jpg)
Page 10
SDP Toolkit
• Science Data Production (SDP) Toolkit is a collection of tools used by
- Science software developers who produce
code to process instrument data.
- HDF-EOS users who need to• Access metadata in HDF-EOS files• Perform Time/Date conversion• Access Digital Elevation Model Data• Access Ancillary Data, such as Digital
Chartof the World database (DCW), Olson World Ecosystem files, DEM data files, etc.
![Page 11: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/11.jpg)
Page 11
SDP Toolkit (cont.)
• Tools provided in (SDP, MTD) Toolkit:- AA (Ancillary Data Access)- CBP (CELESTIAL Body Position)- CSC(Coordinate System Conversion)- CUC (Constant and Unit Conversions)- DEM (Digital Elevation Model Access)- EPH (Ephemeris Data Access)- GCT (Geo Coordinate Transformation)- IO (Input Output (File I/O))- MEM (Memory Management)- MET (Metadata Access)- PC (Process Control)- SMF (Status Message File (Error/Status))- TD (Time and Date Conversion)
![Page 12: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/12.jpg)
Page 12
What is HDF-EOS ?
• HDF-EOS is an extension to HDF which standardizes storage of and access to common Earth Sciences data structures
–By providing a single interface to data structures common to earth science
–By providing a container for EOS inventory, archive and product specific metadata
–By establishing a consistent relationship between geolocation and science data.
![Page 13: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/13.jpg)
Page 13
HDF-EOS 2
• HDF4 - based, storage format for EOS standard products.
• Used operationally by MODIS, MISR, ASTER, Landsat, AIRS and other EOS instruments
• Support for Grid/Point/Swath structures
• HDF4 and HDF-EOS 2 will be supported by THG/NASA for the fore-seeable future.
![Page 14: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/14.jpg)
Page 14
HDF-EOS 5
• Based on HDF5, a complete rewrite of HDF-EOS2 with a different user interface.
–First released in 2000.
–Format used by EOS Aura instruments
• Designed to ‘resemble’ HDF-EOS 2 to the maximum extent possible.
–Supports same data structures
–Added prefix ‘HE5_’ to HDF-EOS 2 functions.
![Page 15: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/15.jpg)
Page 15
HDFView
• HDFView: A Java-based browser from THG for HDF4 and HDF5 – based data.
• HDF-EOS plug-in extends HDFView to browse HDF-EOS data in both versions.
• The extension is a plug-in module and does not change THG HDFView functionality. All copy, save, editing, image display, etc. for HDF4 and 5 objects remains.
![Page 16: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/16.jpg)
Page 16
HDFView
• Identical look and feel when displaying HDF and HDF-EOS objects.
• Users can access a complete breakdown of HDF-EOS objects. e.g. grid information, projection info., dimensions, datafields, and attributes are displayed.
• Current Support: SUN, SGI, LINUX, Mac, WINDOWS platforms
![Page 17: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/17.jpg)
Page 17
Swath, Grid, and Point Objects
![Page 18: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/18.jpg)
Page 18
AMSR-E Swath
Tableview Features in Version 2.4
![Page 19: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/19.jpg)
Page 19
HEG: Utility that converts EOSDIS data from HDF-EOS format to common Geographical Information System (GIS)-compatible formats.
• Two versions available:–A downloadable desktop version –Access through NASA archive online storage (Data Pools)
– Implemented also in some prototypes:▪ Stand-alone Data Pool▪ OGC (Open Geospatial Consortium) for Web Services Chaining▪ DOWS (Deploy OGC Web Services) on the DAACs
HDF-EOS to GeoTIFF Converter (HEG)
![Page 20: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/20.jpg)
Page 20
HEG Functionality
• Conversion:
– HDF-EOS (Swath or Grid ) to a single-band or multi-band GeoTIFF’s
– HDF-EOS Swath to HDF-EOS Grid
– HDF-EOS Swath or Grid to generic Binary
– Metadata preservation/creation.
– Allows file selection from users local storage
![Page 21: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/21.jpg)
Page 21
HEG Functionality
• Subsetting (spatial, field, band)
• Stitching (mosaicing) + subsetting + Reprojecting
• Reprojection ( to UTM, PS, TM, STP, LCC, LAMAZ,
GEO, SIN, Albers)
• Subsampling (Subsample stacks)
• Running from Command Line (batch jobs)
or with Java-based GUI
![Page 22: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/22.jpg)
Page 22
HEG Functionality
• Currently supports MODIS, MISR, ASTER, AIRS and AMSR-E products on TERRA and AQUA (>110 products),
• Operable on Sun, SGI, Win, Linux, MAC
• Integrated into ECS Data Pool– Reduces the transfer time of HDF-EOS data sets (if subset is requested)
– Provides the end-user with the exact file required by their application
• User Interface– Portable, written in Java.
– Not dependent on COTS (eg. IDL).
![Page 23: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/23.jpg)
Page 23
Mt. Etna Eruption: MISR L1B Stitched Blocks
![Page 24: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/24.jpg)
Page 24
2004 Hurricane Francis, MODIS
MOD021KM, Multi-Band GeoTIFF, 3 bands: EV_1KM_RefSB, Bands 1,2,5
![Page 25: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/25.jpg)
Page 25
AST_L1B, 2003 California FiresGeo. projection, Multi-Band GeoTIFF,
(3 bands: Visible Near Infrared (VNIR))
Two Swaths Stitched …..
….. and Subsetted
![Page 26: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/26.jpg)
Page 26
TOOLKIT/HDF-EOS:
SZIP Insatallation: Default decoder,
Choice for encoder
Choice for configuring HDF4 with/without SZIP
Choice for disabling HDF4’s internal netCDF
Choice for installing Toolkit/HDF-EOS in 32 or 64-bit modes in 64-bit Linux platforms
(except for Itanium; no –m32 flag)
Recent Major Development/Fixes
![Page 27: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/27.jpg)
Page 27
Support for new releases of HDF4/HDF5/SZIP
(HDF4.2r4, HDF5-1.8.X, SZIP-2.1)
Auto Configuration capability (like HDF) for
HDF-EOS2 and HDF-EOS5
Support for Mac Intel platform. Support for CYGWIN
Recent Major Development/Fixes
![Page 28: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/28.jpg)
Page 28
Added APIs GDij2ll and GDll2ij to HDF-EOS2 and HDF-EOS5
Added HDF_EOS 2 & 5 utilities for dumping grid lat/lon into ASCII or binary files
Bug fixes such as:- HDFEOS5: Attribute data type inquiry (code returned not useful for reuse)
- HDFEOS5: Tiling/comp. consistency between dataset and structmetadata
- HDF-EOS2: Correct diminfo for a field (value in structmetadata is wrong?)
- HDF-EOS5: wrong datatype info for INT8 data (C code)
- Toolkit: Problem in pccheck utility( hangs if a blank space at a line start )
- Fix for ODL core dump (too many file names for Input Pointer)
Recent Major Development/Fixes
![Page 29: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/29.jpg)
Page 29
HDF-EOS Plug-in for HDFView:
Current Support: SUN, LINUX, WINDOWS, and MAC (Power and Intel)
– Auto installation of plug-in into HDFView running self-extracting archives.
– Adding to plug-in the fixes and enhancement that went into HDFView2.4 and 2.5
– Display image for selected pixels in the viewed tables
Recent Major Development/Fixes
![Page 30: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/30.jpg)
Page 30
– Display Lat/Lon for a selected pixel in the
data field tables
– Full Mac OSX support
– Improved Help Window
– New icons for activating Lat/Lon display for
Swath & Grid
– Format conversion: HDF-EOS2 -> HDF-EOS5
– Correction for compression and tiling values for grid fields
Recent Major Development/Fixes
![Page 31: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/31.jpg)
Page 31
HEG: Product Support:
AMSR-E:- Land products (CEA Projection added)- AMSR-E 5 day Snow, and Monthly Snow- Weekly Ocean, and Monthly Ocean- Daily 12 Km, and Daily 6 Km Sea Ice- L2 Rain, and Rain Grid- ASTR_07XT, ASTR_09XT, ASTR_L1BE,
MOD29E1D- More new versions and products tested. Will
update Users’ Guide and HEG web site
Recent Major Development/Fixes
![Page 32: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/32.jpg)
Page 32
• Stitching fixes:
− MOD14A1 3-D fields (band selection)
− TM grids
• Multi-band geotif/HDF-EOS in Datapool
• Other Fixes:
− Interpolation correction with AIRS products
containing fillvalue in the data fields
− Pixel size unit consistency in conversion and stitch GUIs.
Recent Major Development/Fixes
![Page 33: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/33.jpg)
Page 33
− Correction for droping a column when
stitching MODIS SIN tiles.
− Fixes for processing with default pixel size instead of ignoring it for Geographic projection
− Spatial subsetting fix for global AMSR-E garanules
− Fix for ignoring re-projection of type NN interpolation after stitching
− support pixel resampling for gridded data
Recent Major Development/Fixes
![Page 34: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/34.jpg)
Page 34
Future Plans
TOOLKIT/HDF-EOS:
• Auto Configuration for TOOLKIT
• XML output for Metadata in TOOLKIT
• Update of DEM land-sea mask
• Support for HDF5-1.8.X
• Validation tool for HDF-EOS5 files
![Page 35: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/35.jpg)
Page 35
Future Plans (cont.)
HDF-EOS Plug-in for HDFView:• More functionalities :
- Ability to modify input HDF-EOS2 granules. - Ability to cut/paste objects, modify/delete attributes.- Ability to create new grid/swath
HEG:- Support for AMSR-E Level 3 Land (CEA projection)- New MRT (MODIS Reprojection Tool) release updates- Support HEG for MAC Intel
![Page 36: Page 1 HDF-EOS Status, Related Tools and Issues Abe Taaheri, Raytheon IIS 2009 THG Briefing to ESDIS.](https://reader035.fdocuments.in/reader035/viewer/2022070415/5697bf8b1a28abf838c8b384/html5/thumbnails/36.jpg)
Page 36
Availability
• Access to libraries, applications, Documentation:
TOOLKITTOOLKIT:: http://newsroom.gsfc.nasa.gov/sdptoolkit/toolkit.htmlhttp://newsroom.gsfc.nasa.gov/sdptoolkit/toolkit.html
HDFViewHDFView::http://newsroom.gsfc.nasa.gov/sdptoolkit/HDFView/http://newsroom.gsfc.nasa.gov/sdptoolkit/HDFView/
HDFView_hdfeos_plugin.htmlHDFView_hdfeos_plugin.html
HEGHEG: : http://newsroom.gsfc.nasa.gov/sdptoolkit/HEG/HEGHome.htmlhttp://newsroom.gsfc.nasa.gov/sdptoolkit/HEG/HEGHome.html
• Email–[email protected]–[email protected]