HDF-EOS Tools
-
Upload
the-hdf-eos-tools-and-information-center -
Category
Technology
-
view
276 -
download
6
description
Transcript of HDF-EOS Tools
![Page 1: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/1.jpg)
Page 1
HDF-EOS ToolsHDF-EOS Tools
Abe Taaheri, Raytheon IISESIP Meeting
Chapel Hill, NCJuly 9, 2013
![Page 2: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/2.jpg)
Page 2
Outline
![Page 3: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/3.jpg)
Page 3
Overview
• The HDF has been selected by the EOSDIS Project as the format of choice for standard product distribution of EOS data
• Under NASA contracts (ECS, EMD, EEB, EED) we developed HDF-EOS, and performed other development and maintenance activities
![Page 4: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/4.jpg)
Page 4
Overview
– Libraries HDF-EOS 2 & 5 SDP & MTD Toolkits
– Tools HDF-EOS2 to GeoTIFF Converter (HEG) HDF-EOS plug-in for THG HDFView HDF-EOS2 to HDF-EOS5 conversion
– User support Helpdesk for Toolkit/HDF-EOS/HEG
![Page 5: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/5.jpg)
Page 5
Tools
Why we need some HDF-EOS tools ?
![Page 6: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/6.jpg)
Page 6
• HDF was selected for EOS data since
– Many of the HDF defined datatypes map well
to EOS datatypes Raster images, multi-dimensional arrays,
and text blocks
– HDF provides efficient data management Storage & performance
Why HDF?
![Page 7: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/7.jpg)
Page 7
• Some other EOS datatypes do not map directly to HDF datatypes, particularly
@ geolocated datatypes
Therefore, some additions to traditional HDF are required to fully support these datatypes.
HDF-EOS Datatypes
![Page 8: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/8.jpg)
Page 8
HDF-EOS Datatypes
To bridge the gap between the needs of EOS data products and the capabilities of HDF, 4 new EOS specific datatypes were introduced:
Point, Swath, and Grid Zonal Average (hdf-eos5)
![Page 9: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/9.jpg)
Page 9
HDF-EOS Datatypes
• Point : Unorginzed data (spatial or temporal) that has associated geolocation information
• Swath: Time-ordered data such as satellite swaths (time-ordered series of scanlines),
or
profilers (time-ordered series of profiles).
• Grid: Data stored in a rectilinear array based on a well defined and explicitly supported
projection.
• ZA: Swath like datatype without geolocation mapping
![Page 10: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/10.jpg)
Page 10
HDF4 File Contents – User View
Objects & Relationships
User Metadata
Object Data
![Page 11: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/11.jpg)
Page 11
HDF4 File Contents – Format View
Vgroupname = variable_name
class = Var0.0
NDG
SDDSDNT
variable
name = variable_nameranktype
storagetype
data
Vdataname = attribute_name
class = Attr0.0
1 1
0…* 0…*
1
1
attributename = attribute_name
1 1
1 1
1
1 1
1 1
1 1 0...1
0...1
0...1
1
byte order,chunked storage,
compression, …
Object Data
HDF4 Mapping Project Update - Ruth Aydt
![Page 12: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/12.jpg)
Page 12
HDF/HDF-EOS are Complicated!
• HDF is not efficient for Users since they need to learn
- how to use HDF & HDF-EOS libraries
- how to process the data correctly
• Licenses for commercial tools are expensive
The HDF Group & Raytheon have developed some tools to help users
![Page 13: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/13.jpg)
Page 13
HDF & HDF-EOS Tools
For most users HDF readers or NetCDF/CF conversion applications are enough
- hdp, h5dump, eos2dump(for lat/lon), HDFView
- h4tonccf, h4cf, OpeNDAP Handlers
But some need more processing and tools appropriate for EOS data
• HDF-EOS to GeoTIFF Conversion Tool (HEG)
• HDF-EOS plug-in for HDFView
![Page 14: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/14.jpg)
Page 14
![Page 15: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/15.jpg)
Page 15
Name says HEG is:
A utility to convert EOSDIS data fromHDF-EOS2 to GeoTIFF format.
But in practice:
It can do more than just conversion to GeoTIFF
HEGHEG
![Page 16: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/16.jpg)
Page 16
HEG FunctionalityHEG Functionality
• Conversion
– HDF-EOS (Swath or Grid ) to a single-band or multi-band GeoTIFF’s
– HDF-EOS Swath to HDF-EOS Grid --> GeoTIFF
OR
– ASTER Swath is written directly into GeoTIFF
– HDF-EOS Swath or Grid to generic Binary (with
metadata file)
![Page 17: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/17.jpg)
Page 17
HEG FunctionalityHEG Functionality
• Subsetting
– spatial: using Swath/Grid corner lat/lon or pixel numbers
– field: Grid, Swath, and Field selection
– band: Selection of data layers in 3-D and 4-D datasets ( 3-D to 6-D MISR SOM datasets )
![Page 18: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/18.jpg)
Page 18
HEG FunctionalityHEG Functionality
• Stitching
– (mosaicing) + subsetting + Reprojecting
• Reprojection
@To: UTM, PS, TM, STP, LCC, LAMAZ, GEO, SIN, Albers, CEA
• Subsampling
@Subsample stacks
![Page 19: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/19.jpg)
Page 19
HEG FunctionalityHEG Functionality
• Running from Command Line (batch jobs)
or • with Java-based GUI
• Metadata preservation/creation.
![Page 20: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/20.jpg)
Page 20
Conversion GUI
HEG
![Page 21: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/21.jpg)
Page 21
HEG
Stitch GUI
![Page 22: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/22.jpg)
Page 22
BilinearNearest NeighborCubic Convolution
GeographicPolar StereographicUTMMercatorTransverse MercatorLambert AzimuthalLambert Conformal Conic
SinusoidalAlbersState PlaneCylindrical Eqyual-AreaNo-Reprojection
Conversion GUI
HEG
Lat-LonRows-Columns
![Page 23: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/23.jpg)
Page 23
HEG
HEG Parameter File Used for running HEG on command line.
![Page 24: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/24.jpg)
Page 24
Status log for HEG run
![Page 25: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/25.jpg)
Page 25
HEG Data/Platform SupportHEG Data/Platform Support
• Currently supports
MODIS, MISR, ASTER, AIRS, AMSR-E products on TERRA and AQUA
– more than 210 products
• Operable on Sun, Windows, Linux, MAC
![Page 26: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/26.jpg)
Page 26
Versions Available:
Stand-alone:
A downloadable desktop version with a User Interface
@Portable, written in Java.@Not dependent on COTS (eg. IDL).
HEG VersionsHEG Versions
![Page 27: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/27.jpg)
Page 27
HEG VersionsHEG Versions
Integrated into ECS Data Pool:
Access through NASA archive online storage
@Reduces the transfer time of HDF-EOS data sets (if subset is requested)
@Provides the end-user with the exact file required by their application
![Page 28: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/28.jpg)
Page 28
Examples (Examples (MISRMISR))
Mt. Etna Eruption: Mt. Etna Eruption: MISRMISR L1B Stitched Blocks L1B Stitched Blocks
![Page 29: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/29.jpg)
Page 29
Examples (Examples (MODISMODIS))
MOD021KM, Multi-Band GeoTIFF, 3 bands: EV_1KM_RefSB, Bands 1,2,5
2004 Hurricane Francis, 2004 Hurricane Francis, MODISMODIS
![Page 30: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/30.jpg)
Page 30
Examples (Examples (ASTERASTER))
Two Swaths Stitched …..
….. and Subsetted
AST_L1B, 2003 California FiresAST_L1B, 2003 California FiresGeo. projection, Multi-Band Geo. projection, Multi-Band GeoTIFF (3 bands):GeoTIFF (3 bands):Visible Near Infrared (VNIR)Visible Near Infrared (VNIR)
![Page 31: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/31.jpg)
Page 31
Examples ( Examples ( AMSR_E_L2_Ocean) swath
Swath Conversion to Geographic Grid/HDF-EOS output
![Page 32: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/32.jpg)
Page 32
Future WorkFuture Work
![Page 33: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/33.jpg)
Page 33
![Page 34: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/34.jpg)
Page 34
HDFView HDFView
A Java-based browser from The HDF Group for HDF4 and HDF5 based data.
![Page 35: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/35.jpg)
Page 35
HDFView with HDF-EOS Plug-inHDFView with HDF-EOS Plug-in
![Page 36: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/36.jpg)
Page 36
HDFView with HDF-EOS Plug-inHDFView with HDF-EOS Plug-in
–Extends HDFView to browse HDF-EOS data in both versions (2 & 5).
–Is a plug-in module does not change HDFView functionality
–Identical look and feel when displaying HDF and HDF-EOS objects
–Access to complete breakdown of HDF-EOS objects.
e.g. grid info, projection info, dims, datafields, and attributes
![Page 37: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/37.jpg)
Page 37
HDFView Plug-inHDFView Plug-in
What else HDF-EOS plug-in
adds to HDFView?
![Page 38: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/38.jpg)
Page 38
Swath, Grid, and Point ObjectsSwath, Grid, and Point Objects
38
Plug-in
file
treeview
Swath/Grid
Lat/Lon
display
![Page 39: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/39.jpg)
Page 39
HE2 to HE5 ConversionHE2 to HE5 Conversion
One click to convert HE2 to HE5 file format39
![Page 40: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/40.jpg)
Page 40
AMSR-E SwathAMSR-E Swath
40
Getting Image of selected
pixels in a table Getting lat/lon for a selected pixel
in a table
![Page 41: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/41.jpg)
Page 41
MODIS (HE2) and OMI (HE5) Grids MODIS (HE2) and OMI (HE5) Grids
41
Drawing
Coastlines
&
International
Borders
on
HE2 & HE5
Grids
![Page 42: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/42.jpg)
Page 42
• Coastlines & International borders for
HE2 & HE5 swath
• Adding single-band/multi-band GeoTIFF writer
• Adding GeoTIFF reader with geolocating
capability
Future work
![Page 43: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/43.jpg)
Page 43
AvailabilityAvailability
• Download, Documentation, Help:
HDFViewHDFView::http://newsroom.gsfc.nasa.gov/sdptoolkit/http://newsroom.gsfc.nasa.gov/sdptoolkit/
HDFView/HDFView_hdfeos_plugin.htmlHDFView/HDFView_hdfeos_plugin.html
HEGHEG: : http://newsroom.gsfc.nasa.gov/sdptoolkit/http://newsroom.gsfc.nasa.gov/sdptoolkit/
HEG/HEGHome.htmlHEG/HEGHome.html
![Page 44: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/44.jpg)
Page 44
Help, CommentsHelp, Comments
• Questions, Comments, Requests:
http://hdfeos.org/forums/index.phpforums/index.php
![Page 45: HDF-EOS Tools](https://reader033.fdocuments.in/reader033/viewer/2022061201/5479b8cfb4795968098b4819/html5/thumbnails/45.jpg)
Page 45
Thank You!Thank You!
Questions/comments?