POSTER TEMPLATE BY: VIIRS Active Fire algorithm integration in Suomi NPP Data Exploitation (NDE)...
1
POSTER TEMPLATE BY: www.PosterPresentations.com VIIRS Active Fire algorithm integration in Suomi NPP Data Exploitation (NDE) environment: research to operations Marina Tsidulko 1 , Walter Wolf 2 , Ivan Csiszar 2 , Louis Giglio 3 , Wilfrid Schroeder 3 (1)IMSG at NOAA/NESDIS/STAR, College Park, MD, (2) NOAA/NESDIS/STAR, College Park, MD, (3)University of Maryland, College Park, MD Active Fire Algorithm Basics and History Active Fires Product on Operations Active Fire Algorithm Inputs and Outputs Data Flow Active Fire Algorithm at STAR AIT Summary NDE Environment •The current IDPS version of the VIIRS Active Fire algorithm runs over land and produces a list of fire detections in a sparse array format. • The University of Maryland (UMD) enhanced version of the algorithm: - provides additional outputs including the Fire Radiative Power (FRP) of each fire pixel and a new attribute to describe land for each pixel (Fire Mask). - has global coverage including water - planned to be implemented in the NDE development environment - initially will run on S-NPP data and is planned to create the J1 product in Current Operational Shortfalls: Accuracy of Product Explicit validation is not feasible due to lack of fire mask and sufficient independent reference data Current IDPS implementation lacking: - User-required data layers (fire mask, fire radiative power) - Processing over water pixels (e.g., detection of gas flares) - Latest algorithm updates (MODIS Collection 6-equivalent) New development addresses shortfalls: Address operational satellite active fire data user requirements - Incorporate thematic classification (fire mask) and fire radiative power (FRP) retrieval - Provide data globally, including water Satisfy the JPSS VIIRS Active Fire EDR product requirements Implement latest algorithm updates (MODIS Collection 6) Achieve greater compatibility between EOS/NASA-MODIS and JPSS-VIIRS active fire products The primary mission of the NDE system is to provide near real time products derived from S-NPP observations to NOAA’s operational and climate communities as well as other end users. The NDE receives data from the IDPS operational stream and provides them for algorithms employed in the system. The interface between NDE and the AF driver scripts will consist of Production Control Files (PCF) and Production Status Files (PSF) files. PCF contains: - All the required input files to process a granule, including paths if they’re located outside the working The VIIRS Active Fire Algorithm (AF) has been developed at the University of Maryland (UMD) as heritage of MODIS Active Fires algorithm. An earlier and simplified version of the VIIRS AF algorithm – based on MODIS collection 4 - is implemented in the operational Interface Data Processing System (IDPS). The replacement version of the VIIRS AF algorithm is significantly enhanced. It is based on the EOS/NASA MODIS Fire and Thermal Anomalies algorithm (MOD14/MYD14) and equivalent to MODIS collection 6. The algorithm uses hybrid fixed- threshold and contextual approach to detect sub-pixel fires: - Small (sub-pixel) fires produce contrasting radiometric response across different spectral channels as a result of high temperatures - Fixed thresholds used to detect unambiguous fires - Contextual tests compare target pixel with immediate surrounding using dynamic sampling window - Reflective channels are used to differentiate other bright pixels (e.g., clouds, sandy soils, sun glint) from biomass burning - Land/water mask used to distinguish between biomass burning and gas flares The algorithm provides additional products in output. The UMD replacement code: Works with NASA LPEATE inputs in HDF4 format Provides output in HDF4 format Works on aggregated granules STAR AIT Development: Integrate the replacement code into STAR Linux machines Modify the code to work with IDPS binary inputs (unpacked HDF5 files) on single granules Provide granulation for Land/Water Mask (granulated Land/Water product is non-deliverable in IDPS) Add conversion to NetCDF4 which is a standard NDE output format Create wrapping Perl scripts in compliance with the NDE standards STAR AIT Testing: Generate set of runs for test granules for different geographical areas and chosen for different conditions such as day, night, missing SDR data, off-shore gas flares Compare outputs with NASA LPEATE non- operational AF products Compare outputs with operational IDPS products Fire pixels Example of Fire Mask March 1, 2015 Classes: 0 missing input data; 1 not processed (obsolete) ; 2 not processed (obsolete) ; 3 non-fire water ; 4 cloud ; 5 non-fire land ; 6 unknown; 7 fire (low confidence); 8 fire (nominal confidence); 9 fire (high confidence) Algorithm/ Tiles Input to AF Binary file HDF5 file VIIRS-SDR latitude longitude view zenith angle solar zenith angle view azimuth angle solar azimuth VIIRS-MOD-GEO- TC GMTCO_*.h5 VIIRS-SDR M13 brightness temperature M13 QF1 M13 radiance VIIRS-M13-SDR SVM13_*.h5 VIIRS-SDR M15 scaled brightness temperature, QF1 VIIRS-M15-SDR SVM15_*.h5 VIIRS-SDR M16 scaled brightness temperature, QF1 VIIRS-M16-SDR SVM16_*.h5 VIIRS-SDR M5 scaled reflectance, QF1 VIIRS-M5-SDR SVM05_*.h5 VIIRS-SDR M7 scaled reflectance, QF1 VIIRS-M7-SDR SVM07_*.h5 VIIRS-SDR M11 scaled reflectance, VIIRS-M11-SDR SVM11_*.h5 Quarterly Surface Type Land/Water Mask Tiles Granulated Land/Water Mask Mod-Gran product in IDPS) Fire Algorithm QA Mask: 32-bit unsigned integer Bits Description 0-1 Surface Type (water=0, coastal=1, land=2) 2-3 N/A 4 Day/Night (daytime = 1, nighttime = 0) 5 Potential fire (0/1) 6-10 N/A 11 Fire Test 1 valid (0 - No, 1 - Yes) 12 Fire Test 2 valid (0 - No, 1 - Yes) 13 Fire Test 3 valid (0 - No, 1 - Yes) 14 Fire Test 4 valid (0 - No, 1 - Yes) 15 Fire Test 5 valid (0 - No, 1 - Yes) 16 Fire Test 6 valid (0 - No, 1 - Yes) 17- 19 N/A 20 Adjacent clouds (0/1) 21 Adjacent water (0/1) 22- 23 Sun Glint Level (0-3) 24 N/A 25 False Alarm 1 (excessive rejection of legitimate background pixels) 26 False Alarm 2 (water pixel contimination) 27 Amazon forest-clearing rejection test 28- 31 N/A Name Type Description FP_line 16 bit Integer Fire pixel line Sparse data array (unit- less) FP_sample 16 bit Integer Fire pixel sample Sparse data array (unit- less) FP_latitude 32 bit Float Fire pixel latitude Sparse data array (unit: degrees) FP_longitude 32 bit Float Fire pixel longitude Sparse data array (unit: degrees) FP_power 32 bit Float Fire radiative power Sparse data array (unit: MW) FP_confidence 8 bit Integer Fire detection confidence Sparse data array (unit: %) FP_land 8 bit Integer Land pixel flag Sparse data array (unit- less) Fire Mask: 8-bit unsigned integer Missing – 0 Brightness temperatures for M13 or M15 unavailable Scan – 1 Not processed (obsolete) Other – 2 Not processed (obsolete) Water – 3 Pixel classified as non fire water Cloud – 4 Pixel classified as cloudy No Fire – 5 Pixel classified as non fire land Unknown – 6 Pixel with no valid background pixels Fire Low – 7 Fire pixel with confidence strictly less than 20% fire Fire Medium – 8 Fire pixel with confidence between 20% and 80% Fire High – 9 Fire pixel with confidence greater than or equal to 80% Inputs: HDF5/BLOB Outputs: NetCDF4 for fire pixels Outputs: QA Mask for each pixel in the granule Outputs: Fire Mask for each pixel in the granule
-
Upload
everett-horton -
Category
Documents
-
view
219 -
download
7
Transcript of POSTER TEMPLATE BY: VIIRS Active Fire algorithm integration in Suomi NPP Data Exploitation (NDE)...
POSTER TEMPLATE BY:
www.PosterPresentations.com
VIIRS Active Fire algorithm integration in Suomi NPP Data Exploitation (NDE) environment: research to operations
Marina Tsidulko1, Walter Wolf2, Ivan Csiszar2, Louis Giglio 3, Wilfrid Schroeder 3 (1)IMSG at NOAA/NESDIS/STAR, College Park, MD, (2) NOAA/NESDIS/STAR, College Park, MD, (3)University of Maryland, College Park, MD
Active Fire Algorithm Basics and History
Active Fires Product on Operations
Active Fire Algorithm Inputs and Outputs
Data Flow
Active Fire Algorithm at STAR AIT
SummaryNDE Environment
• The current IDPS version of the VIIRS Active Fire algorithm runs over land and produces a list of fire detections in a sparse array format.
• The University of Maryland (UMD) enhanced version of the algorithm:
- provides additional outputs including the Fire Radiative Power (FRP) of each fire pixel and a new attribute to describe land for each pixel (Fire Mask). - has global coverage including water - planned to be implemented in the NDE development environment - initially will run on S-NPP data and is planned to create the J1 product in the future • The final product is in NetCDF-4 format and will be available for
users through the OSPO distribution system.
Current Operational Shortfalls: Accuracy of Product
Explicit validation is not feasible due to lack of fire mask and sufficient independent reference data
Current IDPS implementation lacking:- User-required data layers (fire mask, fire radiative power)- Processing over water pixels (e.g., detection of gas flares)- Latest algorithm updates (MODIS Collection 6-equivalent) New development addresses shortfalls:
Address operational satellite active fire data user requirements- Incorporate thematic classification (fire mask) and fire radiative
power (FRP) retrieval- Provide data globally, including water
Satisfy the JPSS VIIRS Active Fire EDR product requirements Implement latest algorithm updates (MODIS Collection 6) Achieve greater compatibility between EOS/NASA-MODIS and
JPSS-VIIRS active fire products
The primary mission of the NDE system is to provide near real time products derived from S-NPP observations to NOAA’s operational and climate communities as well as other end users.
The NDE receives data from the IDPS operational stream and provides them for algorithms employed in the system.
The interface between NDE and the AF driver scripts will consist of Production Control Files (PCF) and Production Status Files (PSF) files. PCF contains:
- All the required input files to process a granule, including paths if they’re located outside the working directory. This includes input instrument and ancillary data, static files such as templates and lookup tables.- Any run parameters or flags
PSF contains all successfully generated output files
The VIIRS Active Fire Algorithm (AF) has been developed at the University of Maryland (UMD) as heritage of MODIS Active Fires algorithm.
An earlier and simplified version of the VIIRS AF algorithm – based on MODIS collection 4 - is implemented in the operational Interface Data Processing System (IDPS).
The replacement version of the VIIRS AF algorithm is significantly enhanced. It is based on the EOS/NASA MODIS Fire and Thermal Anomalies algorithm (MOD14/MYD14) and equivalent to MODIS collection 6.
The algorithm uses hybrid fixed-threshold and contextual approach to detect sub-pixel fires:
- Small (sub-pixel) fires produce contrasting radiometric response across different spectral channels as a result of high temperatures- Fixed thresholds used to detect unambiguous fires- Contextual tests compare target pixel with immediate surrounding using dynamic sampling window- Reflective channels are used to differentiate other bright pixels (e.g., clouds, sandy soils, sun glint) from biomass burning - Land/water mask used to distinguish between biomass burning and gas flares The algorithm provides additional products in output.
The UMD replacement code: Works with NASA LPEATE inputs in HDF4 format Provides output in HDF4 format Works on aggregated granulesSTAR AIT Development: Integrate the replacement code into STAR Linux machines Modify the code to work with IDPS binary inputs (unpacked HDF5
files) on single granules Provide granulation for Land/Water Mask (granulated Land/Water
product is non-deliverable in IDPS) Add conversion to NetCDF4 which is a standard NDE output
format Create wrapping Perl scripts in compliance with the NDE standards STAR AIT Testing: Generate set of runs for test granules for different geographical
areas and chosen for different conditions such as day, night, missing SDR data, off-shore gas flares
Compare outputs with NASA LPEATE non-operational AF products Compare outputs with operational IDPS products
Fire pixelsExample of Fire Mask March 1, 2015
Classes: 0 missing input data; 1 not processed (obsolete) ; 2 not processed (obsolete) ; 3 non-fire water ; 4 cloud ; 5 non-fire land ; 6 unknown; 7 fire (low confidence); 8 fire (nominal confidence); 9 fire (high confidence)
Algorithm/Tiles
Input to AF Binary file HDF5 file
VIIRS-SDR latitude longitude view zenith anglesolar zenith angleview azimuth angle solar azimuth angle
VIIRS-MOD-GEO-TC
GMTCO_*.h5
VIIRS-SDR M13 brightness temperatureM13 QF1M13 radiance
VIIRS-M13-SDR SVM13_*.h5
VIIRS-SDR M15 scaled brightness temperature, QF1
VIIRS-M15-SDR SVM15_*.h5
VIIRS-SDR M16 scaled brightness temperature, QF1
VIIRS-M16-SDR SVM16_*.h5
VIIRS-SDR M5 scaled reflectance, QF1
VIIRS-M5-SDR SVM05_*.h5
VIIRS-SDR M7 scaled reflectance, QF1
VIIRS-M7-SDR SVM07_*.h5
VIIRS-SDR M11 scaled reflectance, QF1
VIIRS-M11-SDR SVM11_*.h5
Quarterly Surface Type Land/Water Mask Tiles
Granulated Land/Water Mask
VIIRS-GridIP-VIIRS-Qst-Lwm-Mod-Gran
N/A (non-deliverable product in IDPS)
Fire Algorithm QA Mask: 32-bit unsigned integerBits Description0-1 Surface Type (water=0, coastal=1, land=2)2-3 N/A4 Day/Night (daytime = 1, nighttime = 0)5 Potential fire (0/1)6-10 N/A11 Fire Test 1 valid (0 - No, 1 - Yes)12 Fire Test 2 valid (0 - No, 1 - Yes)13 Fire Test 3 valid (0 - No, 1 - Yes)14 Fire Test 4 valid (0 - No, 1 - Yes)15 Fire Test 5 valid (0 - No, 1 - Yes)16 Fire Test 6 valid (0 - No, 1 - Yes)17-19 N/A20 Adjacent clouds (0/1)21 Adjacent water (0/1)22-23 Sun Glint Level (0-3)24 N/A25 False Alarm 1 (excessive rejection of
legitimate background pixels)
26 False Alarm 2 (water pixel contimination)27 Amazon forest-clearing rejection test28-31 N/A
Name Type Description
FP_line 16 bit IntegerFire pixel lineSparse data array (unit-less)
FP_sample 16 bit IntegerFire pixel sampleSparse data array (unit-less)
FP_latitude 32 bit FloatFire pixel latitudeSparse data array (unit: degrees)
FP_longitude 32 bit FloatFire pixel longitudeSparse data array (unit: degrees)
FP_power 32 bit FloatFire radiative powerSparse data array (unit: MW)
FP_confidence 8 bit IntegerFire detection confidenceSparse data array (unit: %)
FP_land 8 bit IntegerLand pixel flagSparse data array (unit-less)
Fire Mask: 8-bit unsigned integer
Missing – 0 Brightness temperatures for M13 or M15 unavailable
Scan – 1 Not processed (obsolete)
Other – 2 Not processed (obsolete)
Water – 3 Pixel classified as non fire water
Cloud – 4 Pixel classified as cloudy
No Fire – 5 Pixel classified as non fire land
Unknown – 6 Pixel with no valid background pixels
Fire Low – 7 Fire pixel with confidence strictly less than 20% fire
Fire Medium – 8 Fire pixel with confidence between 20% and 80%
Fire High – 9 Fire pixel with confidence greater than or equal to 80%
Inputs: HDF5/BLOBOutputs: NetCDF4 for fire pixels
Outputs: QA Mask for each pixel in the granule
Outputs: Fire Mask for each pixel in the granule
