HEAP: Getting NUCAPS CrISand IASI products on the same pageLetitia SoulliardGAMA-1 TechnologiesNOAA/NESDIS/STAR

Outline• What is the HEAP?

• Current NUCAPS CrIS Status (NOAA-20 and S-NPP)

• Current IASI Status (MetOp-A and MetOp-B)

• Similarities between CrIS and IASI

• Differences between CrIS and IASI

• Combining the Systems

• Other related talks

• Questions

2

What is the HEAP?• Hyper-Spectral Enterprise Algorithm Package

• Main Goal: Get both sets of instruments using the same NUCAPS retrieval algorithm to ensure consistency in the processing of both instrument packages.

• Create a system that can run NOAA-20 and S-NPP CrIS data, as well as MetOp-A and MetOp-B IASI data. (And easily expandable to include MetOp-C)

• Separate the “NUCAPS” STAR wrapper system name (preprocessor, subsetter, ect) from the “NUCAPS” retrieval algorithm name

3

What is the HEAP?• In the future will be able to run Eumetsat Polar System Second

Generation (EPS SG)

• Have outputs from both systems be more compatible, same naming structure, and variable names

• Have this running in NDE as well as Direct Broadcast

• Deliver to NDE Spring of 2019 for NOAA-20 (and S-NPP)

• Deliver to NDE Fall of 2019 for MetOp-C

4

Current NUCAPS CrIS Status• Runs both NOAA-20 and S-NPP

• Running in NDE with latest and greatest algorithm code– V2.1.12c was delivered to NDE on August 15, 2018

• Running at Direct Broadcast

5

NUCAPS System Layer Data Flow

Preprocessor

Subsetter Retrieval

Validation

JPSS RR CM & CH + Geo files (HDF5

& netCDF4)

CrIS FSRSDR& SDR Geo files (HDF5)

ATMS TDR & SDR Geo files

(HDF5)GFS (GRIB2)

CrIS/ATMS/VIIRS(netCDF4)

Thinned CrIS/VIIRS

(netCDF4) and SDR monitoring

files

EDR & CCR (netCDF4)

CrIS/ATMS/VIIRSnetCDF4

OLR and EDR Daily Global Grids

EDR & CCR (netCDF4) and

EDR monitoring files

Retrieval resource and .nc template files

DEM, nctemplates

FSR Eigenvector

files, nctemplates

PCF

PCF

PSF

PSF

PCF

PSF

PSF

CrIS OLR

CrIS OLR

CrIS/ATMS/VIIRSnetCDF4

6

Current IASI Status• Running on an AIX Unix stand alone system at OSPO

• Runs both MetOp-A and MetOp-B

• Last DAP Update: 10/27/2015 for a AMSU-A channel that was seriously degrading the retrievals

• A modified version of the code called ISRP (IASI Simplified Retrieval Package) which runs on Linux is running at Direct Broadcast.

7

8

GFT

Check L1C

L1C Sub-

Setter

L2 Proc. Global Grids

Global Binaries

Global Match-

ups

L1B Proc.

AMSU/MHS/AVHRR Level 1B orbital files

IASIL1C

IASIL1C

IASI L1C + metadata

L1CT L2/CCR

GFS & GDAS grib file forecasts

IASI L1CT BUFR

IASI L1CT NetCDF

3x3 & 0.5x2 global grids

Global binaries

L2/L1CT matchups

L2 NetCDF

L2 and CCR NetCDF + metadata

granule times

MW granules

SystemManagement

Unit

OSPOMonitoring

IASI PPS System FlowSystem Infrastructure

Software Units

Remote Servers

AVHRR Proc.

IASI L1CT NetCDF

AVHRR CLAVRX

DDS

OLROLR

Existing

New or Modified

AVHRR/IASI

Similarities between CrIS and IASI• Both use IR instrument data input

– CrIS files

– IASI files

• Both can use MW supplementary instrument data input– ATMS files– AMSU-A and MHS files

• Both can use Cloud supplementary instrument data input– JPSS-RR Cloud files– AVHRR Clavr-X files

9

Differences between CrIS and IASI• Things unique about the CrIS system

– GEO data is in a separate file– Cloud data is only used in creating the BUFR file output not in the retrieval– Already set up to run in NDE and is organized in an NDE like manner

– Wrapping scripts are separate from System scripts

• Things unique about the IASI system– Cloud data is used in the retrieval– Stand alone so many of the scripts are a spaghetti mess when separating

NDE system stuff and wrapper script stuff

10

Combining the Systems• Updated the IASI codes to work on Linux instead of Unix

• Updated the IASI codes to work with netCDF-4 instead of netCDF-3

• Broke out the OLR and Clouds units from the Preprocessor

• Removed the IASI BUFR conversion from the system. That will be handled by the Reformatting Toolkit

• Renamed and edited some of the Fortran codes that were similarly named but had different internal data structures

11

Daily Processing

IASINUCAPS InstantaneousCrIS SDR/

Geo

ATMS TDR/ Geo

IASI L1

NWP GFS(GRIB2)

Clouds (optional)

HDF5/ NetCDF4 Granule data PreprocessNetCDF4

OLR

SO2 detection

from volcanic eruptionStatic

Ancillary

Thinned Rad./ PCS

AMSUA L1

MHS L1

EDR/ CCR Retrieval

Subsetter

Validation

VIIRS SDR Geo (.h5)

VIIRS Cloud Mask (.nc)

VIIRS Cloud Height (.nc) AVHRR Cloud

Mask (.h5)

Yes

IASI Yes

Collated Clouds

NUCAPS Clouds BUFR

Atmospheric Temperature/

Moisture/ IR Ozone/ Pofiles, CO/ CH4/ CO2

EDR and OLR Daily Global

Grids

Outgoing Longwave

Radiative Flux

Spatially and temporally collocated SDR data, bounding box metadata, Averaged

Cloud Information onto the FOVs

Subsetted NetCDF

**BUFR Conversion

IR Radiances(BUFR)

EDR/ OLR on 0.5X2 deg. grid

Binary orbit files

L1C subset Files

NetCDF4

Hi/Low-res eigen vectors, templates, resource files etc.

**BUFR Conversion

NoEDR/CCR Granule (NetCDF4)

EDR/ CCR Granules

Optional i/p

SO2 File(NetCDF4)

OLR Granules** Reformatting

Toolkit for NUCAPS BUFR

Conversion

Cloud Mask/Height info

NetCDF4 NUCAPS

12

Related Talks• Callyn Bloch

– Near real time CAPE combining hyper spectral IR satellite soundings and surface meteorological stations– After the break at 10:50am

– An example of using a baselined system, which the HEAP will contribute to in the future

• Antonia Gambacorta– Status of the NOAA Unique Combined Atmospheric Processing System (NUCAPS). Latest algorithm

developments and first glance results from NOAA20

– Today 4:40pm

– The science that drives the HEAP

• Nicholas Nalli– Status of SNPP and NOAA-20 NUCAPS Validation– Today 5:00pm

– The Validation of the results of HEAP data

13

Questions• Any Questions?

• Letitia.Soulliard@noaa.gov

14