A Generalized Approach to Microwave Satellite Data Assimilation Quality Control and Preprocessing

Kevin Garrett1 and Sid Boukabara2

11th JCSDA Science Workshop on Satellite Data AssimilationJune 6, 2013

College Park, MD

1: RTI @ NOAA/NESDIS/STAR, JCSDA 2: NOAA/NESDIS/STAR, JCSDA

211th JCSDA Workshop on Satellite Data Assimilation - College Park, MD

Outline• Introduction to the MIIDAPS• 1DVAR retrieval process• Potential applications to NWP (GSI)• Current progress• Next steps

Goal: To have a flexible, consistent algorithm applicable to a variety of sensors for use as a preprocessor to NWP data assimilation systems which:• provides quality control information about radiance observations• provides dynamic information about scenes (precip, surface conditions)• has consistent error characteristics across all sensors (using retrieved parameters)• is flexible and easily extended to new/future sensors• is mindful of computing resources/overhead/latency• has meaningful, positive impact on analyses and forecasts

311th JCSDA Workshop on Satellite Data Assimilation - College Park, MD

Assimilation/Retrieval All parameters retrieved simultaneously Valid globally over all surface types Valid in all weather conditions Retrieved parameters depend on information content from sensor frequencies

MIIDAPS OverviewMulti-Instrument Inversion and Data Assimilation Preprocessing System

MIIDAPS

S-NPP ATMS

DMSP F16 SSMI/SDMSP F17 SSMI/SDMSP F18 SSMI/S

GPM GMI

MetOp-A AMSU/MHSMetOp-B AMSU/MHS

GCOM-W1 AMSR2

Megha-TropiquesSAPHIR/MADRAS

TRMM TMI

NOAA-18 AMSU/MHSNOAA-19 AMSU/MHS

Inversion Process Consistent algorithm across all sensors Uses CRTM for forward and jacobian operators Use forecast, fast regression or climatology as first guess/background

MIIDAPS 1DVAR is based on the Microwave Integrated Retrieval System (MiRS)

411th JCSDA Workshop on Satellite Data Assimilation - College Park, MD

MIIDAPS Overview

Over All Surfaces

Using All Channels

-90°

90°0°Latitude

170°

170°10

0 La

yers

Temperature

Emissivity Skin Temperature

Core state variables (products) from MIIDAPSExample of MIIDAPS retrieval using S-NPP ATMS with vertical cross sections at 170° longitude.

Water Vapor

Rain & Graupel

Cloud

Post ProcessingTPW Rain Rate

511th JCSDA Workshop on Satellite Data Assimilation - College Park, MD

Obs Error [E]

No

Convergence

1DVAR Retrieval/Assimilation Process

Init

ial S

tate

Vecto

r [X

]

Climatology

Forecast

Retrieval mode

Assimilation mode CRTM

Simulated TBs

Observed TBs

(processed)

Compare

ConvergenceSolution

[X]Reached

ComputeDX

K

Update State Vector

[X]

Iterative Processes

CovarianceMatrix [B]

Bias Correction

2. Retrieval done in reduced (EOF) space

Reduce the dimensionality of the covariance matrix from 400x400 to 22x22 (or less depending on sensor)

Transform [K] and [B] to EOF space for minimization

LBTLΘB 1. Solution is found by minimizing the cost function:

Convergence is determined by non-constrained cost function:

Y(X)YEY(X)Y

2

1XXBXX

2

1J(X) m1Tm

01T

0

1,

2XYmY1E

TXYmY2

3. X is updated through the Levenberg-Marquardt equation:

nΔXnK)nY(XmY1

ETnBKnK

TnBK

1nΔX

611th JCSDA Workshop on Satellite Data Assimilation - College Park, MD

1DVAR Retrieval/Assimilation Process

1st Attempt 2nd Attempt

Temperature Temperature

Water Vapor Water Vapor

Cloud Liquid Water Rain Water Profile

Ice Water Profile

Skin Temperature Skin Temperature

Surface Emissivity Surface Emissivity

State Vector Parameters per Attempt

• MIIDAPS allows a maximum of 2 retrieval attempts per observation– 1st attempt assumes no scattering signal in the TBs– 2nd attempt assumes scattering from rain/ice is present in TBs– Maximum of 7 iterations per attempt

• Tunable parameters: nattempts, niterations, channels used (optimize efficiency without degrading outputs)

Chi-square with out scattering

Chi-square with scattering

711th JCSDA Workshop on Satellite Data Assimilation - College Park, MD

1DVAR Retrieval/Assimilation Process

Surface preclassifier determines which background and covariances to initialize for retrieval (left)

Retrieved 23 GHz Emissivity Retrieved TPW

Over All Surfaces In All Weather

Seamless transition along surface boundaries

Emissivity inclusion in the state vector is vital for retrieval/assimilation

Emissivity sensitivity to rainfall rate for AMSU-A frequencies

Retrieved Rainfall Rate error as a function of retrieved emissivity error

811th JCSDA Workshop on Satellite Data Assimilation - College Park, MD

Applications to NWPPrimary objective for a 1DVAR preprocessor on microwave observations

Chi-square based QC

Cloudy/rainy radiance detection

Emissivity constraint/assimilation

Focus for Global NWP using GSI• Use chi-sq for QC/filtering• Use CLW/RWP/GWP for detecting

cloudy/rainy obs• For filtering or assimilation• Non-precip cloud/precipitating cloud

• Use surface emissivity as boundary condition for forward simulations to increase surface channel observations

Chi-square based QC

Cloudy/rainy radiance detection

Emissivity constraint/assimilation

911th JCSDA Workshop on Satellite Data Assimilation - College Park, MD

Applications to NWP

Retrieved TPW

Emissivity vital for assimilation of surface sensitive channels in all weather

NEXRAD NEXRAD NEXRAD

0.75

0.8

0.85

0.9

0.95

1

0 20 40 60 80 100 120 140 160

Frequency (GHz)

MIR

S E

mis

siv

ity

04-May

0.75

0.8

0.85

0.9

0.95

1

0 20 40 60 80 100 120 140 160

Frequency (GHz)

MIR

S E

mis

siv

ity

08-May

0.75

0.8

0.85

0.9

0.95

1

0 20 40 60 80 100 120 140 160

Frequency (GHz)

MIR

S E

mis

siv

ity

10-May

5/4

5/8

5/10

Average emissivity spectra before/after a 3-day rain event in May 2008.5/4-5/8 shows ~8% change in 23 GHz emissivity.

5/5-5/7

1011th JCSDA Workshop on Satellite Data Assimilation - College Park, MD

Applications to NWPPrimary objective for a 1DVAR preprocessor on microwave observations

Temperature 400 mb

TPW

Rainfall Rate

Focus for Regional NWP using GSI +HWRF• Assimilation of sounding data near tropical

storm cores• Assimilation of TPW• Assimilation of rainfall rate retrievals

Temperature 400 mb

TPW

Rainfall Rate

1111th JCSDA Workshop on Satellite Data Assimilation - College Park, MD

Applications to NWP

BUFRFiles

‘read_sensor’routines

“setuprad”(clw, O-B filtering)

Implementation of the 1DVAR preprocessor

Implementation of 1DVAR preprocessing at the Bufferization stage:• Process all radiance observations during time window• CPU time spent outside of assimilation (minimize effect on latency)• Encode 1DVAR output in BUFR as metadata or in unique BUFR file• Increased control for radiance thinning/selection during GSI read process• Maintain ability to use 1DVAR geophysical outputs on optimized set

Implementation of 1DVAR preprocessing BUFR read stage:• Process all radiance observations during time window• Increased control for radiance thinning/selection during GSI read process• Maintain ability to use 1DVAR geophysical outputs on optimized set• Separate 1DVAR interface for each satellite sensor• Read routines must be parallelized• CPU time added to the analysis (how much can be afforded?)

Implementation of 1DVAR in “setuprad” stage:• Process only on thinned set of observations• Maintain ability to use 1DVAR geophysical outputs on optimized set• CPU time used in analysis (how much can be afforded?)• Code is universal for all satellite datasets (single interface to 1DVAR)

1211th JCSDA Workshop on Satellite Data Assimilation - College Park, MD

Current Status• Testing currently underway with implementation in read_atms

routine– 1DVAR called for additional QC (based on chisq)– No optimized thinning implemented (every 5 FOVs/Scanlines)

• Prelimenary implementation in setuprad routine– Still testing the interface

Current operational With additional 1DVAR filter

1311th JCSDA Workshop on Satellite Data Assimilation - College Park, MD

Future Work

• Continue with implementation both in setuprad and in the read routines for optimized thinning

• Test impact of cloud filters, use of emissivity in number of obs, O-B, O-A, etc.

• Extend to other sensors, starting with infrared• Apply to regional HWRF (product assimilation)• Involve other interested JCSDA partners (NCEP, OAR,

GMAO, Navy, AFWA, NCAR)