1

STAR CrIS SDR CalVal Task Performance

Yong Han, Denis Tremblay, Xin Jin, Likun Wang and Yong Chen

October 23-24, 2012, CrIS Review MeetingProvisional Maturity Level

2

Tasks Performed at STAR

Task Name Description Status

1.Management Managerial and technical lead Continuous activity

2. Trending and Monitoring Monitor the instrument , RDR and SDR data quality

Functional, continuous activity

3. Inter-sensor comparison CrIS with IASI, AIRS, and VIIRS; CrIS/IASI/AIRS with GOES

Provisional status

4. Measurement assessment by CRTM

Double Difference of CrIS vs. IASI; CrIS inter-FOV consistency

Provisional status.

5. Spectral Uncertainty Cross correlation method On-going

6. Geolocation Assessment Using VIIRS as baseline. Bug fixed; Continue monitoring

7. Algorithm and software improvement

DR investigation and bug fix.High resolution RDR Processing.

On-going

Task 2 Web-based Trending and Monitoring System

Spacecraft reset into Earth point safe mode on June 21; the measured laser wavelength slowly moved back to normal after June 25.

The upload of EngPkt v34 on June 27 fixed the pixel-level quality flag bug: the percentage of degraded pixels decrease to nearly zero.

More than 60 parameters are automatically monitored: http://www.star.nesdis.noaa.gov/smcd/spb/xjin

Measured laser wavelength shows downward trend since 09/2012 ; the CMO will be re-calculate when wavelength drift exceeds the threshold.

4

Task 3: Inter-Sensor Comparison CrIS vs. IASI at SNOs

CrIS and IASI have an agreement of 0.2K or better for all 3 bands.

CrISIASI

CrIS - IASI

Total of 852 SNO Pairs411 North Pole441 South Pole

5

Task 3: Inter-Sensor Comparison CrIS vs. AIRS at SNOs

CrIS and AIRS have an agreement of 0.3K or better over SNO locations at North Pole for 1253 pairs.

CrISAIRS

CrIS - AIRS

6

Task 3: Inter-Sensor Comparison CrIS vs. VIIRS

Scene-temperature dependence can be seen at Band M15

CrIS is warmer than VIIRS at M15 and M16 but is colder at M13

CrIS-VIIRS BT difference for SNO cases are consistent with the whole orbital data.

CrIS and VIIRS have a general agreement of 0.4K or better.

NEdT large at low temperature

CrIS

-VIIR

SCr

IS-V

IIRS

CrIS

-VIIR

S

Task 3: Inter-Sensor Comparison CrIS vs. GOES

Inter-calibration results among CrIS-GOES, AIRS-GOES, and IASI-GOES agree well.

AIRS/CrIS

Under the GSICS framework, CrIS, AIRS, and IASI are routinely compared with GOES imager, which will help for GOES imager calibration.

Courtesy of Fangfang Yu, STAR/GSICS

8

Task 4: Measurement Assessment by CRTMCrIS and IASI double difference

Double difference of CrIS and IASI relative to CRTM calculated radiance are within ±0.2 K for most of channels.

CrISIASICrIS CRTMObsCRTMObsDD 2)()(

Double Difference Clear Sky over OceanOver 400,000 data points

9

CrIS Inter-FOV Consistency allFOVFOV CRTMObsCRTMObsBIAS

ii)()(

Task 4: Measurement Assessment by CRTMInter-FOV difference

FOV-2-FOV variability is small, within ±0.2 K for all the channels

Over 100,000 data points

10

- FOV spectral calibration is consistent within 1ppm- Spectral calibration has 3ppm offset wrt CRTM for LWIR, and 4ppm offset for MWIR (both meet requirement)

LWIR

Task 5: Spectral CalibrationSpectral Uncertainties characterized by CRTM

MWIR

Cross-correlation Method under Clear Sky between Observation and CRTM Simulation

11

Spectral Shift Caused by Earth-rotation Doppler Effect

)sin()sin()sin( iiRc sDoppler shift:

ν : channel frequency; Ω: Earth angular velocityRs: Distance of the satellite from the center of the Earthi : Inclination of the orbit; λ: LatitudeФ: Scan angle

The Earth-rotation Doppler effect can cause a channel frequency bias up to 1.3 ppmVery stable CrIS performance allowing detection of very small frequency shifts

FOR1 frequency shift relative to FOR30 FOR1 relative to FOR30FOR15 relative to FOR16

Task 5: Spectral CalibrationDoppler Effects in CrIS SDR data

12

Task 6: Geolocation AssessmentUsing VIIRS as a reference

Original Collocationat (0, 0)

Collocation after shift at minimum of cost function

- 3.5 Km cross track and -2.7 Km along track are found in CrIS SDR data- Root cause was found to be a coding error.

13

• Software bug was discovered in the operational code.• Code fix was engineered by STAR and delivered to IDPS for implementation.

- Geolocation code has been fixed and implemented in MX6.3 (10/16/2012)- Geolocation accuracy is estimated to be 1.0 km within 30° scan angles.

Task 6: Geolocation AssessmentBefore and After correction

Without correction With correction

CrIS CrIS-VIIRS CrIS-VIIRS

LW radiance before fix

Task 7: Algorithm/software Improvement Time Stamp Error in SDR codes

-STAR fixed the code error and delivered it to IDPS.- It will be implemented in MX7.0 (April 2013).

LW radiance after fix

UInt16 leftDays = CcsdsSecondaryHeader::numDaysSinceJanuary1_1958(&leftOperand);UInt16 rightDays = CcsdsSecondaryHeader::numDaysSinceJanuary1_1958(&rightOperand);

UInt32 leftMils = (((((((leftDays * HOURS_PER_DAY) + …UInt32 rightMils =(((((((rightDays * HOURS_PER_DAY) + …

Code fix (in red) in /ADL/SDR/CrIS/src/SystemTimeUtilities.cpp

Incorrect radiance of all bands, all FOVs and all scenes caused by the time stamp coding error with respect to the change of month.

Task 7: Algorithm/software Improvement: Processing Full Spectra

- “Truncation” code to process high resolution RDR and produce low resolution SDR developed at STAR using ADL4.0. Two files were modified: ProSdrCris.cpp and ProSdrCris.h- One step processing (changes are done in memory)- Code given to IDPS for future implementation.

CrIS AIRS

Hi-Res RDR

Low-Res SDR

Read RDR

Modify Processing Parameters

Truncate IFGM

Replace truncated IFGM into CCSDS Pkt

Continue SDR processing at

Low-Res

16

Task 7: Algorithm/software Improvement: New Quality Flag based on Imaginary Parts

September 25th 2012 (Ascending)

The abnormal imaginary part radiance will be used as a basis for a new pixel-level quality flag, available on October 16th, 2012

- New DQF based on the imaginary part of the spectrum is an excellent indicator of the quality of the real part. - This DQF is now implemented as part of MX 6.3 (October 16th 2012).

With contributions from other team members

17

Task 7: Algorithm and Software/Improvement Overall Data Quality Flag Improvement

June 13th 2012 (Beta), EngPkt V33 October 13th 2012, MX6.2, EngPkt V34

With contributions from other team members

• The Beta product has 60% of the SDRs labeled Degraded• CrIS Overall DQF fixed with threshold parameters tuning and code changes.• Last change is related to the new Stage cooler 2 temperature drift limit (4.0K instead of 1.0K). This change will be part of EngPkt V35 tentatively effective on October 22nd.

18

Discrepancy Report (DR) StatusClosed

DR Number

Description Date Status

4821 Incorrect geolocation calculation 07/03/12 Closed

4794 Make the CMO-EngPkt available to end user 06/21/12 Closed

4680 CrIS Overall DQF set to degraded 04/13/12 Closed

4661 CrIS Overall DQF set to value 3 04/10/12 Closed

4557 CrIS IFGM Packets with “Fill” release to RDR/SDR 02/06/12 Closed

4478 CrIS Overall DQF set to invalid for cold Earth scenes 12/02/11 Closed

-These DRs were identified as major and prioritized accordingly.

- Since April 2012 (BETA), all high priority DRs have been closed allowing to achieve the Provisional status.

19

Discrepancy Report (DR) StatusOn-going or Open

DR Number

Description Date Status

4941 Action for CLASS to distribute the CMO-EngPkt file 10/03/12 On-going

4909 Scan anomaly (time stamp issue) 09/21/12 On-going

4812 Add a flag indicating the CMO values have changed 06/28/12 On-going

4774 Corrupted CMO matrix leading with radiance with zero values (May 30th –June 7th radiance anomaly

06/11/12 Approved for future evaluation

4508 Earth spectra quality flag set to degraded when FCE detected

12/22/11 Approved for future evaluation

4481 FCE correction algorithm does not work for cold Earth scenes. (substantial level of effort).

12/06/11 On-going

4407 Inconsistency with serialization (related to CMO archiving)

10/19/11 open

20

DR Number

Description Date Status

4391 Some Interferogram.cpp operators need to be updated 10/13/11 Open

4389 Earth scene ZPD magnitude data type field inconsistencies

10/13/11 Open

Discrepancy Report (DR) StatusOn-going or Open

-The remaining open DRs are minor.

- DPE/DPA and NOAA-STAR agree that these remaining DRs are not needed for provisional status.

21

Issues and Challenges

• SW cold scene FOV-to-FOV difference.

• Spectral (un-apodized) ringing.

• Operational ground data system instability causing one week of bad radiance product (May 30th to June 7th). Problem was not replicated and root cause is unknown.

• FCE detection/correction module was turned off– Although, currently missing the component in the Ops code does not pose a significant

problem as the FCE is a rare event up to now, but there is no guarantee that it will not be a problem in the future.

– Software bug fixes and possibly some algorithm redesign are needed, which require a large team effort.

• Inefficient and ineffective code changing process, causing delays and uncertainties of algorithm and software fixes and updates.

• Augment the code with diagnostic products that allows extraction of key internal values such as raw spectra, temperatures, and many others.

22

FY-13 Schedule and Milestones

• STAR’s responsibility for changing EngPkt parameters and its upload, v35 Oct. 20, 2012

• CrIS SDR user guide, December 2012

• Tools for trending and monitoring radiance differences between CrIS and IASI/AIRS/VIIRS, March 2013

• Geolocation uncertainty trending and monitoring tool, March 2013

• STAR CrIS SDR processor (IDL code): CrIS Interferometry Transformation System (CITS), March 2013

• Spectral uncertainty trending and monitoring tool, June 2013

• RDR generator, September 2013

23

Summary of SDR Uncertainty

Method Result Requirement

Radiometric Assessment

-SNO CrIS vs AIRS/IASI-Double difference-GEO

SNO match 0.4K or betterDD match 0.2K or better0.3K or better

0.45% LW0.58% MW0.77% SW at 287K BB.

Radiometric Assessment

-Double Difference

0.2K or better Same as above

Radiometric Assessment

-GEO vs LEO 0.3K or better Same as above

Spectral Calibration

Using CRTM 3 ppm LW4-5 ppm MW

< 10 ppm

Geolocation Accuracy

CrIS vs VIIRS 1.0Km accuracy at nadir

1.5 Km

24

Summary

• CrIS and IASI have excellent radiometric agreement.

• CrIS data and CRTM have a 3ppm spectral calibration offset in LW.

• NOAA-STAR fixed the geolocation calculations.

• The high priority DRs have been addressed, now focusing on lower priority DRs.

• NOAA-STAR assessment is that the CrIS SDR product is ready to be declared provisional.