MOBY Uncertainties

32
Research and Operations Marine Optical Buoy Design Review July 18-19, Plan for calibration and maintenance of AHAB Uncertainty Budget: Laboratory Components Carol Johnson NIST

description

Plan for calibration and maintenance of AHAB Uncertainty Budget: Laboratory Components Carol Johnson NIST. MOBY Uncertainties. MOBY Calibration Workshop Nov 2003 to address uncertainties in measured water-leaving radiance Radiometric components Uncertainty in the primary calibration sources - PowerPoint PPT Presentation

Transcript of MOBY Uncertainties

Page 1: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

Plan for calibration and maintenance of AHAB

Uncertainty Budget:Laboratory Components

Carol JohnsonNIST

Page 2: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

MOBY Uncertainties

• MOBY Calibration Workshop Nov 2003 to address uncertainties in measured water-leaving radiance– Radiometric components

• Uncertainty in the primary calibration sources

• Transfer uncertainty

• MOBY radiometric scale maintenance during deployment

• Systematic effects– Temperature

– Stray light

– Wavelength error

– Environmental components (Ken’s Talk)• Instrument self-shading

• Wave focusing: environmental ‘noise’

• Polarization

Page 3: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

MOBY Radiometric Uncertainties

• Established rigorous measurement protocols ensuring direct traceability to primary national radiometric standards

• Established radiometric uncertainty budget conforming with international recommendations

• Goal: uncertainty budget to be dominated by environmental factors.– Uncertainty goal: ~ 3 % (k=1)

Radiometric calibration uncertainties of 4 % to 8 % (6 % > 400 nm)

D. K. Clark, et al., Proc. SPIE 4483, 64-76 (2002)

Satellite sensors uncertainty requirements: 5 % in water-leaving radiance

Page 4: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

MOBY Uncertainties – Why do they matter?To evaluate what uncertainty components are important, it is

important to understand how the MOBY data are used.

• SeaWiFS and MODIS: MOBY data are used to set the T=0 post-launch gains.

• In using a large number of MOBY matchups, random uncertainty components will be reduced, but systematic effects will not.

• For a large-enough data set, final uncertainty in gain coefficients will be dominated by systematic effects.

SeaWiFS

Page 5: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

MOBY Radiometric Calibration Flow Diagram

NISTStandards

NISTStandards

Correction for temperature

and stray light

Correction for temperature

and stray light

Uncertainty

Page 6: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

Primary Calibration Sources

Page 7: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

How well can you do?Uncertainties in Irradiance Standards from NIST

Irradiance Standard Lamps (FEL), typical expanded uncertainties in spectral irradiance Source of Uncertainty Relative Expanded Uncertainties [%] (k=2) 250 350 655 900 1600 2000 2300 2400 HTBB temperature 0.57 0.41 0.22 0.16 0.09 0.08 0.07 0.07 HTBB emittance 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1

HTBB uniformity 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1

HTBB stability 0.07 0.05 0.03 0.02 0.01 0.01 0.01 0.01 Geometric factors 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1

SR stability (HTBB) 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.5

Wavelength accuracy 0.58 0.38 0.18 0.005 0.011 0.013 0.011 0.011 SR stability (lamp) 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 Current stability (lamp) 0.08 0.06 0.03 0.02 0.02 0.01 0.01 0.01 Unc. of PWS (k=2) 0.85 0.60 0.36 0.28 0.24 0.24 0.23 0.54 Lamp-to-lamp transfer 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1

Long term stability of PWS 1.31 0.94 0.50 0.36 0.20 0.16 0.14 0.14 Unc. of issued lamps (k=2) 1.56 1.12 0.63 0.47 0.33 0.31 0.29 0.57

Page 8: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

How well can you do?Uncertainties in Radiance Standards from NIST

“Land-level” integrating sphere source, expanded uncertainties in spectral radiance Source of Uncertainty Relative Expanded Uncertainties [%] (k=2) 300 400 500 600 700 800 900 1000 Blackbody quality 0.12 0.07 0.03 0.01 0.00 0.01 0.03 0.04 Calibration of pyrometer lamp 0.33 0.27 0.22 0.18 0.15 0.12 0.11 0.10 BB temperature determination and transfer to lamp 1.02 0.21 0.36 0.16 0.37 0.31 0.31 0.55 Wavelength accuracy 0.12 0.10 0.07 0.06 0.05 0.04 0.04 0.03 Temperature scale (thermodynamic vs ITS-90) 0.58 0.46 0.37 0.30 0.27 0.24 0.20 0.19 Unc. for NPR (4 lamps) (k=2) 1.23 0.59 0.57 0.39 0.48 0.41 0.39 0.59

Page 9: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

Uncertainties from Transfer of Scales

Page 10: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

J. J. Butler, et al., J. Res. Natl. Inst. Stand. Technol. 108, 199-228 (2003).

Results of measurements of Santa Barbara Remote Sensing SIS100 lamp-illuminated integrating sphere: sphere was used for MODIS and Landsat ETM+ pre-launch calibrations

Transfer radiometers from NIST, NASA’s GSFC, and the University of Arizona measured the sphere radiance under different illumination conditions and compared their results with the SBRS-determined radiance.

This +/- 2% agreement is good result, based on our experience.

How well can you do?EOS Laboratory Intercomparison Experiments

Page 11: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

MOBY Calibration Sources & Uncertainties

Re-calibrated every 6 months or 50 H of use

- Calibrated first with original lamps (1 %

to 2% agreement)

- Re-lamped and calibrated a second time

Monitored during operation using NIST-

calibrated filter radiometers called

Standard Lamp Monitors (SLMs)

Yearly NIST visits with transfer radiometers

and sources to validate the MOBY

radiance scales

NIST-traceable calibration: 3-5 % uncertainties (secondary standards laboratory)

NIST calibration unc (400 to 700 nm): < 1 %

Page 12: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

Calibration Stability & Repeatability

0.020

0.022

0.024

0.026

0.028

0.030

1996 1997 1998 1999 2000 2001 2002

Optronics Calibration

Calibration Series 1

Calibration Series 2

Calibration Series 3

Calibration Series 4

SLM412

Ra

dia

nce

(W

cm

-2 s

r-1 n

m-1

)Date

OL420 and SLM412 Radiances at 411.8 nm

a

OL Calibration History

within the calibration uncertainty Stability ~ 0.5 % (SLM)

OL420 and SLM RadianceAt 412 nm

(870 nm results similar)

Page 13: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

Uncertainties Associated with the Deployments

Page 14: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

Internal Reference Lamps - Stability QC

Blue < 0.5 %

Both + - 0.5%

Red < 0.5%

Page 15: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

Diver Reference Lamp Calibrations

The changes observed are well within the uncertainty of the method. Hence we cannot draw any useful conclusions.

Page 16: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

Pre to Post Deployment Calibrations

Assume a rectangular probability distribution, the associated uncertainty is about 0.6%

For top arm input and all deployments

0 5 10 15 20 25 30 350.98

1

1.02

1.04

1.06

1.08

Deployment Number

Lu

To

p S

yste

m R

esp

on

se R

atio

(P

reC

al/P

ost

Ca

l)

411.8 nm442.1 nm546.8 nm665.6 nm

411.8 442.1 546.8 665.6-0.01

0

0.01

0.02

0.03

0.04

0.05

0.06

Wavelength (nm)

Pre

/ P

ost S

yste

m R

espo

nse

Sta

bilit

y

LuTopLuMidLuBot

Averaged over deployments by wavelength

Page 17: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

In Situ Wavelength Calibration with Spectral Features

Red Spectrograph2.5 years

Approx. +/- 1 nm

Blue Spectrograph2.5 years

Approx. +/- 0.6 nm

Page 18: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

Uncertainties Associated with Systematic Effects

Page 19: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

Temperature Affects the System Responsivity

We measured and then applied a temperature correction to the pre- and post-deployment calibrations (as well as all MOBY data during deployments).

Page 20: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

Impact of Stray Light or Spectral out-of-band

-200 -100 0 100 200 300 400 500 600 7001E-6

1E-5

1E-4

1E-3

0.01

0.1

1

Re

lativ

e S

pe

ctra

l Re

spo

nsi

vity

(a

u)

Relative Wavelength (nm)

Band 8, 411.8 nm Band 9, 442.1 nm Band 12, 546.8 nm

Spectral out-of-band of representative MODIS bands

What is its magnitude?

Does it impact the measurement requirements?

No instrument is perfect: every instrument measures unwanted radiation

Stray light causes systematic errors: that is, errors that don’t

average to zero with repeat measurements.

Page 21: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

Stray Light in MOBYStray light correction to MODIS BandsImages of Laser Lines

Correction to Responsivity Multiple Deployment Time Series

Single Deployment

Page 22: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

Evaluation of the uncertainties: Monte Carlo

Uncertainty in SLC for in-water upwelling radiance Lu

There are a number of parameters that go into the model; each has an uncertaintyWe doubled the uncertainty in the fits to those parameters to account for drift, etc.

Then ran a Monte Carlo simulation: for each component we used a Gaussian probability distribution.Simulations run a minimum of 100 times & uncertainties calculated.

Page 23: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

MOBY Radiometric UncertaintyMOBY WorkshopPreliminary Results

Uncertainty Component Value [%]

Calibration Source Radiance 3% to 5% (Comm. Lab); 0.5% (NIST)

Source stability between calibrations ~3% to ~4% (Repeat Cals.); ~0.5% (SLMs)

Responsivity during deployment ~0.6%

Wavelength Within 1 Pixel

Temperature ~ Negligible

Stray light <~ 0.25%

Combined Standard Uncertainty [%] 3.2% to 5.1%

Page 24: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

MOBY Summary: Lessons Learned for AHAB

• Traceability to primary national and international radiometric standards and the SI

• Good experiment design– multiple measurements (pre- & post- calibrations)

– verify and validate

– strict protocols

– methods to monitor detectors and monitor sources

• Characterize instruments thoroughly

Page 25: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

AHAB Implementations

• Primary Calibration Sources– Blue-rich, tailored for ocean color spectral distributions

• Transfer of Scales– Calibrate sources using NIST facilities

– Expand SLM concept to hyperspectral (more spectral information)

• During deployment– Internal sources (LEDs)

– System level stability monitoring

– Improvements to diver calibration lamps

• Systematic– 2-D stray light characterization at NIST SIRCUS facility

Page 26: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

Primary Calibration Sources

350 400 450 500 550 600 650

0.0

0.5

1.0

1.5

2.0

2.5

3.0

Lu(

) [

W/c

m2 /s

r/nm

]

Wavelength [nm]

-0.05

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

Measured Source

Calibration Source

Lca

l()

[W

/cm

2 /sr/

nm]

Source Spectral Power Distributions

Page 27: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

LED Sources

12 mm12 mm

Fiber-coupled Spectroradiometer

Multiple Channel Power Supply

Data Acquisition & Control

Realized Spectrum

Target Spectrum

L()

Integrating Sphere

LED Heads

Photometer

Fiber-coupled Spectroradiometer

Multiple Channel Power Supply

Multiple Channel Power Supply

Data Acquisition & Control

Realized Spectrum

Target Spectrum

L()

L()

Integrating Sphere

LED Heads

Photometer

Page 28: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

LED Source: Field Tests

LED source MOS204 23 degC sets

0

2

4

6

8

10

12

300 400 500 600 700

Wavelength

AD

U/p

xl/s

ec

0

0.01

0.02

0.03

0.04

0.05

mean

stdev/mean

OL420 source MOS204 13 degC sets

0

10

20

30

40

50

60

300 400 500 600 700

Wavelength

AD

U/p

xl/s

ec

0

0.01

0.02

0.03

0.04

0.05

mean

stdev/mean

More flux in the blue;Better matched to the ocean’s spectral distribution

Page 29: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

Spectrally Tunable, Detector-based Source

Under development using GOES-R funding

Dispersing element Spatial Light Modulator(SLM)

Recombine the Light

SLM Light Guide

• New Technology: SLM– Digital micromirror devices (DMDs)

– Liquid crystal on silicon arrays (LCOS arrays)

375 425 475 525 575 625 675 725

Wavelength (nm)In

tens

ity

(a. u

.)

Target

Output

Page 30: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

System Level Stability Monitoring

• Stable LED sources, as proven in MOBY (0.5% during deployments)

• Fiber-coupled to external optical input

• Allows Daily system level monitoring of AHAB responsivity

• Eliminates major source of unknown behavior for MOBY

Page 31: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

AHAB Characterization

• Thermal characterizations at NIST– design for good thermal control and stability

• Detailed and thorough stray light characterization at NIST– spectral and spatial

– smaller system means all tests can be done at NIST, resulting in full and dense wavelength coverage

– Spatial effects have been dealt with under R&O work

• Excellent wavelength stability– Monitor using solar lines as with MOBY

Page 32: MOBY Uncertainties

NOAA Research and Operations Marine Optical Buoy Design Review July 18-19, 2006

Summary

• AHAB’s radiometric uncertainties will be the lowest possible for this type of field activities.