MPI-Meteorology

Hamburg, Germany

Evaluation

of year 2004 monthly

GlobAERaerosol products

Stefan Kinne

the task

an evaluation of

GlobAER 2004 global maps for aerosol optical depth (info on amount) for Angstrom parameter (info on size)

by ASTR (dual view, global, at best 10 per month) by MERIS (nadir view, global, at best 1 per day) by SEVIRI (nadir view, regional, at best 30 per day) by a merged (ATSR / MERIS / SEVIRI) composite

the questions

how well do the data compare to trusted data references (e.g. AERONET) ?

how well do the data compare to existing data sets – even for same sensor data ?

can the performance be quantified ?

more specifically … what are the scores of a new (outlier-

resistant) method … examiningbias, spatial and temporal variability ?

the investigated properties

aerosol optical depth (AOD)extinction along a (vertical) direction due to

scattering and absorption by aerosol here for the entire atmosphere here for the mid-visible (0.55m wavelength)

Angstrom parameter (Ang)spectral dependence of AOD in the visible spectrum

small dependence (Ang ~ 0) aerosol > 1m size strong decrease (Ang > 1.2) aerosol < 0.5m size

monthly data-sets

GlobAER 2004 maps GAa - ATSR GAs - SEVIRI GAm - MERIS GAx – merged

other multi-ann. maps med – model median clim – med & aer(sun) sky – med & aer(sky) TO – TOMS

other 2004 maps ATs - ATSR Swansea SEb- SEVIRI Bruxelles Mdb - MODIS deep blu MO - MODIS std coll.5 MI – MISR version 22 Ag – AVHRR, GACP Ap – AVHRR, Patmos Aer – AERONET 2004

AOD map comparisons

AOD annual mapsall available data

AOD seasonal mapsATSR GlobAER vs SwanseySEVIRI GloabAER vs RUIB-Bruexelles

difference maps

… to a remote sensing ‘best’ compositeall available data-sets focus on the four GloabAER products

AOD – 2004 annual maps

ATSR / SEVIRI – seasonal AOD

AOD diff to ‘composite’

underestimates overestimates

quick (annual) AOD check

ATSRunderestimates in dust regionsoverestimate in biomass regions

SEVIRIsevere biomass overestimatesuseful over-land estimates ?

MERISapparent land snow cover issue

mergednot the envisioned improvement

‘-’ ’+’

Angstrom map comparisons

Angstrom annual mapsall available data

Angstrom seasonal mapsATSR GlobAER vs SwanseySEVIRI GloabAER vs RUIB-Bruexelles

difference maps

… to a climatology (model & AERONET) all available data-sets focus on the four GloabAER products

Angstrom – 2004 annual maps

ATSR / SEVIRI – seasonal Angstr.

Angstrom diff to ‘climatology’

underestimates overestimates

quick annual Angstrom check

ATSRunderestimates in tropicsoverestimates in south. oceans

SEVIRIunderestimates over oceansstrong overestimates over land

MERISoverestimates over land

mergednot the envisioned improvement

‘-’ ’+’

the SCORING challenge

quantify data performance by one number

develop a score such that contributing errors to be traceable back tobiasspatial correlation temporal correlationspatial sub-scale (e.g. region) temporal sub-scale (e.g. month, day)

make this score outlier resistant

one number !

- 0.504

info on overall bias

- 0.504

sign of the bias

| 1 | is perfect …. 0 is poor

- 0.504

sign of the bias

the closer toabsolute 1.0… the better

product of sub-scores

- 0.504 = 0.9 *- 0.7 * 0.8

the closer toabsolute 1.0… the better

temporalcorrelationsub-score

biassub-score

spatialcorrelationsub-score

sign of the bias

spatial stratification

- 0.504 = 0.9 * -0.7 * 0.8

timescore

biasscore

spatialscore

spatial sub-scale scores

overall score

regional surface area weights

TRANSCOM regions

temporal stratification

- 0.504 = 0.9 * -0.7 * 0.8

timescore

biasscore

spatialscore

spatial sub-scale scores

overall score

temporal sub-scale scores (e.g. month or days)

averaging in timeinstantaneous median data

sub-score definition

each sub-score S

is defined by an error e andby an error weight w

0.9 * -0.7 * 0.8 S = 1 – w * e

timescore S

biasscore S

spatialscore S

spatial sub-scale scores

temporal sub-scale scores (e.g. month or days)

instantaneous median data

definition of errors e

S = 1 – w * e

all values for theerrors e are rank - based

for “time score” and “spatial score” rank correlation coefficients for data pairs are determined

e, correlation = (1- rank_correlation coeff.) /2

(correlated: e = 0, anti-correlated: e = 1)

timescore

biasscore

spatialscore

definition of errors e S = 1 – w * e

for “bias score” allall data-pairs are placed in a single array and ranked by value

then ranks are separated according to data origin, summed and (rank-sums are) compared

e, bias = (sum1 – sum2) / (sum1 + sum2)(strong neg.bias e = -1, strong pos.bias e= +1)

an example (“how does the rank bias error work ?”) set 1: 1 7 8 value: 9 8 7 4 3 1 rank-sum 1: 11 set 2: 3 4 9 rank: 1 2 3 4 5 6 rank-sum 2: 10

e = (1-2)/(1+2) = (11-10)/21 ~zero no clear bias

timescore

biasscore

spatialscore

definition of error weight w

S = 1 – w * e

w is a weight factor based on the inter-quartile range / median ratiow = (75%pdf - 25%pdf) / 50%pdf

… but not larger than 1.0 (w<1.0)

simply put …

if there is no variability an error does not matter

biasscore

spatialscore

timescore

scoring summary one single score … … without sacrificing spatial and temporal

detail ! stratification into error contribution from

biasspatial correlation temporal correlation

robustness against outliers

still … just one of many possible approaches now to some applications …

questions

how did GlobAER products score? overall ?seasonality ?spatial correlation ?bias ? in what regions ? in what months ?

how did scores place to other retrievals …with the same sensor (for the same year 2004)with other sensors (for the same year 2004)

monthly data-sets

GlobAER 2004 maps GAa - ATSR GAs - SEVIRI GAm - MERIS GAx – merged

other multi-ann. maps med – model median clim – med & aer(sun) sky – med & aer(sky) TO – TOMS

other 2004 maps ATs - ATSR Swansea SEb- SEVIRI Bruxelles Mdb - MODIS deep blu MO - MODIS std coll.5 MI – MISR version 22 Ag – AVHRR, GACP Ap – AVHRR, Patmos Aer – AERONET 2004

ann global scores – AOD / Angstrom year 2004 - AOD

TOTAL seas bias corr GAa .47 .81 .81 .72 GAm .43 .73 .82 .72 GAx .46 .77 .81 .75 GAs -- -- -- -- ATs .57 .88 .86 .75 SEb -- -- -- -- clim .72 .94 .88 .87 MISR .59 .90 .87 .76 MOD .65 .92 .88 .80 best .69 .91 .88 .87

year 2004 - Angstrom TOTAL seas bias corr

GAa -.57 .83 -.88 .79 GAm .41 .73 .83 .66 GAx .55 .84 .87 .75 GAs -- -- -- -- ATs -.49 .77 -.87 .73 SEb -- -- -- -- clim .79 .94 .93 .90 MISR .67 .92 .90 .81 med -.59 .81 -.87 .83 MISR .62 .90 .90 .77

vs sun-photometry

annual AOD scores – diff refs year 2004 – AOD (aeronet)

TOTAL seas bias corr GAa .47 .81 .81 .72 GAm .43 .73 .82 .72 GAx .46 .77 .81 .75 GAs -- -- -- -- ATs .57 .88 .86 .75 SEb -- -- -- -- clim .72 .94 .88 .87 MISR .59 .90 .87 .76 MOD .65 .92 .88 .80 best .69 .91 .88 .87

year 2004 – AOD (climat.) TOTAL seas bias corr

GAa .47 .86 .80 .69 GAm .37 .77 .83 .58 GAx .41 .82 .77 .66 GAs -- -- -- -- ATs .47 .85 .80 .69 SEb -- -- -- -- aer -.72 .94 -.88 .87 MISR .51 .89 .80 .71 MOD .56 .87 .85 .75 best .65 .89 .87 .84

ATSR AOD - regional errors / data

vs sun-photometry

merged AOD - regional errors / data

vs sun-photometry

ATSR-s AOD - regional errors / data

vs sun-photometry

ann. AOD scores – land/ocean year 2004 – land AOD

TOTAL seas bias corr GAa -.38 .72 .82 .64 GAm .42 .73 .80 .72 GAx .36 .64 .82 .70 GAs -- -- -- -- ATs .55 .88 .88 .72 SEb -- -- -- -- clim .76 .95 .91 .89 MISR -.63 .92 -.89 .77 MOD -.59 .92 -.85 .75 best .68 .92 .87 .85

year 2004 – ocean AOD TOTAL seas bias corr

GAa .52 .85 .80 .76 GAm .43 .73 .83 .71 GAx .53 .86 .80 .77 GAs -- -- -- -- ATs .58 .89 .85 .77 SEb -- -- -- -- clim .69 .92 .87 .86 MISR .57 .89 .86 .75 MOD .70 .93 .90 .84 best .70 .90 .88 .88

ATSR AOD – temporal total errors

vs sun-photometry

monthly / regional ‘error’- change

improvement deteriation

ATSR (GlobAer) minus ATSR (Swan) : total AOD error (=1-|S|)

vs sun-photometry

monthly / regional ‘error’- change

improvement deteriation

SEVIRI (GlobAer) vs SEVIRI (Brux) : total AOD error (=1-|S|)

vs sun-photometry

AOD summary

ATSR by GlobAerpoorer than MODIS, MISR and even ATSR-sstronger deductions over land than oceans

ocean scores are usually better than land scores low bias over land, high bias over oceanserrors are larger for the northern hemisphere

MERIS by GlobAerpoorer than ATSR … and also the ‘merged’

ATSR Angstrom - regional errors

vs sun-photometry

ann. Ang scores – land/ocean year 2004 – land Angstr

TOTAL seas bias corr

GAa .54 .79 .88 .78 GAm .39 .73 .80 .65 GAx .50 .79 .83 .76 GAs -- -- -- --

ATs -.44 .72 -.85 .73 SEb -- -- -- --

clim .82 .97 .93 .91 MISR -.66 .91 -.90 .81 med -.66 .91 -.90 .81

year 2004 – ocean Angstr TOTAL seas bias corr

GAa -.58 .85 -.87 .78 GAm -- -- -- -- GAx -.58 .87 -.90 .75 GAs -- -- -- --

ATs .53 .81 .89 .74 SEb -- -- -- --

clim .76 .93 .93 .88 MISR .68 .93 .91 .81 med -.54 .75 -.85 .84

ATSR Angstr. – temporal total errors

vs sun-photometry

monthly / regional ‘error’- change

improvement deteriation

ATSR (GlobAer) vs ATSR (Swansey) : total Ang. error (=1-|S|)

vs sun-photometry

Angstrom summary

ATSR by GlobAer (model-based !)

poorer than MODIS, MISR, better than ATSR-socean scores are slightly above land scores

(ocean scores are usually better than land scores)high bias over land, low bias over oceanshigher errors during (continental) summers

no benefits from merged productsLack of MERIS Angstrom data over oceans

outlook

focus should be on (long-term) ATSRstill improvement needed Angstrom constrain should helpcollaborate with Swansey

merged data is conceptually interesting… but limited by the poorest link (MERIS) if using diff sensors …use the same model !

extras

AOD ATSR – GlobAER 2004

AOD SEVIRI – GlobAER 2004

AOD MERIS – GlobAER 2004

AOD merged – GlobAER 2004

AOD SEVIRI – RUIB 2004

AOD ATSR – Swansea 2004

annual maps – AOD 2004med median cli ‘climatology’ sun sun-photo Globaer products

MO MODIS Efc forecastMI MISR Eas assimilation

SEVIRIAATSR MERIS

annual maps – Angstrom 2004med median cli ‘climatology’ sun sun-photo Globaer products

MO MODIS Efc forecastMI MISR Eas assimilation

AATSR MERIS SEVIRI

ATSR AOD - regional errors

e_T total error

e_C corr error

e_B bias error

s_B bias sign

e_S seas error

r_B rel. bias

r_E rel. error

m_d median

dev med. diff

vs sun-photometry

SEVIRI AOD - regional errors

e_T total error

e_C corr error

e_B bias error

s_B bias sign

e_S seas error

r_B rel. bias

r_E rel. error

m_d median

dev med. diff

vs sun-photometry

MERIS AOD - regional errors

e_T total error

e_C corr error

e_B bias error

s_B bias sign

e_S seas error

r_B rel. bias

r_E rel. error

m_d median

dev med. diff

vs sun-photometry

‘merged’ AOD - regional errors

e_T total error

e_C corr error

e_B bias error

s_B bias sign

e_S seas error

r_B rel. bias

r_E rel. error

m_d median

dev med. diff

vs sun-photometry

ATSR AOD – temporal total errors

vs sun-photometry

ATSR-s AOD – temporal total errors

vs sun-photometry

SEVIRI AOD – temporal total errors

vs sun-photometry

ATSR/SEVIRI – temporal AOD errors

GlobAER other sources

ATSR

SEVIRI

vs sun-photometry

ATSR Angstrom - regional errors

e_T total error

e_C corr error

e_B bias error

s_B bias sign

e_S seas error

r_B rel. bias

r_E rel. error

m_d median

dev med. diff

vs sun-photometry

SEVIRI Angstrom - regional errors

e_T total error

e_C corr error

e_B bias error

s_B bias sign

e_S seas error

r_B rel. bias

r_E rel. error

m_d median

dev med. diff

vs sun-photometry

MERIS Angstrom - regional errors

e_T total error

e_C corr error

e_B bias error

s_B bias sign

e_S seas error

r_B rel. bias

r_E rel. error

m_d median

dev med. diff

vs sun-photometry

‘merged’ Angstr. - regional errors

e_T total error

e_C corr error

e_B bias error

s_B bias sign

e_S seas error

r_B rel. bias

r_E rel. error

m_d median

dev med. diff

vs sun-photometry

ATSR Angstr. – temporal total errors

vs sun-photometry

SEVIRI Ang. – temporal total errors

vs sun-photometry

ATSR/SEVIRI – temporal Ang errors

other sources

ATSR

GlobAER

SEVIRI

vs sun-photometry

monthly / regional ‘error’- change

improvement deteriation

ATSR (GlobAer vs Swansey) : total Ang. error (=1-|S|) change

vs sun-photometry

monthly / regional ‘error’- change

improvement deteriation

SEVIRI (GlobAer vs Bruxelles): total Ang. error (=1-|S|) change

vs sun-photometry