Terrestrial Ecology and Land Cover/Change Science Review:

Basis of Successful Land Product Validation

Jeffrey L. PrivetteNASA’s Goddard Space Flight Center

Warren Cohen, John Dwyer, S. Tom Gower, Simon Hook, Chris Justice, Alfredo Huete, Jeff Morisette, Ranga Myneni, Dick Olson, Steve Running,

David Starr

Challenge

One of the five primary objectives of the Earth Science Enterprise:

Identify and measure the primary causes of change

Impossible to accomplish without knowledge of measurement uncertainties

No precedent for global validation of land products

What We Knew Going In…

Excellent knowledge of point issuesLimited knowledge of site (25 km2) issues

spatial and temporal sampling scaling of point measurements error budgeting

Poor knowledge of global sampling issues except inherently cost-prohibitive without strategic planning

EOS Response

EOS Validation Program Office/D. Starr

Guidance from Instrument Teams Priorities (e.g., low level products first)

Approaches

Community resources

1997 NRA for EOS Validation Investigators Shared responsibility with product developers

Strategic Plan 1:Focus Sites

EOS Land Validation Core Sites Facilitate resource sharing and synergies in validating multiple sensors and products

Built on existing stations, resources, experts• FLUXNET, AERONET, LTER, etc.

• Long-term duration

Stratified by “6 biomes” of Myneni/Running

Jointly nominated by Instrument and (newly selected) Validation PIs

EOS Land Validation Core Sites

Solutions Data “stockpiles” for EOS Core Sites

• Unique online ftp directories at EROS Data Center 200 km x 200 km MODLAND product subsets Landsat 7 ETM+ scenes (coordinated by MODLAND)

• SeaWiFS, ASTER, UMD/CRESS, others joined

Centralized field data registry• WWW metadata catalog and data linking at ORNL• Field investigator upkeep and control of native data• Accommodates 6-month public release policy for

EOS Validation Investigators

Strategic Plan 2:Simple, Fast Data Provision

/1999 /2000 /2001

EOS Land Product Validation FTP Server

/GLCTS /Landsat7

/”site name”

/Geocover /MODIS /ASTER

/”site name”/”site name”

Level-0RLevel-1G

MOD09A1MOD13A2MOD43B3

AST_L1B

• Accessible from Land Validation Web Site or ORNL DAAC Mercury

http://modis-land.gsfc.nasa.gov/val/http://mercury.ornl.gov/

or enter specificcoordinates

2. Spatial Search(use cursor to define

area of interest)

Mercury Searching

3. Temporal Search

1. Fielded Search(using picklists for

Project, Site PI, Parameter, etc.)

Strategic Plan 3:International Buy-in

Committee on Earth Observing Satellites (CEOS)Composed of the world’s space agencies Coordinates civilian observations from spaceWorking Group for Calibration and Validation (WGCV)

Facilitates and encourages standards, efficiencies, best practices Four Subgroups

Synthetic Aperture Radar Microwave Sensors Infrared and Visible Optical Sensors (IVOS) Terrain Mapping

No mechanism for coordinating or advancing validation of higher level land products

Calibrationemphasis

Strategic Plan 3:International Buy-in

Committee on Earth Observing Satellites (CEOS)Composed of the world’s space agencies Coordinates civilian observations from spaceWorking Group for Calibration and Validation (WGCV)

Facilitates and encourages standards, efficiencies, best practices Five Subgroups

Synthetic Aperture Radar Microwave Sensors Infrared and Visible Optical Sensors (IVOS) Terrain Mapping Land Product Validation

Calibrationemphasis

WGCV Subgroup onLand Product Validation

Chartered in 2000 with strong NASA advocacy

Founding focus: Support of Global Observations of Forest Cover (GOFC/GOLD) needs (fire, land cover and biophysical products)

Multinational endorsement and participationparticularly from Europe and Japan (ENVISAT, ADEOS2/GLI/ POLDER2)

Case Study 1: ASTER/Hook

MODIS and ASTER thermal infrared systems and products

ApproachVerification of sensor calibration at simple sites

Determination of lower- to higher level product uncertainties at realistic sites

Data from ground, aircraft, other sensor and lunar looks

Sites: 6 in U.S., 3 in Australia

MkIV Raft with Radiometer, Meteorological Station and Temperature Loggers

% Radiance Difference between Vicarious and OBC DerivedRadiances, v1.0.2 (1-7) and v2.05 (8-10) ((V-O)/V)*100

-2.00

-1.50

-1.00

-0.50

0.00

0.50

1.00

1.50

2.00

1(03/12/00)

2(06/24/00)

3(08/04/00)

4(08/19/00)

5(09/28/00)

6(11/07/00)

7(03/16/01)

8(06/03/01)

9(07/22/01)

10(08/22/01)

Overpass Date

Band 10 (8.29 um)

Band 11 (8.63 um)

Band 12 (9.08 um)

Band 13 (10.66 um)

Band 14 (11.29 um)

ASTER Spectral Calibration Accuracy with Time

Case Study II:BigFoot/FLUXNET

BigFoot: Site-based Sampling and Modeling Methodology

BigFoot: Site-based Sampling and Modeling Methodology

Parameters v. Products• Landcover 1,2

LAI 1,2

FPAR 1,2

NPP 1,2

GPP 1,3,4

NEP 1,3,4

1 = MODIS2 = field measure3 = modeled4 = tower/eddy flux

Example:Comparing

Landsat and MODIS LAI

Products

BigFoot MODLAND

Generalizing BigFoot’s Approach Over FLUXNET

Data from Flux Towers

Micro-meteorological data (hourly/daily)

Incident PAR

Air Temperatures

Vapor Pressure

Rainfall

Wind

Land surface properties (seasonal/annual)

Land cover

Leaf area index

Soil carbon

Soil nitrogen

Biomass (stem)

Fluxes: CO2, water and heat

MODIS PRODUCTSMOD-17 8-d GPPMOD-17 ANPP

Ecosystem models are used to relate flux measurements

to MODIS derived products

VALIDATION

Based on Nemani/2/28/00

FLUXNET Validation Web Site:http://daac.ornl.gov/FLUXNET/data.html

FLUXNET Validation Web Site:http://daac.ornl.gov/FLUXNET/data.html

Online Access to FLUXNET Data

Case Study III:LAI Network/Myneni

Joint activity of CEOS LPV & MODLAND

MODIS LAI and FPAR

Open-invitation workshops from 1998- to 2000. Findings:

Several foreign groups sampling LAI, some addressing scaling

Strong interest in cross-comparisons, technical exchange and trouble-shooting

Regional networks could be developed for MODLAND LAI validation needs

Majura ValleyMajura Valley KioloaKioloa

Moreton BayMoreton Bay

Lake GeorgeLake George

Batt ReefBatt ReefCooper CreekCooper Creek

MossmanMossmanDaintreeDaintree

Cape TribulationCape Tribulation

Townsville ReefTownsville Reef

Lake ArgyleLake Argyle

GoldsworthyGoldsworthy

Lake Eyre Lake Eyre

Mt FittonMt Fitton

UardryUardry

Tinga tinganaTinga tingana

Tanjil BrenTanjil Bren

AmburlaAmburla

Port HedlandPort Hedland

KakaduKakadu

PanoramaPanorama

Hyperion SitesHyperion Sites

Other CSIRO SitesOther CSIRO Sites

KeyKeyToolabinToolabin

KunothKunoth

X band sitesX band sites

ThangooThangoo

Lake FromeLake Frome

CIGSN SitesCIGSN Sites

Endevour ReefEndevour Reef

Ray MertonRay Merton

Earth Observation Earth Observation CentreCentre

http://www.eoc.csiro.au

LAI Product Evaluation

12-18 month pathfinder

Leverage existing 2000-2001 field campaigns

Workshop: Rome, June 2001

MODLAND coordinating Landsat ETM+ acquisitions and MODLAND LAI subsets

LAI Product Evaluation: Participating Sites

Site Country BiomeBarton Bendish UK CroplandBondville IN, USA CroplandBOREAS NSA Canada Conif. ForestBOREAS SSA Canada Conif. ForestBrasschaat (De Inslag) BelgiumHarvard Forest MA, USA Decid. ForestJrvselja Estonia Boreal ForestKejimkujik Park, Nova Scotia Canada Decid. ForestKonza KS, USA GrasslandKrasnoyarsk Russia Boreal ForestLandes France Conif. ForestMali Mali ShrublandMaun Botswana WoodlandMongu Zambia WoodlandOkwa River Botswana ShrublandPandamentanga Botswana WoodlandPark Falls WI, USA Decid. ForestPodkamennaya Russia Boreal ForestRomilly France CroplandRuokolahti Finland Conif. ForestSkukuza S. Africa SavannaTshane Botswana SavannaWatson Lake, Yukon Canada Conif. Forest

Validation… What Works?

Dedicated science team liaisons for validationSimplified data access (dedicated archives) Strong communication between Product PIs and Validation PIs

Algorithm or product familiarity

Strong attention to error budgetingSatellite constellationsInternational partnerships and coordinationNetworks, networks, networks

Duration (mission succession), consistency

Validation… To Be Resolved

Global RepresentivityHow many points in time, space and biome?

ScalingInfancy for many products

Communication of ResultsFormal venues lacking

User InputBetter knowledge = greater cost How much is enough?

Uncertainty EmbeddingProduct data layer with pixel-by-pixel uncertainties

“Under-represented” Climate Combinations for North America and Asia

http://www-eosdis.ornl.gov/daacpages/climate_collections.html0 800 1600 2400 3200 4000

Precipitation, [mm y-1]

-30

-20

-10

0

10

20

30

Mea

n T

air,

[de

g.C

]

Europe AsiaN-America

0 800 1600 2400 3200 4000Precipitation, [mm y-1]

0

5

10

15

20

25

30

Mea

n T

air,

[de

g.C

]

AfricaAust/OzeaS-America

0 800 1600 2400 3200 4000Precipitation, [mm y-1]

-30

-20

-10

0

10

20

30

Mea

n T

air,

[de

g.C

]

Europe AsiaN-America

0 800 1600 2400 3200 4000Precipitation, [mm y-1]

0

5

10

15

20

25

30

Mea

n T

air,

[de

g.C

]

AfricaAust/OzeaS-America

Concerns and Recommendation

EOS underscored two relevant lessons:Sensor characteristics change throughout their lifetimes, including prelaunchAlgorithms change throughout their lifetimes

• Reprocessing happens.

Throughout NPP/NPOESS era (present to ~2020), NASA is purchasing products from industry

Is NASA ready to conduct performance verification?Could NASA credibly challenge a vendor on an underperforming high level product?

Permanent NASA Office on Calibration and Validation

Status of ASTER Data ProductsProduct Name Date of

Latest Upgrade

Current Release

State

Upgrade Schedule

Level 1a 28 Apr '01 ValidatedLevel 1b (AST03) 28 Apr '01 ValidatedDecorrelation Stretch (AST06) 19 Jun '01 ValidatedBrightness Temperature (AST04) 12 Jun '01 ValidatedSurface Reflectance (AST07) 24 Jan '01 Provisional Nov '01 (V)Surface Radiance VNIR, SWIR (AST09) 24 Jan '01 Provisional Nov '01 (V)

Surface Radiance TIR (AST09) 28 Mar '01 Provisional Nov '01 (V)Surface Kinetic Temperature (AST08) 09 Apr '01 Beta Nov '01 (P)

Feb '02 (V)Surface Emissivity (AST05) 09 Apr '01 Beta Nov '01 (P)

Feb '02 (V)Digital Elevation Model (DEM) (AST04) absolute

10 July '01 Validated

Digital Elevation Model (DEM) (AST04) relative

10 July '01 Beta Sep '01 (V)

Polar Classification (AST13) Dec '01 TBD tbd