The North American Carbon The North American Carbon Program Site-level Interim Program Site-level Interim

Synthesis Model Data Comparison Synthesis Model Data Comparison

(NACP Site Synthesis)(NACP Site Synthesis)Daniel Ricciuto, Peter Thornton, Daniel Ricciuto, Peter Thornton, Kevin Schaefer, Kenneth DavisKevin Schaefer, Kenneth Davis

Flux Tower PIsFlux Tower PIs

Modeling TeamsModeling Teams

NACP Site Synthesis TeamNACP Site Synthesis Team

Site Synthesis ObjectivesSite Synthesis Objectives

Activity initiated in 2008 by NACP to answer:Activity initiated in 2008 by NACP to answer:

- Are the various measurement and modeling estimates of - Are the various measurement and modeling estimates of carbon fluxes consistent with each other - and if not, why?carbon fluxes consistent with each other - and if not, why?

• Quantify model and observation uncertaintyQuantify model and observation uncertainty• 58 flux tower sites; 29 models58 flux tower sites; 29 models• Gap-filled observed weatherGap-filled observed weather• Observed fluxes, uncertainty, ancillary dataObserved fluxes, uncertainty, ancillary data

• Link model performance to model structureLink model performance to model structure• Which model characteristics associated with “best” models?Which model characteristics associated with “best” models?• How does this performance vary among sites?How does this performance vary among sites?

Flux Tower SitesFlux Tower SitesAmeriFlux sites over 35 sites Data provided by CDIACStandardized “Level 2” format

Canadian sitesOver 15 sitesData provided by - La Thuile synthesis activity - FLUXNET Canada

Site selection based on:Representativeness of biomesLength of recordQuality of data - gap fractionAncillary data availability

Meteorological drivers and flux observations gap-filled by NACP synthesis team

ModelsModels

• Results submitted from 22 models to dateResults submitted from 22 models to date• On average 10 simulations per siteOn average 10 simulations per site• Total of over 1000 simulated site yearsTotal of over 1000 simulated site years

Num Model Num Model1 Agro-IBIS 16 GFDL LSM2 BEPS 17 GTEC3 Biome-BGC 18 ISAM4 Can-IBIS 19 ISOLSM5 CLASS-CTEM (TRIPLEX) 19 LoTEC6 CLM-CASA' 20 LoTEC-DA7 CLM-CN 21 LPJml8 CN-CLASS 22 ORCHIDEE9 DAYCENT 23 ORCHIDEE-STICS10 DLEM 24 SiB311 DNDC 25 SiBCASA12 ecosys 26 SiBCrop13 ED2 27 SIPNET14 EDCM 28 SSiB215 EPIC 29 TECO

Analysis ProjectsAnalysis ProjectsNum Status Title Lead

1 Accepted Seasonal NEE Christopher Schwam2 Writing Continental flux estimates at flux tower sites Brett Raczka3 Writing Flux Uncertainty Analysis Alan Barr4 Writing Gap-Filled Weather Uncertainty Dan Ricciuto5 Writing GPP Comparison Kevin Schaefer6 Writing Spectral Analysis Michael Dietze7 Writing Agriculture Sites Erandi Lokupitiya8 Started Disturbance History effect on fluxes Peter Thornton9 Started Forest Ecosystems at diurnal to seasonal time scales Bill Munger

10 Started Hot Spots in Inter-annual Variability Guerric Lemaire11 Started Intra- and Inter-Model Uncertainties for TECO Ensheng Weng12 Started Isotope analysis Chun-Ta Lai13 Started Model Parameter Comparison Hans Verbeeck14 Started Phenology Andrew Richardson15 Started Sensible and Latent Heat fluxes Alok Sahoo16 TBD Algorithm Comparison Dave Hollinger17 TBD Biomass Comparison Leo Liu18 TBD Eastern Temperate Forests Michael Dietze19 TBD Env. Conditions: Soil Temperature, Moisture, & Snow TBD20 TBD Fluxes and soil temperatures at permafrost sites Kevin Schaefer21 TBD Grassland Sites TBD22 TBD Nutrient cycling and Carbon Fluxes Peter Thornton23 TBD Precipitation Patterns and Carbon Flux Sebastian Leuzinger24 TBD Prognostic LAI, Soil Temperature and Soil Moisture TBD25 TBD Representation Error for Transport Inversions Scott Denning26 TBD Spatial Residual Analysis Philippe Peylin27 TBD Temperature and Light Response Curves TBD28 TBD Weather Events and Carbon Fluxes Hanqin Tian29 TBD Wetland Sites Ankur Desai

Selected resultsSelected resultsObserved flux uncertainty (Observed flux uncertainty (Barr et alBarr et al.).)

• NEE: random, U* filtering, gap-fillingNEE: random, U* filtering, gap-filling• GPP & Re: random, U* filtering, gap-filling, partitioningGPP & Re: random, U* filtering, gap-filling, partitioning

Random Uncertainty U* Threshold Uncertainty

Selected resultsSelected resultsOverall model performance (Overall model performance (Schwalm et alSchwalm et al.).)

Tay

lor

Ski

ll

Normalized Mean Absolute Error Chi-squared

Based on monthly model-data differencesLarge spread among models, sites

Perfect Model

Taylor Skill by Model Characteristics(Schwalm et al.[2010])

Spectral NEE Error (Spectral NEE Error (Dietze et alDietze et al.).)

Diurnal

Annual

Noise level based on NEE observation uncertainty

Largest errors associated with diurnal and annual cycles

Large variation in performance at synoptic scales

Phenology (Phenology (Richardson et alRichardson et al.).)

• Harvard ForestHarvard Forest• Leafout too earlyLeafout too early• Senescence too late Senescence too late • Errors of 25-50 days Errors of 25-50 days

based on NEEbased on NEE• Errors in GPP/NEE Errors in GPP/NEE

correlated with LAI in correlated with LAI in spring, but not spring, but not autumnautumn

Future workFuture work• Objectives for new simulationsObjectives for new simulations

• Non steady-state Non steady-state Previous simulations assumed steady state, not consistent with observed fluxesPrevious simulations assumed steady state, not consistent with observed fluxes Incorporate known information about disturbance historyIncorporate known information about disturbance history

• Under-analyzed biomesUnder-analyzed biomes e.g. wetland, tundrae.g. wetland, tundra

• Model sensitivity analysesModel sensitivity analyses Good idea of inter-model uncertainty, but intra-model uncertainty?Good idea of inter-model uncertainty, but intra-model uncertainty? What are the key parameters?What are the key parameters?

• Recruit more modeling teamsRecruit more modeling teams• Invite wetland modeling teamsInvite wetland modeling teams• Expand number of IPCC GCMsExpand number of IPCC GCMs

• Coordinate with other syntheses Coordinate with other syntheses • LBA DMIPLBA DMIP• NACP regional interim synthesis, MsTIMIPNACP regional interim synthesis, MsTIMIP

• Make our database more visible, user-friendlyMake our database more visible, user-friendly• 29 potential analysis teams making use of interim synthesis dataset29 potential analysis teams making use of interim synthesis dataset• Long-term, dynamic dataset Long-term, dynamic dataset • Coordinate with CDIAC, La Thuile, ESG, other activitiesCoordinate with CDIAC, La Thuile, ESG, other activities

SummarySummary• Highly collaborative effort, made possible byHighly collaborative effort, made possible by

• Efforts (largely unfunded) of model and tower investigators Efforts (largely unfunded) of model and tower investigators • Bringing together data, model and observation communitiesBringing together data, model and observation communities• A productive series of workshops discussing protocol, analysisA productive series of workshops discussing protocol, analysis• Standardized inputs and flux observationsStandardized inputs and flux observations• Coordination by NACP team, CDIAC, FLUXNET to determine and Coordination by NACP team, CDIAC, FLUXNET to determine and

collect necessary ancillary data for models not already available collect necessary ancillary data for models not already available

• Valuable dataset for model developersValuable dataset for model developers• First formal estimates of observation uncertainty in a standard datasetFirst formal estimates of observation uncertainty in a standard dataset• Testbed for regional/global models to validate against a large Testbed for regional/global models to validate against a large

observation networkobservation network• Opportunity for model, observation PIs to learn from each otherOpportunity for model, observation PIs to learn from each other

Additional SlidesAdditional Slides

Missing AffiliationsMissing Affiliations

DNDCEDCMLoTEC-DATECO

Missing Model Affiliations

Missing Site Affiliations

CA-Mer

US-Ho1US-IB1

US-IB2

US-MOz

US-Ton

US-Var

Lessons LearnedLessons Learned• Baseline parameter vs. structureBaseline parameter vs. structure

• Std vs. CADM parameter runsStd vs. CADM parameter runs

• Better way to process submission filesBetter way to process submission files• Better IC criteria and dataBetter IC criteria and data• Need so many sites?Need so many sites?

• Focus on what we do not haveFocus on what we do not have• Not random missing sites: which are missing?Not random missing sites: which are missing?

• NSS vs SS runsNSS vs SS runs• Coordinate model needs with Site data collectionsCoordinate model needs with Site data collections

• Better detail site info/ancillary data (tree bands, resp chambers)Better detail site info/ancillary data (tree bands, resp chambers)• Mike Dietze leaf level photosynthesisMike Dietze leaf level photosynthesis• Need support for background/CADM dataNeed support for background/CADM data• Weeks per CADM fileWeeks per CADM file• Central lab model e.g., for leaf N Central lab model e.g., for leaf N • Encourage repository for data, esp ancillary dataEncourage repository for data, esp ancillary data

Lessons learnedLessons learned• Chance to improve model (not tuning, use Chance to improve model (not tuning, use

CADM)CADM)• Clarify protocol not “out of box”Clarify protocol not “out of box”

• Need better phenology obsNeed better phenology obs

New siteNew site

• BondvilleBondville• Not much anc dataNot much anc data

• PermafrostPermafrost• Daring Lake, Toolik Lake, other Canadian sitesDaring Lake, Toolik Lake, other Canadian sites• 8-mile lake (schuur)8-mile lake (schuur)

• Chronosequence sites (priority 3 UCI)Chronosequence sites (priority 3 UCI)• Augment under rep biomesAugment under rep biomes

• Grassland, savanna, shrubland, wetlandsGrassland, savanna, shrubland, wetlands

Next roundNext round• ObjectivesObjectives

• Non-SSNon-SS• Under-analyzed biomesUnder-analyzed biomes• Sensitivity analysesSensitivity analyses

Survey existing analysesSurvey existing analyses OAT few sites survey paramOAT few sites survey param

• Recruit Model teamsRecruit Model teams• Invite wetland model teamsInvite wetland model teams• IPCC GCMsIPCC GCMs

• Coordinate with LBA DMIPCoordinate with LBA DMIP• LULC input to models (Peter T.)LULC input to models (Peter T.)• Weather (Dan R.)Weather (Dan R.)• SupportSupport

• Money to model teams, proposal to CCIWGMoney to model teams, proposal to CCIWG• Postdoc to coordinatePostdoc to coordinate

Improving InfrastructureImproving Infrastructure

• Model submission tool (alma_var)Model submission tool (alma_var)

• Standard model processing (Dan Ricciuto)Standard model processing (Dan Ricciuto)

• Tool to Process Barr et al. uncertainty filesTool to Process Barr et al. uncertainty files

• Manpower (Barbara Jackson)Manpower (Barbara Jackson)

• Consistency across productsConsistency across products

• Update Wiki and FTP Update Wiki and FTP

Inter-annual (Raczka et al.)Inter-annual (Raczka et al.)

Annual total NEE at US-Ha1Annual total NEE at US-Ha1

NEE Seasonal Cycle (Schwalm et al.)NEE Seasonal Cycle (Schwalm et al.)

Taylor Plot: All Sites• Forest sites better Forest sites better

than non-forest than non-forest • Ag models do best Ag models do best

at Ag sitesat Ag sites• Mean (P) and Mean (P) and

optimized model optimized model (N) do well(N) do well

NEE Error by Time Scale (Dietze et al.)NEE Error by Time Scale (Dietze et al.)

GPP All Sites (Schaefer et al.)GPP All Sites (Schaefer et al.)Model chisqr RMSE

(-) (mole m-2 s-1)

Mean 1.59 4.02LOTEC 1.96 4.45SIB 2.13 4.43ISOLSM 2.31 3.93DLEM 2.35 5.9SIBCASA 2.51 4.5BIOMEBGC 2.96 7.24LPJ 3.02 6ISAM 3.04 5.15BEPS 3.09 6.49ECLUEEDCM 3.14 6.63ECOSYS 3.22 4.78SIBCROP 3.44 7.4CAN-IBIS 3.65 5.13ORCHIDEE 4.07 5.9DNDC 4.16 12.15TRIPLEX 4.24 8.36ED2 4.27 6.96AGROIBIS 4.39 7.97SSIB2 8.11 6.72TECO 10.66 7.76CNCLASS 15.63 12.35

Mean is bestOptimized

Unit Problems?

Top 3 models for NEE

GPP Bias and PhenologyGPP Bias and Phenology

-8

-6

-4

-2

0

2

4

1 2 3 4 5 6 7 8 9 10 11 12

Month

Bia

s (

mol

e/m

2/s) CRO

CSH

DBF

ENF

GRA

MF

WET

WSA

Bia

s (

mol

m-2 s

-1)

CA-Ca1US-Ne3

What does all this mean?What does all this mean?

• Model performance varies with structureModel performance varies with structure

• Peak NEE error at 1 day and 1 year periodPeak NEE error at 1 day and 1 year period

• Bias & phenology dominate GPP errorBias & phenology dominate GPP error

• GPP error large source of NEE errorGPP error large source of NEE error

• Must link model structure with performanceMust link model structure with performance

Flux Tower SitesFlux Tower SitesNum Priority Site Code Num Priority Site Code Num Priority Site Code

1 1 CA-Ca1 21 1 US-Los 41 2 CA-SJ32 1 CA-Gro 22 1 US-Me2 42 2 CA-TP13 1 CA-Let 23 1 US-MMS 43 2 CA-TP2

4 1 CA-Man 24 1 US-MOz 44 2 CA-TP3

5 1 CA-Mer 25 1 US-Ne1 45 2 US-Me3

6 1 CA-Oas 26 1 US-Ne2 46 2 US-Me4

7 1 CA-Obs 27 1 US-Ne3 47 2 US-Me5

8 1 CA-Ojp 28 1 US-NR1 48 3 CA-NS1

9 1 CA-Qfo 29 1 US-PFa 49 3 CA-NS2

10 1 CA-TP4 30 1 US-Shd 50 3 CA-NS3

11 1 CA-WP1 31 1 US-SO2 51 3 CA-NS4

12 1 US-ARM 32 1 US-Syv 52 3 CA-NS5

13 1 US-Atq 33 1 US-Ton 53 3 CA-NS6

14 1 US-Brw 34 1 US-UMB 54 3 CA-NS7

15 1 US-Dk2 35 1 US-Var 55 3 US-Blo

16 1 US-Dk3 36 1 US-WCr 56 3 US-Bo1

17 1 US-Ha1 37 2 CA-Ca2 57 3 US-FPe

18 1 US-Ho1 38 2 CA-Ca3 58 3 US-Ivo

19 1 US-IB1 39 2 CA-SJ1

20 1 US-IB2 40 2 CA-SJ2

Disturbance UncertaintyDisturbance UncertaintyORCHIDEE at 1850 burn site, ManitobaORCHIDEE at 1850 burn site, Manitoba

NEE Seasonal CycleNEE Seasonal Cycle

US-UMB CA-MerCA-Ca1

Best Typical Worst

GPP Seasonal CycleGPP Seasonal CycleCA-Ca1 US-Ne3 CA-Mer

Best Typical Worst

NEE Diurnal CycleNEE Diurnal Cycle

CA-Ca1 CA-Obs US-Ha1

Best Typical Worst

GPP Diurnal CycleGPP Diurnal Cycle

CA-Ca1 CA-Obs CA-Oas

Best Typical Worst

Uncertainty at Diurnal Time ScaleUncertainty at Diurnal Time Scale

Corn Year

Soybean Year

Mead rain-fed corn-soy rotation site (Nebraska)Mead rain-fed corn-soy rotation site (Nebraska)

Observed Flux Uncertainty Observed Flux Uncertainty

(Based on Richardson et al., 2006, Ag. For. Met. 136:1-18)

-35

239

BEPS

CNCLASS

ISOLSM

ecosys

SiBCASA

SiB

LoTEC_DA

can-ibis