A Comparative Performance Evaluation of the AURAMS and CMAQ Air Quality Modelling Systems

Steven C. Smyth, Weimin Jiang, Helmut Roth, and Fuquan YangICPET, National Research Council of Canada, Ottawa, Ontario

Michael D. Moran and Paul A. MakarMSC, Environment Canada, Toronto, Ontario

Véronique S. Bouchet and Hugo LandryCMC, Environment Canada, Dorval, Québec

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 2

Outline

• Introduction• AURAMS vs. CMAQ – Differences in science,

input file preparation, etc.

• O3, total PM2.5, and speciated PM2.5 performance comparison

• Summary and Conclusions

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 3

Introduction

• Many aspects of the AURAMS and CMAQ simulations were “aligned” to reduce some of the common sources of differences:– same input meteorology from Environment Canada’s GEM

model– same raw emissions inventories processed by SMOKE– same biogenic emissions model– same grid resolution

• Confidence that the differences in model results are caused by the AQ models themselves rather than by meteorological and/or emissions inputs

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 4

Modelling Systems

• CMAQ v4.6– SAPRC-99 chemical mechanism; AERO4; NRC PMx post-

processor

• AURAMS v1.3.1b– A Unified Regional Air-quality Modelling System– AQ modelling system with size- and composition-resolved PM– Designed to be a “one” atmosphere or “unified” model in order

to address a variety of interconnected tropospheric air pollution problems ranging from ground level O3 to PM to acid rain

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 5

AURAMS v1.3.1b (cont.)

• 9 PM species/components: sulphate (SU), nitrate (NI), ammonium (AM), black carbon (EC), primary organic aerosols (PC), secondary organic aerosols (OC), crustal material (CM), sea-salt (SE), and particle bound water (WA)

• 12 PM size distribution bins: 0.01 to 40.96 µm in diameter– Bins 1 thru 8 – PM2.5

– Bins 9 and 10 – PMC; where PM10-PM2.5 = PMC– Bins 11 and 12 – PM greater than 10 µm in diameter

• Gas phase chemistry – modified version of ADOM-II

• Includes sea-salt emissions but not chemistry at this time

• Zero-gradient lateral boundary conditions

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 6

Domains and Simulation period

• AURAMS - Polar Stereographic; true scale at 60°N; 150 x 106 grid; 42-km resolution

• CMAQ - Lambert conformal conic; standard parallels of 50°N and 70°N; 139 x 99 grid; 42-km resolution

• 01:00 July 1, 2002 to 00:00 July 30, 2002 UTC

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 7

Model Inputs - Meteorology

• GEM v3.2– AURAMS meteorological

pre-processor– GEM-MCIP (based on

MCIP v3.1)

• Overlapping grid cell comparison of surface fields – NMEs of 0.25% for pressure; 0.4% for temperature; 3.8% for specific humidity (HU)

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 8

Model Inputs - Emissions

• SMOKE v2.2

• Canadian Emissions– 2000 CAC inventory

• U.S. Emissions– 2001 CAIR

• Mexican Emissions– 1999 inventory

• Biogenic Emissions– BEISv3.09

• AURAMS – online• CMAQ – offline using

SMOKE

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 9

Model Inputs - Emissions (cont.)

• Point source processing– AURAMS – plume-rise of major point sources calculated within

CTM– CMAQ – meteorological data used to calculate plume rise

within SMOKE• Emissions files

– AURAMS: grams/sec• representative week of emissions for each month of simulation• 3 emissions files (non-mobile, mobile, minor-point) in RPN format• 1 emissions file (major-point sources) in ASCII format

– CMAQ: gaseous - moles/sec; PM - grams/sec• daily emissions files• single comprehensive file in I/O API format

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 10

Measurement Data

• O3 - hourly measurements from: the EC NAPS network (190 sites) and U.S. EPA AQS network (1087 sites)

• PM2.5 - hourly measurements from: NAPS (92 sites) and AQS (262 sites)

• Speciated PM2.5 - daily averaged measurements from: NAPS (17 sites) and U.S. EPA STN network (205 sites)

O3 Measurement Sites PM Measurement Sites

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 11

O3 Performance

statisticsO3 (ppb) daily peak O3 (ppb) daily low O3 (ppb)

AURAMS CMAQ AURAMS CMAQ AURAMS CMAQ

meas. mean 35.6 35.6 60.8 60.7 11.7 11.7

mod. mean 42.0 51.4 66.7 70.2 16.4 32.7

MB 6.4 15.8 5.9 9.4 4.6 20.9

NMB (%) 18 % 44 % 10 % 16 % 39 % 178 %

ME 16.2 18.8 16.4 14.3 11.0 21.9

NME (%) 46 % 53 % 27 % 24 % 94 % 187 %

r2 0.393 0.438 0.350 0.505 0.104 0.077• AURAMS lower bias• Similar levels of error• CMAQ over prediction mainly due to inability in predicting daily lows

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 12

O3 Performance (cont.)

• Both AURAMS and CMAQ over-predict daily peaks

• AURAMS much better at predicting daily lows

• Both models show correct diurnal patterns and overall trends in concentration level

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 13

Total PM2.5 Performance

statistics

total PM2.5

(µg m-3)

daily peak PM2.5

(µg m-3)

AURAMS CMAQ AURAMS CMAQ

meas. mean 14.4 14.4 28.3 28.1

mod. mean 12.2 5.1 21.6 9.1

MB -2.2 -9.3 -6.6 -19.0

NMB - 15 % - 64 % - 23 % - 68 %

ME 9.7 10.2 16.3 19.5

NME 67 % 71 % 58 % 69 %

r2 0.074 0.151 0.040 0.086

• AURAMS lower bias

• Similar levels of error

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 14

Total PM2.5 Performance (cont.)

• Both models under-predict PM2.5

• Forest-fires not included in emissions contributes to under-prediction in both models

• Much more PM2.5 sea-salt in AURAMS

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 15

PM2.5 Species Performance

statistics

PM2.5 SO4

(µg m-3)

PM2.5 NO3

(µg m-3)

PM2.5 NH4

(µg m-3)

AURAMS CMAQ AURAMS CMAQ AURAMS CMAQ

meas. mean 5.0 0.90 1.6

mod. mean 5.3 2.4 2.0 0.26 1.6 0.82

MB 0.3 -2.6 1.1 -0.64 0.0 -0.81

NMB (%) 6 % -51 % 121 % -71 % 1 % -50 %

ME 3.0 2.8 1.6 0.7 0.9 0.9

NME (%) 60 % 56 % 175 % 79 % 54 % 57 %

r2 0.367 0.524 0.397 0.437 0.428 0.458• AURAMS better bias for SO4 and NH4; similar levels of error• CMAQ better correlation

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 16

PM2.5 Species Performance (cont.)

statistics

PM2.5 EC

(µg m-3)

PM2.5 TOA

(µg m-3)

AURAMS CMAQ AURAMS CMAQ

meas. mean 0.52 10.6

mod. mean 0.28 0.34 3.9 0.9

MB -0.24 -0.18 -6.6 -9.6

NMB - 46 % - 36 % - 63 % - 91 %

ME 0.30 0.30 6.9 9.6

NME 58 % 58 % 66 % 91 %

r2 0.166 0.204 0.0005 0.007

• CMAQ better bias for EC; similar levels of error

• AURAMS much better performance for TOA– Due to difference

in SOA algorithms

• Poor TOA correlation for both models impacts overall correlation for total PM2.5 (AURAMS – 0.074; CMAQ = 0.151)

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 17

PM2.5 Species – Temporal Comparison

SO4

SOAPOA

NO3

EC

NH4

Other PM2.5

Sea-salt

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 18

PM2.5 Species – Spatial Comparison

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 19

PM2.5 Species – Spatial Comparison

(cont.)

• Similar spatial and temporal patterns for most species

• Sea-salt aerosols vastly different

• Concentration levels quite different

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 20

PM Composition

• PM2.5 sea-salt contributes over half to AURAMS total PM2.5 mass for all grid cells; only 5% in CMAQ

• For land grid cells only, PM2.5 sea-salt contributes 15% in AURAMS and 2% in CMAQ

• If sea-salt is excluded from model results, total PM2.5 performance still better in AURAMS results

PM composition avg. over all grid cells

PM composition avg. over land grid cells only

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 21

Discussion and Summary

• Similar levels of error for O3, total PM2.5, and most PM2.5 species

• AURAMS better bias for all species except PM2.5 nitrate and elemental carbon

• Enhanced AURAMS bias due to cancellation of positive and negative biases

• Sea-salts contribute much more to overall PM composition in AURAMS than CMAQ

– Does not impact overall conclusions regarding relative PM2.5 performance of the models

• Spatial and temporal patterns similar, but overall concentration levels quite different

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 22

Acknowledgements

• Radenko Pavlovic and Sylvain Ménard of Environment Canada– transfer of AURAMS code and help in understanding and compiling

various AURAMS related material• Wanmin Gong of EC

– help in identifying problem in AURAMS land-use file• Pollution Data Division of EC

– 2000 Canadian raw emissions inventories• U.S. EPA and CMAS

– U.S. emissions data, SMOKE, CMAQ, MCIP• Colorado State University

– VIEWS database for measurement data• Meteorological Service of Canada

– NAtChem database for measurement data• Environment Canada and the Program of Energy Research and

Development (PERD) for funding support

6th Annual Models-3 Conference, Chapel Hill, NC, October 1-3, 2007 23