Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

WRF-CMAQ 2-way coupled system: Part IDavid Wong, Jonathan Pleim, Rohit Mathur, Robert Gilliam, Tanya Otte, Jeffery YoungNERL/AMD

Frank Binkowski and Aijun XiuInstitute for the Environment, UNC

CMAS 08October 6-8, 2008

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Outline

Current operation schemeMotivationCoupling issuesDesign and featuresOverall coupled structurePreliminary results

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Current operation scheme

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Motivation

Shortcomings of the current way of running met model- MCIP-CMAQ• 32-bit data storage in files• excessive data interpolation• lack of feedback (direct and indirect

effect) from CMAQ to the met model

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Coupling Issues

Compatibility of met model and CMAQ• model physical aspects• computational aspects

Coupling tool

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Coupling Issues: compatibility

Model physical aspects• map projection• coordinate system and grid format• layer structure• time step size• etc

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Coupling Issues: compatibility (cont’d)

Computational aspects• domain decomposition• processor and subdomain mapping

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Coupling Issues: compatibility (cont’d)

Runtime System Library, RSLRSL (MM5 and WRF), RSL-Lite (WRF)RSL-Lite is slightly faster than RSL

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Coupling Issues: Tools

Earth System Modeling Framework (ESMF)

Other choices: OpenMI (UK), Model Coupling Toolkit (MCT) (ANL)

M3IO IOAPI Buffered file

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Design Strategy

Maintain integrity of WRF and CMAQ modelsMinimal code changes:

• Keeping the same IOAPI calls• Easy to incorporate new version of WRF

Efficiency: Low coupling overhead• buffered file for data transfer (forward and backward)• buffered file always holds two time steps of data

allowing interpolation Flexibility: easy to add feedback parameters and outputs

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Features

Flexible time stepping • CMAQ can be called every WRF

timestep or at any user defined multiple Simple run time switch of feedbackSubdomain trackingSwitch back to disk I/O file for uncoupled mode

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

WRF-CMAQ system overview

WRF CMAQSolve.F

physics drivers

dynamics

aq_prep

CMAQ (time, t)

aerosol data

radiationmicrophysics

emissiondatavdiff

biogenic emisplume rise

surface fluxhadvzadvhdiffcldprocchemaero

feedback

initialconditions

boundaryconditions

met data

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Example of a call frequency ratio of 4

time line

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Preliminary results

Offline – 4:1O3 PM25

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Preliminary results (cont’d)

4:1 – 1:1O3 PM25

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Run Exec. Time

Met Only

MCIP

Offline CMAQ

Inline Model 4:1

Inline Model 1:1

Execution time

4:45:022:15:23

0:04:21

2:25:18

4:52:07

9:02:52

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Preliminary results (cont’d)

4:1 w/o – w/ feedback

O3 PM25

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Preliminary results (cont’d)w/ feedback 4:1 – 1:1

O3 PM25

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Preliminary results (cont’d)

w/ feedback 4x8 – 2x4O3 PM25

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

Execution time (cont’d)

Exec. time

w/o feedback w/ feedbackcase

2x4 1:1

1:1

1:1

4:1

4:1

4:1

4x4

4x8

16:36:04

8:18:00

9:34:28

5:20:56

5:55:10

3:34:47

9:26:11

5:13:50

Office of Research and DevelopmentAtmospheric Modeling Division, National Exposure Research Laboratory

To be continued …