CESM2/CIME Update

Mariana VertensteinCESM Software Engineering Group

Outline

What is CIME revisited

CIME5 improvements

CMEPS - New community coupling infrastructure

Data Assimilation in CESM

Priorities for the next year

InfrastructurePUBLICOpenSourceGithub

Repository

Paradigm forDOE, NOAA, NSF

InfrastructureCollaborations

Driver-CouplerData Models

ScriptsSystem/Unit testing

Mapping Utilities

DOE/ACME

SciencecodeRestrictedorPublic

Repositories

ESPC and/orNOAA/NEMS

CESM

CIME (new python-based CESM infrastructure)

addresses needs of multiple efforts

MPAS/WRF

Configure/Build/Testing/Archiving Scripts

all python based as of CIME5

Driver/Coupler

CIME

PIO

Data Models

Statistical VerificationTool (PyCECT)

Mapping weight generation and checking tools (ESMF_Regrid and runoff map maker)

Post Processing/Diagnostics

Workflow Tool(Cylc)

Externals

MCTTimers

MCT Driver

ACME/CESM

Data, Stub,

CPLTestModels

CIMECORE CIME

functionality PIO

Externals

MCTTimers

Configure/Build/Test/Archive Case Control System

all python based

NUOPCDriver/

Mediator

Both MCT and NUOPCcaps

In Progress

CIME5 Improvements

CIME5 improvements New extensive documentation for both novice and expert users

http://esmci.github.io/cime/

CIME5 improvements (cont) Robustness: Extensive error checking and error reporting in Case

Control System

Porting New simplified porting process (can easily port CIME to a

MAC, linux cluster ….)

New python based component namelist generation New namelist definition file that combines definition and

default values for namelist entry Implemented in CIME driver and data models Implemented in CESM CICE, RTM, MOSART

components

CIME5 Improvements (cont) User Case Customization: Improved ability to define model grids and masks and user

specified compsets

Science functionality: DOCN: new aquaplanet SOM as well as analytic SST

aquaplanet modes DATM: new CPLHISTForcing can be used to spin up either

CLM or POP Smooth, conservative downscaling from lnd -> glc: bilinear interpolation + linear vertical interpolation, followed by a

global conservation correction New ESP component for DART data assimilation Multiple couplers for performance improvement in data

assimilation (upcoming)

Verification: new ultra-fast CECT test

New Coupling Infrastructure

Driver/Mediator

atmosphere

ocean

Inter-componentModel components only

communicate via exchangewith a driver/mediator

Intra-componentsub-components only

communicate via exchangewith a driver – requires sub-components to be modular

ATMDriver/

Mediator

physics chemistrydynamics

MCTDriver/

Mediator

Current CESM Coupling – data components permit flexible activation/deactivation of feedbacks

Introduction of alternative NUOPC driver-mediator will permit coupling to new prognostic components

River DROF MOSART

MCT

Land Ice CISMMCT

Sea Ice DICE CICE5

MCT

Ocean DOCN POP2

MCT

WAVE DWAV WW3

MCT

ESPDESP DART

MCT

ATM DATM CAM6

MCTLand

DLND CLM5

MCT

NUOPCDriver/

Mediator

Moving Forwards: New NUOPC Driver/Mediator will enable community collaboration with NGGPS

River DROF MOSART

NUOPC

Land Ice CISM

NUOPC

Sea Ice DICE CICE5

NUOPC

Ocean DOCN MOM6

HYCOMNUOPC

WAVE DWAV WW3

NUOPC

ESPDESP DART

NUOPC

ATM DATM CAM7

NUOPCLand

DLND CLM6

NUOPC

Advantages of NUOPC Infrastructure

Data-driven run-time configurable sequencing of driver phases currently this is all hard-coded and fragile

Run-time generation of mapping weights Eliminates need to compute AND maintain growing

number of mapping files generated offline

Support of ESMF/NUOPC The ESMF group is committed to supporting the needs of

CESM as this infrastructure is implemented and deployed

CMEPS Community Mediator for Earth Predictive Systems

New CIME-based community inter-component coupling infrastructure.

Based on the NOAA Environmental Modeling Center (NEMS) mediator.

Goal: have CMEPS v0 replicate the current cpl7 functionality and have all CESM components be CMEPS v0 compliant by year’s end.

Status: have a version working with CIME data and cpl-test components and CIME testing infrastructure Thanks to Tony Craig and lots of partnership from the ESMF group

CMEPS collaborations CMEPS could be run at at EMC as a possible replacement

for the NEMS mediator since it is so closely based on the NEMS mediator.

Exploratory work has already begun to determine if EMC is interested in leveraging the CIME case control system to utilize this mediator.

Also exploring collaborating with GFDL on a common CMEPS mediator

Would enable collaborating more with NOAA and operations – a recommendation from NCAR NSF SVT.

DATA Assimilation in CESM and new ESP (external

system processing) component

• CESM uses the DART ensemble Kalman Filter for data assimilation.

• Requires CESM to run multiple-instance simulations.

• Currently, CESM and DART are run separately, leading to inefficient use of simulation time.

Data Assimilation in CESM

Multiple instance capability and DART data assimilation

ocn obs

atm obsAtmosphere

(CAM)Land(CLM)

Ocean(POP)

SEA ICE(CICE)

River-Runoff

Atmosphere(CAM)Atmosphere

(CAM)

Land(CLM)Land

(CLM)

Ocean(POP)Ocean

(POP)

SEA ICE(CICE)SEA ICE

(CICE)

River-RunoffRiver-Runoff

DART

DART

CPL CPL CPL

Multi-coupler functionality contributed by Rafaelle Montuoro (NOAA/ESMF)

Currently, DART and CESM still run as separate executables –

ESP component allows data assimilation — DART — to interact directly with component data via I/O files

External tools can easily interact with CESM during simulation

MCTDriver/

Mediator

External System Processing

(ESP)DART

ATM CAM6

MCTOcean POP2

MCT

New ESP component facilitates coupling with data assimilation

Status New data ESP component introduced in CIME Pause/Resume has been implement in coupler and POP

Next Steps DART will be a prognostic ESP component of CESM coupled system Other CESM components will implement Pause/Resume (CAM is next)

Future enhancements Pause/Resume can also be extended to Pause/Rewind – to explore

new approaches to fault tolerance Implement in-memory communication between DART and CESM

Next Steps for CESM/CIME development

Next Steps in CIME/CESM development Move all CESM prognostic components to github!

Create CMEPS v0 NUOPC driver/mediator that duplicates the functionality of the current CESM cpl7 MCT driver/mediator

Provide new user-friendly capability to create new grids in CIME with one end-to-end utility

Incorporate DART as an ESP component and pause/resume functionality in the CESM system

Provide new method to specify initial conditions – needed to target prediction requirements

Collaborate on creation of Community Physics Driver needed for unified NCAR earth system prediction capability

Thank you!

Questions?

Extra Slides

CESM Post-Processing Tools New user-friendly integration of post-processing

diagnostics into CIME experiment cases Diagnostics (for atm, lnd, ice, ocn) are now parallel and much

faster: CMIP6 features have been added – transformation to CMOR

compliant output Regularly used on yellowstone and cheyenne for CESM2.0

development and ASD experiments

CESM2.0 support: Will be supported on yellowstone and cheyenne for the CESM2.0

release. Support for other platforms will follow in post-release updates.

https://github.com/NCAR/CESM_postprocessing

create_postprocessNew command included with the CESM_Postprocessing Tools

create_newcase –case mycase –compset B1850 –res r09_g16

create_postprocess –‐caseroot mycase

case.run

case.st_archive

atm_averages

lnd_averages

ice_averages

ocn_averages

atm_diagnos2cs

lnd_diagnos2cs

Ice_diagnos2cs

ocn_diagnos2cs

atm/hist

lnd/hist

ice/hist

ocn/hist

rof/hist

mycase/postprocess

mycase

case.submit

Post Processing and CIME