PE The NOAA Environmental Modeling System (NEMS)NMME Meeting February 2011 10 Dynamics, physics and...

NMME Meeting February 2011 1

William. M. LapentaActing Director

Environmental Modeling Center

NOAA/NWS/NCEP

NCEP

The NOAA Environmental

Modeling System (NEMS)


The EMC Mission…..

Develop and Enhance numerical guidance– Improve NCEP’s numerical forecast model

systems via:

• Scientific upgrades

• Optimization

• Additional observations

Transition operational numerical forecast models from research to operations

– Transform & integrate• Code

• Algorithms

• Techniques

– Manages and executes transition process including technical and system performance review before implementation

Maintain operational model suite– The scientific correctness and integrity of

operational forecast modeling systems

– Modify current operational system to adapt to ever-present external changes

EMC location

within the funnel

In response to operational requirements:

45%

25%

30%


Forecast

Uncertainty

Minutes

Hours

Days

1 Week

2 Week

Months

Seasons

Years

NWS Seamless Suite of Forecast

Products Spanning Weather and Climate

Fo

rec

as

t L

ea

d T

ime

Warnings & Alert

Coordination

Watches

Forecasts

Threats

Assessments

Guidance

Outlook

Benefits

NCEP Model Perspective

•North American Ensemble Forecast System

•Climate Forecast System

•Short-Range Ensemble Forecast

•Global Forecast System

•North American Mesoscale

•Rapid Update Cycle for Aviation

•Dispersion Models for DHS

•Global Ensemble Forecast System

Hurricane WRF & GFDLWaves

Real Time Ocean Forecast System

Space Weather

Tsunami


‘Mostly’Satellite+Radar

North American

MesoscaleNMM

North American Ensemble

Forecast System

GlobalForecastSystem

Rapid Updatefor AviationGSD’s RUC

CLIMATE

CFS

~3.5B Obs / Day

GFS + MOM3/4

Global Data

Assimilation

Air Quality

NMM + ARW

Dispersion, Ash,Smoke & DustARL’s HYSPLIT

Severe Weather

Oceans

HYCOM

WaveWatch III

NAM + EPA/

ARL’s CMAQ

Hurricane GFDL

HWRF (NMM)

Regional Data

Assimilation

NMM + ARW +

ETA + RSM

GFS + Canadian Global +… Very Short Range

Ensemble ForecastsTime-Lagged RUC+NAM

Short-Range

Ensemble Forecast

Linkage of Model Systems Within

Production Suite


5

NOAA Environmental Modeling System

A shared, portable, high performance software superstructure and infrastructure

For use in operational prediction models at National Centers for Environmental Prediction (NCEP)

National Unified Operational Prediction Capability (NUOPC) with Navy and Air Force

Eventual support to community through Developmental Test Center (DTC)

http://www.emc.ncep.noaa.gov/NEMS/


6

Motivation for NEMS

Develop a common superstructure for all NCEP models

Modularize large pieces of the models with ESMF components

and interfaces

Divide atmospheric models down into Dynamics and Physics

components but no further

Take history file I/O outside the science parts and into a

common Write component

Keep science code and parallelization code in the respective

models the same as before


7

NEMS Project Developers

Tom Black

Dusan Jovic

Jim Abeles

NAM

S Moorthi

Henry Juang

GFS

Jesse Meng

Jim Geiger

Land

Sarah Lu

Arlindo da Silva

GOCART

Tom Henderson

Jim Rosinski

FIM

Eugene Mirvis DTC


NEMS Component Structure

2

MAIN

EARTH(1:NM)

Ocean Atm Ice

NMM GFS FIM

Dyn Phy

Domains(1:ND)

Below the dashed line the source codes are organized by the model developers.

Wrt Dyn Phy Wrt Dyn Phy Wrt

NEMS

EnsembleCoupler

Chem

All boxes represent ESMF components.

NEMSLAYER

Start time

Run length

Start time

Run length

Ensemble

Component

Ensemble

Component

Choice of

physical domain

and component

Choice of

physical domain

and component

Component

execution

Component

execution

Individual

nests

Individual

nests

NOAA Environmental Modeling System

(NEMS)


9

Operational Implementation Q3FY11

12 km NAM will still run to 84 hr

Fixed domain nests run to 60 hr– 4 km CONUS – 6 km Alaska– 3 km HI & PR

• Single locatable 1.33 km (CONUS) or 1.5 km (Alaska) nest to 36hr

• Nests

• Static, 1-way

• Boundaries from parent every timestep

• Nest is ―grid-associated‖ with parent (same orientation w.r.t. earth)

• Moving nests and 2-way interaction under development


Dynamics, physics and chemistry run on the same grid in the same decomposition

GOCART does not own aerosol tracers (i.e, do not allocate aerosol tracer fields)

PHY2CHEM coupler component transfers/converts data from physics export state

to GOCART import state

– Convert units (e.g., precip rate, surface roughness)

– Calculations (e.g., soil wetness, tropopause pressure, relative humidity, air

density, geopotential height)

– Flip the vertical index for 3D fields from bottom-up to top-down

CHEM2PHY coupler component transfers data from GOCART export state to

physics export state

– Flip vertical index back to bottom-up

– Update 2d aerosol diagnostic fields

NEMS GFS Aerosol Component (NGAC)

Color Key

Generic Component

Generic Coupler

Completed Instance

Atmosphere

Dynamics PhysicsDyn-Phy

Coupler

NMM-B

Spectral

NAM Phy

GFS Phy

unified atmosphere

Including digital filter

GOCARTPhy-Chem

Coupler


11

NEMS Delivery Plans

• 2011

– GFS

– GEFS

– Postprocessor

– FIM

– Multimodel ensemble

– GRIB2 output

• 2012+

– NMM nested in GFS

– Moving nests

– Coupled ocean atmosphere

– Tiled land model

– netCDF output

– ARW


Questions Welcome


Forecast

Uncertainty

Minutes

Hours

Days

1 Week

2 Week

Months

Seasons

Years

NWS Seamless Suite of Forecast

Products Spanning Weather and Climate

Fo

rec

as

t L

ea

d T

ime

Warnings & Alert

Coordination

Watches

Forecasts

Threats

Assessments

Guidance

Outlook

Benefits

NCEP Model Perspective

•North American Ensemble Forecast System

•Climate Forecast System

•Short-Range Ensemble Forecast

•Global Forecast System

•North American Mesoscale

•Rapid Update Cycle for Aviation

•Dispersion Models for DHS

•Global Ensemble Forecast System

Hurricane WRF & GFDLWaves

Real Time Ocean Forecast System

Space Weather

Tsunami


STRATUS HWM

0

10

20

30

40

50

60

70

80

90

100

110

120

130

140

0:0

0:0

0

0:4

5:0

0

1:3

0:0

0

2:1

5:0

0

3:0

0:0

0

3:4

5:0

0

4:3

0:0

0

5:1

5:0

0

6:0

0:0

0

6:4

5:0

0

7:3

0:0

0

8:1

5:0

0

9:0

0:0

0

9:4

5:0

0

10

:30

:00

11

:15

:00

12

:00

:00

12

:45

:00

13

:30

:00

14

:15

:00

15

:00

:00

15

:45

:00

16

:30

:00

17

:15

:00

18

:00

:00

18

:45

:00

19

:30

:00

20

:15

:00

21

:00

:00

21

:45

:00

22

:30

:00

23

:15

:00

dev

ofs_atl

test

para

multi

misc

cdc

dgex

aqm

cdas

hiresw

sref

godas

cfs

mrf

firewx

merge

ruc2

hur

wave

gefs

gdas

gfs_analysisgfs

ndas

nam

Node use for 20091202

Production Suite on Supercomputer

Time of the day (utc)

Nu

mb

er

of N

od

es

STRATUS HWM

0

10

20

30

40

50

60

70

80

90

100

110

120

130

140

0:00

:00

0:45

:00

1:30

:00

2:15

:00

3:00

:00

3:45

:00

4:30

:00

5:15

:00

6:00

:00

6:45

:00

7:30

:00

8:15

:00

9:00

:00

9:45

:00

10:3

0:00

11:1

5:00

12:0

0:00

12:4

5:00

13:3

0:00

14:1

5:00

15:0

0:00

15:4

5:00

16:3

0:00

17:1

5:00

18:0

0:00

18:4

5:00

19:3

0:00

20:1

5:00

21:0

0:00

21:4

5:00

22:3

0:00

23:1

5:00

dev

ofs_atl

test

para

multi

misc

cdc

dgex

aqm

cdas

hiresw

sref

godas

cfs

mrf

firewx

merge

ruc2

hur

wave

gefs

gdas

gfs_analysisgfs

ndas

nam

Node use for 20091202

00 06 12 18

Development Work

January 2010

High Water Mark

Fence


Time of the day (utc)

Nu

mb

er

of N

od

es

00 06 12 18

Development Work

December 2010

High Water Mark

Fence

Capacity Change:

50% increase in production

80% decrease in development

Production Suite on Supercomputer


16

SR

EF

Pa

ra

GE

FS

Pa

ra


Comparison of the NCEP and ECMWF Production Suites

from a Computational Perspective……

CPU Utilization for 24 Hour Cycle of Production Suite on IBM P6

ECMWF NCEP

Operational

Production Suite

ECMWF: 2 cycles/day; NCEP 4 cycles/day

NCEP: note complexity of production suite (many colors)

ECMWF: fills the ―valleys‖ in production; NCEP developing capability


ECMWF NCEP

Operational

Production Suite




With compute capacity scaled……

ECMWF: High water mark is ~210 nodes (EPS)

NCEP: maximum available for production ~132 nodes

NCEP: High water mark is ~ 110 nodes


ECMWF NCEP

Operational

Production Suite




ECMWF to a IBM P7 in late FY11…….

ECMWF: High water mark is ~210 nodes (EPS)

NCEP: maximum available for production ~132 nodes

NCEP: High water mark is ~ 110 nodes


NCEP Aggressively Porting Codes

to Other Compute Centers

Milestone DateSystem

Configuration/Milestone

CPU

CoresTflops

Memory

Per Core

(GB)

October ’10 CMRS.1 Available to Users ~31,000 260 ~2.67

October ’11 CMRS.2 Available to Users ~78,000 720 2.0

Oct ’11 – Feb ‘12 CMRS.1 + CMRS.2 ~109,000 980

February ’12 CMRS.1 Upgrade ~41,000 386 2.0

Feb’12 – Sep’14 Final CMRS Configuration 119,000 1,106 2.0

•Current Porting Activities:HEVDAS development in Boulder

NASA—ARC, GSFC (JCSDA), projects

Benchmarks for computer acquisition

In discussion with NSF

•Coming Attractions:NOAA Climate Computing at ORNL (GAEA)

NOAA R&D at Fairmont WV– First drop early FY12

Upgrade of Operational Compute in FY14

Schedule for NOAA Computing at ORNL CRAY XT6 (GAEA)

PE The NOAA Environmental Modeling System (NEMS)NMME Meeting February 2011 10 Dynamics, physics and...

Documents

Transcript of PE The NOAA Environmental Modeling System (NEMS)NMME Meeting February 2011 10 Dynamics, physics and...