Www.esmf.ucar.edu Cecelia DeLuca, Don Stark, Chris Hill Arctic System Model Workshop May 20, 2008...
-
Upload
corey-oneal -
Category
Documents
-
view
213 -
download
0
Transcript of Www.esmf.ucar.edu Cecelia DeLuca, Don Stark, Chris Hill Arctic System Model Workshop May 20, 2008...
www.esmf.ucar.edu
Cecelia DeLuca, Don Stark, Chris HillArctic System Model WorkshopMay 20, 2008
Earth System Modeling Framework
www.esmf.ucar.edu
Outline
• About ESMF• Best Practices, On and Off List• Governance• Personal History• Conclusion
www.esmf.ucar.edu
About ESMF
• ESMF provides component wrappers with standard interfaces, a set of data structures for data exchanges, and customizable drivers.
• ESMF provides common utilities, such as data communications, regridding, time management, configuration, and message logging.
• Goals are component interoperability and software reuse.
www.esmf.ucar.edu
GEOS-5
surface fvcore gravity_wave_drag
history agcm
dynamics physics
chemistry moist_processes radiation turbulence
infrared solar lake land_ice data_ocean land
vegetation catchment
coupler
coupler coupler
coupler
coupler
coupler
coupler
• Each box is an ESMF component• ESMF State objects carry data between components• Every component (including couplers) has a standard interface to facilitate exchanges• The ESMF architecture enables the assembly of many different systems
ESMF Application Example
GEOS-5 AtmosphericGeneral Circulation ModelApplication Example
www.esmf.ucar.edu
Standardization of Component APIs
• There are only three ESMF component methods: Initialize, Run, and Finalize (I/R/F)
• Users create a Component by assigning their user code I/R/F methods to an ESMF Component type
• The ESMF Component calls back into the specific user-assigned methods
• I/R/F methods cascade down the tree• Small set of standard arguments:
Child GridComp “Atmosphere”
Parent GridComp “Hurricane Model”
Finalize
Child GridComp “Ocean”
Finalize
Child CplComp “Atm-Ocean Coupler”
Finalize
Call Initialize Call FinalizeCall Run
Initialize Run Finalize
Initialize
Initialize
Initialize
Run
Run
Run
AppDriver (“Main”)
Call Initialize Call FinalizeCall Run
call ESMF_CompRun (myComp, importState, exportState, clock, phase, blockingFlag, rc)
www.esmf.ucar.edu
Standardization of Data Structures• ESMF State data structures contain all data exchanged between
components• Flexibility in data representation - States can contain lists of varied
data structures, including Arrays, ArrayBundles, Fields, FieldBundles, and other States
The ESMF philosophy is to constrain the number of component methods and their arguments, but retain flexibility in the data structures used in inter-component exchanges
www.esmf.ucar.edu
ESMF Adoption - PARSSSEPrepare user code• Split user code into initialize, run and finalize methods and decide on components and coupling
fields and control flowAdapt data structures• Wrap native data structures either as
◦ ESMF Arrays (represent data in index space only)Regrid via user-supplied weights + sparse matrix multiply
◦ ESMF Field objects (Arrays + coordinates and metadata) Regrid via on-line weight generation
Pack Arrays or Fields into States.Wrap time information in ESMF Clocks.
Register user methods• Attach user code initialize, run and finalize methods to ESMF Components through a registration
processSchedule, Synchronize, and Send data between components• Write couplers using ESMF redistribution, sparse matrix multiply, regridding, or user-specified
transformationsExecute the application• Run components using an ESMF driver
www.esmf.ucar.edu
ESMF Earth Science Components & Models
HAF
GAIM
www.esmf.ucar.edu
ESMF Timeline • First phase started in 2002 with NASA funding• Second phase began in 2005 with a transition to multi-agency support and
management – NASA, NOAA, NSF, and DoD sponsors• ESMF component interfaces are the basis of programs at multiple agencies:
◦ DoD Battlespace Environments Institute (BEI)◦ NOAA National Environmental Modeling System (NEMS) ◦ NASA Modeling Analysis and Prediction Program for Climate Variability
and Change (MAP)◦ Community Sediment Transport Model (ONR), others…
• In operational use since August 2006, initially at the National Weather Service, ongoing transition of other ESMF coupled systems to operations
• Scope of current work will be completed in 2010• Ongoing diversified funding expected
www.esmf.ucar.edu
Release Path2002 2003 2004 2005 2006 2007 2008 2009 2010
ESMF v1Prototype
ESMF v2Components, VM and UtilsESMF_GridCompRun()
ESMF v3Index Space OperationsESMF_ArraySparseMatMul()
ESMF v4Grid OperationsESMF_GridCreate()ESMF_FieldRegrid()
ESMFv5StandardizationBuild, init, data types, error handling, …
Public releaseESMF v3.1.0r
Full system prototype
Building bottom-up
Standardization
www.esmf.ucar.edu
ESMF Package Description, the Basics• Open source, open development (browsable repository)• Components can run concurrently, sequentially or in mixed
mode• Serial or parallel• Single or multiple executable or combinations• Shared or distributed memory or hybrid • Support for model ensembles, including execution of multiple
instances in the same address space• 2000+ unit tests, system tests, and examples regression tested
nightly on 26+ platform/compiler combinations• Reference Manual, Users Guide and the examples therein
updated automatically with code changes
www.esmf.ucar.edu
ESMF Package Description, Coupling• Data transformations can be executed within a coupler
component, or arranged in a coupler component and executed directly between model components
• Coupling can be done in index space or physical space• Performance: < 5% overhead in time to solution vs customized
native approaches, highly scalable in performance and memory (performance reports online)
www.esmf.ucar.edu
Upcoming ReleasesPublic Release 3.1.0r, May 2008
• Main capabilities added since last public release - 2.2.2rp3, December 2006:
◦ General grid and data representation and parallel regridding support through an Array index space layer (user supplied interpolation weights) – logically rectangular grids, multi-patch grids, unstructured grids
◦ Representation of rectilinear and curvilinear grids in a Grid class (preparation for framework-generated interpolation weights)
◦ New ports including gfortran and Solaris
• Compact, simpler interface: cut number of public data handling methods in half (304 -> 147, with caveats)
www.esmf.ucar.edu
Upcoming Releases (cont.)Internal Release 3.1.1, June 2008
• Initial implementation of observational data streams
• Generation of regridding interpolation weights for logically rectangular grids, bilinear and higher order method
• Attribute class can store standardized metadata packets and represent metadata hierarchies - e.g., State metadata includes the metadata of Fields that are stored in it
• Initial implementation of unstructured grid interface
Patch Release 3.1.0rp*, Summer 2008
• Petascale optimizations for Array and ArrayBundle sparse matrix multiply
www.esmf.ucar.edu
Relationship to CCSM
• Passed CCSM Stage 1 evaluation of ESMF in May 2007, showing < 5% overhead, comparable memory use, no change to component internals.
• Result: creation of cpl7 based on ESMF design• Currently working on Stage 2 evaluation:
Creation of an ESMF version of cpl7 called ESMCoupler• Expected advantages:
◦ Greater interoperability with other components◦ More flexible architecture that doesn’t rely on a single central coupler◦ On-line regridding◦ Integrated handling of metadata
www.esmf.ucar.edu
Team CompositionTeam composition (blue is off-site)
1. Manager (agency coordination and technical coordination)2. Operations Manager (website, metrics, space issues, local administration)3. Integrator/Test lead (regression testing, release management)4. Tester for numerical methods
7 developers:5. Systems level/porting/build6. Low level data structures and architecture 7. Structured grids 8. Meshes9. High level numerical data structures 10. Language interfaces11. Utilities including calendaring12. Metadata and attributes
External and related13. External: ½ FTE performance testing14. Related: FTE for component distribution portal
www.esmf.ucar.edu
Distributed Development
GOAL - Everybody on the team has access to all information, current and past
• Archived email list where all development correspondence gets cc’d• Frequent telecons with minutes• Web browsable repository, mail summary on check-ins• Daily archived test results• Monthly archived metrics• Public archived trackers (bugs, feature requests, support requests,
etc.)
www.esmf.ucar.edu
ESMF Governance
Executive BoardStrategic Direction
Organizational ChangesBoard Appointments
Interagency Working GroupStakeholder Liaison
Programmatic Assessment & Feedback
Advisory BoardExternal Projects CoordinationGeneral Guidance & Evaluation
Functionality Change Requests
Joint Specification TeamRequirements Definition
Design and Code ReviewsExternal Code Contributions
Implementation Schedule
Resource Constraints
Collaborative DesignBeta Testing
Working Project
ExecutiveManagement
Reporting
Reporting
weekly
Core Development TeamProject Management
Software DevelopmentTesting & Maintenance
Distribution & User Supportdaily
annually
Change Review BoardDevelopment Priorities
Release Review & Approvalquarterly
• Multiple timescales, all staff levels• Places for structured argument
www.esmf.ucar.edu
Values• Community driven development and community ownership• Openness of project processes, management, code and information• Correctness• Commitment to a globally distributed and diverse development and
customer base• Simplicity• Efficiency• Public storage of project records and other information• Engagement
Web link for detail: http://www.esmf.ucar.edu/about_us/values.shtml
www.esmf.ucar.edu
How to Get Started & Get Help• Web meeting with the development team• Support list – [email protected]• Code Examples page and FAQ• Tutorials, on-line and in person• Annual Meeting (Fort Lauderdale, FL May 28-30, 2008)• Reference Manual and Users Guide
www.esmf.ucar.edu
Governance
Management of ESMF requires governance that recognizes social and cultural factors as well as technical factors
Main objectives of governance:• Enabling people to argue in a civilized, contained,
constructive way• Enabling people to set priorities and make decisions based
on resource realities• Coordinating at many levels, from developer to center director
to program manager
www.esmf.ucar.edu
Facilitating Science:Coupled Climate-Chemistry with ESMF
The image shows results from a version of the GEOS-5 atmosphericgeneral circulation model coupled to a stratospheric chemistry package(STRAT-CHEM), also developed at NASA, but independently of GEOS-5,which has now been made ESMF compliant.