Latest Update: August 14, 2007 GEOS-5 AGCM Tutorial Using The GMAO GEOS-5 AGCM Software Integration and Visualization Office NASA/GSFC Latest Update: August 14, 2007 GEOS-5 AGCM Tutorial About GEOS-5 Developed by the NASA/GSFC Global Modeling and Assimilation Office (http://gmao.gsfc.nasa.gov)http://gmao.gsfc.nasa.gov First model to be constructed ground up based upon Earth System Modeling Framework Designed for weather & climate research Makes use of finite-volume dynamical core (Lin, 2004) About GEOS-5 Latest Update: August 14, 2007 GEOS-5 AGCM Tutorial Outline Model Overview Getting the code Directory Structure Browsing the code (Understand for Fortran) Setting your Environment Building the code Setting up an experiment The run.script Submitting/Monitoring Jobs at NCCS Diagnostics Post-Processing Verification Quick Start Guide Where to get help Latest Update: August 14, 2007 GEOS-5 AGCM Tutorial Lets Get an interactive job in the Training Queue Interactive Batch qsub -I -l ncpus=4 -l walltime=8:00:00 -W group_list=k3002 -q training /usr/bin/tcsh Latest Update: August 14, 2007 GEOS-5 AGCM Tutorial GEOS-5 Overview from the GMAO The Goddard Earth Observing System Model, Version 5 (GEOS-5) is a system of models integrated using the Earth System Modeling Framework (ESMF). The system includes an atmospheric general circulation model (AGCM) developed for both weather and climate applications using the finite-volume dynamical core (Lin, 2004). Developments of GEOS-5 were guided by the need for realistic representation of tracer transports and stratospheric dynamics and the need for a realistic representation of moist processes for climate applications and for atmospheric data assimilation. The GEOS-5 AGCM is coupled to the Catchment Land Surface Model (LSM). The GEOS-5 atmospheric data assimilation system (DAS) couples the AGCM with the Grid-point Statistical Interpolation (GSI) analysis scheme being developed by the Environmental Modeling Center of the National Centers for Environmental Prediction (NCEP/EMC) and GMAO. The AGCM and the DAS have been run with 72 levels to 0.01 hPa and at horizontal resolutions from 2 to 0.25 . Latest Update: August 14, 2007 GEOS-5 AGCM Tutorial GEOS-5 Overview from the GMAO The Aerochem component of GEOS-5 incorporates parameterized tropospheric chemistry, the stratospheric chemistry and GOCART aerosol modules from the Atmospheric Chemistry and Dynamics Branch at GSFC as well as a carbon transport module. The GEOS-5 Coupled Climate General Circulation Model (CGCM) couples the AGCM with the Poseidon V5 ocean model developed by Paul Schopf at George Mason University. The GMAO Ocean Data Assimilation System (ODAS) is fully integrated into GEOS- 5 using ESMF couplers. Other chemistry and ocean models will be included over time. The GEOS-5 architecture supports running each of the component models in stand-alone mode as well as in various coupled configurations. The GEOS-5 system undergoes continual development to support the scientific investigations of GMAO and collaborators. Major new releases can be expected every 6-12 months, with minor upgrade releases for bug fixes as needed. Latest Update: August 14, 2007 GEOS-5 AGCM Tutorial GEOS-5 GCM Documentationhttp://gmao.gsfc.nasa.gov/systems/geos5/test.phphttp://gmao.gsfc.nasa.gov/systems/geos5/test.php http://gmao.gsfc.nasa.gov/systems/geos5/index_arch.phphttp://gmao.gsfc.nasa.gov/systems/geos5/index_arch.phphttp://map.nasa.gov/GMAO_GEOS5.htmlhttp://map.nasa.gov/GMAO_GEOS5.html http://map.nasa.gov/GEOSGCMbrowser/index.htmlhttp://map.nasa.gov/GEOSGCMbrowser/index.html http://map.nasa.gov/GEOSGCMbrowser/GEOSgcm_5_lvls.jpghttp://map.nasa.gov/GEOSGCMbrowser/GEOSgcm_5_lvls.jpg Latest Update: August 14, 2007 GEOS-5 AGCM Tutorial GEOS-5 GCM Structure Latest Update: August 14, 2007 GEOS-5 AGCM Tutorial Getting the Code Sourcemotel / CVS cd $HOME cvs -z3 co -r GEOSagcm-Eros-beta7p12 GEOSGCM Tarball from MAP website cd $HOME tar -zxvf GEOSagcm-Eros-beta7p12.tgz Latest Update: August 14, 2007 GEOS-5 AGCM Tutorial GEOSGCM Model Directory Structure src src/GNUmakefile src/Config src/Application src/Shared/GEOS_Shared src/Shared/GEOS_Base src/Shared/GEOS_Util src/Shared/GEOS_Util/post src/Shared/GEOS_Util/plots src/Shared/GMAO_pilgrim src/GEOShistory_GridComp src/GEOSgcm_GridComp src/GEOSgcm_GridComp/GEOSogcm_GridComp src/GEOSgcm_GridComp/GEOSagcm_GridComp src/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSsuperdyn_GridComp src/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSsuperdyn_GridComp/FVdycore_GridComp src/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSsuperdyn_GridComp/GEOSgwd_GridComp src/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp src/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSturbulence_GridComp src/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSradiation_GridComp src/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSradiation_GridComp/GEOSsolar_GridComp src/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSradiation_GridComp/GEOSirrad_GridComp src/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSmoist_GridComp src/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/GEOSpchem_GridComp src/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp src/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/GEOSland_GridComp src/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/GEOSland_GridComp/GEOScatch_GridComp Latest Update: August 14, 2007 GEOS-5 AGCM Tutorial Understand for Fortran What is it? An interactive development environment tool providing reverse engineering, automatic documentation, metrics and cross referencing of FORTRAN source code. Uses Browsing source code Understanding connections in the code Understanding variable declaration/usage Viewing use/use-by/invocation trees Generating in-depth reports References http://www.scitools.com/uf.htmlhttp://www.scitools.com/uf.html http://scitools.com/documents/manuals/pdf/understand_f.pdfhttp://scitools.com/documents/manuals/pdf/understand_f.pdf Latest Update: August 14, 2007 GEOS-5 AGCM Tutorial Creating your development branch Make a branch: You have checked out a tagged version. You cannot commit changes to a tagged version, since that would make the tag meaningless. Before committing your changes, you must create a new branch and then commit to the branch: cvs tag b Branch_Name $files where $files is the list of files that you have modified, and Branch_Name is a name of your choice. The branch name should be somewhat descriptive, including things such as the tag you are branching from (eg., b_GEOSagcm-Eros-beta7p12) or the name of a control experiment (eg., b_e0247). Once the branch has been created, you can update your files to that branch: cvs upd -r Branch_Name $files This does not change the files you modified, only their cvs status in your sandbox. Commit: To commit your changes, simply type: cvs commit $files Tag: Finally you are ready to tag ALL files (newly modified as well as untouched) to reflect your new experiment. At the GEOSGCM/src level: cvs tag Your_Tag_Name Latest Update: August 14, 2007 GEOS-5 AGCM Tutorial Building on Palm/Explore at NCCS Set Module Environment (preferably within.cshrc or.login): module purge module load intel-comp module load mpt nas module load scsl setenv BASEDIR /share/ESMA/baselibs/v2_2rp2.34meta setenv LD_LIBRARY_PATH ${LD_LIBRARY_PATH}:/share/ESMA/baselibs/v2_2rp2.34meta/Linux/lib setenv GEOSUTIL ~/GEOSGCM/src/Shared/GEOS_Util Issue install command: cd ~/GEOSGCM/src gmake install Latest Update: August 14, 2007 GEOS-5 AGCM Tutorial Building the Code Installation gmake install builds the executable Dependencies Generated during the build process May reduce recompilation times after changes Errors/Warnings Warnings /home1/wputman/GEOSagcm-Eros-beta7p12/GEOSGCM/Config/GMAO_base.mk:160: WARNING: No environment check performed. fortcom: Warning: mpi_isend.F90, line 12: A dummy argument with an explicit INTENT(OUT) declaration is not given an explicit value. [REQUEST] comm, request, ierror) ^ fortcom: Warning: mpi_recv.F90, line 11: A dummy argument with an explicit INTENT(OUT) declaration is not given an explicit value. [STATUS] subroutine mpi_recv( buf, count, type, pe, tag, comm, status, ierror) ^ fortcom: Warning: mpi_irecv.F90, line 11: A dummy argument with an explicit INTENT(OUT) declaration is not given an explicit value. [REQUEST] subroutine mpi_irecv( buf, count, type, pe, tag, comm, request, ierror) ^ fortcom: Warning: cloudnew.F90, line 100: A dummy argument with an explicit INTENT(OUT) declaration is not given an explicit value. [CLDNCCN] CLDNCCN, & ^ Where is my executable? ~/GEOSGCM/src/Application/GEOSgcm.x Latest Update: August 14, 2007 GEOS-5 AGCM Tutorial Setting up an Experiment Experiment Directory mkdir -p $HOME/geos5/e0247 cd $HOME/geos5/e0247 cp $HOME/GEOSGCM/src/Application/run.script. edit run.script (change group ID from g0620 to [k3002]) Run Directory (Work/Scratch space) Work directory mkdir /nobackup/$USER/e0247 cd /nobackup/$USER/e0247 ln -s $HOME/GEOSGCM/src/Application/GEOSgcm.x. cp /nobackup/wputman/GEOS-5.0_restarts/*. Scratch directory (where the model runs) /nobackup/$USER/e0247/scratch (ONLY if $TMPDIR is not defined on the system) Mass Storage Created by run.script /output/$USER/GEOS5.0/e0247 Latest Update: August 14, 2007 GEOS-5 AGCM Tutorial Getting to know the run.script Basic modifications processor count MPI layout resolution timestep begin/end dates Job segments (YYYYMMDD HHMMS) Iterations/counters diagnostics Latest Update: August 14, 2007 GEOS-5 AGCM Tutorial Running Your Experiment PBS Batch Set appropriate group ID qsub run.script Interactive qsub -I -l ncpus=32 -l walltime=2:00:00 -W group_list=your_gid /usr/bin/tcsh Monitoring job progress qstat -s palm user_id general e :00 R 01:21 Job run at Tue Nov 21 at 17:12 on e1:ssinodes=120:ncpus=120:mem= ssh e1 ls -l /pbs/spool/* ssh e1 tail -f /pbs/spool/ palm.OU Latest Update: August 14, 2007 GEOS-5 AGCM Tutorial Diagnostics Where are they defined History.rc file created in run.script Example list(1)%filename: 'geosgcm_prog', list(1)%template: '%y4%m2%d2_%h2%n2z', list(1)%format: 'CFIO', list(1)%frequency: , list(1)%fields: 'PHIS', 'SUPERDYNAMICS', 'SLP', 'DYN', 'U', 'DYN', 'V', 'DYN', 'T', 'DYN', 'PS', 'DYN', 'DELP', 'DYN', 'OMEGA', 'DYN', 'QV', 'MOIST', 'H2O_PROD', 'CHEMISTRY', 'RH2', 'MOIST', 'RH', :: An experiment may have Multiple output streams Varying formats (binary [raw] or HDF [CFIO]) Consistent vertical dimension 1 to LM for vertical centered variables 1 to LM+1 for edge variables surface or 2D variables The default output streams in run.script are required for the GEOS_Util post- processing plotting utility to execute cleanly. Post-processing of eta to pressure level conversion should still occur Latest Update: August 14, 2007 GEOS-5 AGCM TutorialDiagnostics Options for Output Streams filename Character string defining the filename of a particular diagnostic output stream. template Character string defining the time stamping template following GraDS conventions. format Character string defining the file format (flat of CFIO) flat (raw binary) is default, CFIO is HDF. mode Character string equal to "instantaneous" or "time-averaged". Default = "instantaneous". descr Character string equal to the list description. Defaults to "expdsc". frequency Integer (HHMMSS) for the frequency of output. Default = acc_interval Integer (HHMMSS) for the acculation interval (