Boulder, CO25 June 2004WRF/MM5 Users Workshop The WRF Portal Effort Brian Jewett, Robert Wilhelmson, & Matt Gilmore U. Illinois Atmospheric Sciences / NCSA Jay Alameda, Al Rossi & Shawn Hampton NCSA - Univ. Illinois, Urbana, IL Boulder, CO25 June 2004WRF/MM5 Users Workshop Portal Development Key work: In collaboration with Dennis Gannon, Indiana University Funded in part by MEAD NCSA Portal Team: Jay Alameda, project lead Al Rossi (OGRE) Shawn Hampton (OGRE GUI & application) Essential components will be part of LEAD ITR (next talk) Boulder, CO25 June 2004WRF/MM5 Users Workshop Why are we here? Or: So what? Several reasons: Boulder, CO25 June 2004WRF/MM5 Users Workshop Why are we here? Or: So what? Several reasons: Need to share data and code Run computationally intensive problems Ability to run where resources available (or best suited, e.g. data location) Ability to run at (or spread jobs across) other computing centers Split tasks among machines according to their purpose (analysis, vis, computation) Boulder, CO25 June 2004WRF/MM5 Users Workshop Motivation: A computationally intensive study - storm interaction Boulder, CO25 June 2004WRF/MM5 Users Workshop IL tornado outbreak Midwest severe reports Red: tornadoes Blue: hail Yellow: wind damage Over 30 tornadoes struck IL Some were strong (F2-F3) Most tornadoes were short-lived Boulder, CO25 June 2004WRF/MM5 Users Workshop IL tornado outbreak Analysis of radar data Storm splitting: one storm Radar reflectivity - western IL (red = heaviest rain/hail) Boulder, CO25 June 2004WRF/MM5 Users Workshop IL tornado outbreak Analysis of radar data becomes two. Radar reflectivity - western IL (red = heaviest rain/hail) Boulder, CO25 June 2004WRF/MM5 Users Workshop IL tornado outbreak Analysis of radar data Radar reflectivity - western IL (red = heaviest rain/hail) Storm merging: two storms Boulder, CO25 June 2004WRF/MM5 Users Workshop IL tornado outbreak Analysis of radar data become one. Radar reflectivity - western IL (red = heaviest rain/hail) Boulder, CO25 June 2004WRF/MM5 Users Workshop IL tornado outbreak Analysis of radar data Newly strengthened supercell storms Radar reflectivity - western IL (red = heaviest rain/hail) A B Boulder, CO25 June 2004WRF/MM5 Users Workshop IL tornado outbreak Analysis of radar data Bath, IL Jacksonville, IL T T A B later produce tornadoes Boulder, CO25 June 2004WRF/MM5 Users Workshop IL tornado outbreak Analysis of radar data Full data simulations MM5 Inner 4 model domains Boulder, CO25 June 2004WRF/MM5 Users Workshop Hundreds of WRF simulations were carried out to examine how two storm cells interacted One primary storm at the middle of the modeling domain. A weaker initial cell at each position shown. 232 simulations completed Boulder, CO25 June 2004WRF/MM5 Users Workshop Results from all WRF simulations Vorticity duration field (minutes) 1-km WRF Rapid rotation (wind at the ground) was maintained for over an hour Rapid rotation for less than 25 min. Boulder, CO25 June 2004WRF/MM5 Users Workshop Thunderstorm cell interaction: a large parameter space Boulder, CO25 June 2004WRF/MM5 Users Workshop Thunderstorm cell interaction: a large parameter space Full problem - thousands of runs? Most production systems: 1 run at a time How do we prepare & start all runs? How do we interrogate the output data? How can we manage so many cases? How do we keep track of everything? Boulder, CO25 June 2004WRF/MM5 Users Workshop What Is a Grid Portal? A Grid Portal is a problem solving environment that allows scientists to program, access, and execute distributed Grid applications from a conventional Web Browser & other desktop tools. Grid Portals: A Scientists Access Point for Grid Services - Gannon et al. 2003 Boulder, CO25 June 2004WRF/MM5 Users Workshop What Is a Grid Portal? A Grid Portal is a problem solving environment that allows scientists to program, access, and execute distributed Grid applications from a conventional Web Browser & other desktop tools. The goal is to allow the scientist to focus completely on the science problem at hand by making the Grid a transparent extension of the their desktop computing environment. Grid Portals: A Scientists Access Point for Grid Services - Gannon et al. 2003 Boulder, CO25 June 2004WRF/MM5 Users Workshop Courtesy Dan Weber, University of Oklahoma There has to be a better way (Part 1) Boulder, CO25 June 2004WRF/MM5 Users Workshop MEAD-PI WRF Input File Components Boulder, CO25 June 2004WRF/MM5 Users Workshop MEAD-PI WRF Input Parameters Boulder, CO25 June 2004WRF/MM5 Users Workshop The link between the ADAS and the WRF forecast model Need a capable initialization system that utilizes Doppler radar data for use in predicting hurricanes (not available in WRFSI) Enables other key features to be used within MEAD e.g. ensemble pre- and post- processing and forecast verification ARPS2WRF Data Converter Boulder, CO25 June 2004WRF/MM5 Users Workshop ARPS2WRF Flow Diagram Boulder, CO25 June 2004WRF/MM5 Users Workshop Our WRF storm interaction study is one of several testbeds within the Portal effort. OGRE version 1.0-beta, developed by Al Rossi, Jay Alameda, and Shawn Hampton, has recently been released. It is Java-based, and runs on computer platforms with installed Grid services. This work is now part of the LEAD project - Linked Environments for Atmospheric Discovery (NCSA, Oklahoma, other institutions; lead.ou.edu) There has to be a better way (Part 2) Boulder, CO25 June 2004WRF/MM5 Users Workshop Open Grid Computing Environments Runtime Engine (OGRE) OGRE is a tool for describing and executing the flow of control around scientific computational codes. It also incorporates tasks for secure file transfer using the Java GridFTP client a task for invoking methods on web services using the Apache Axis client, a notification framework for the remote publication / reception of application and "build" (workflow) events Extension of Apache ANT http://ant.apache.org Configurable Runtime Engine Through workflow script Courtesy Jay Alameda, NCSA Boulder, CO25 June 2004WRF/MM5 Users Workshop Starting OGRE (Unix terminal session on laptop) Boulder, CO25 June 2004WRF/MM5 Users Workshop Starting OGRE (Java window opened on laptop) Boulder, CO25 June 2004WRF/MM5 Users Workshop Starting OGRE Boulder, CO25 June 2004WRF/MM5 Users Workshop Selecting a remote compute host Boulder, CO25 June 2004WRF/MM5 Users Workshop Defining the new job Boulder, CO25 June 2004WRF/MM5 Users Workshop Job monitoring Boulder, CO25 June 2004WRF/MM5 Users Workshop Job monitoring Boulder, CO25 June 2004WRF/MM5 Users Workshop Job monitoring Boulder, CO25 June 2004WRF/MM5 Users Workshop Job monitoring Boulder, CO25 June 2004WRF/MM5 Users Workshop Job monitoring Boulder, CO25 June 2004WRF/MM5 Users Workshop Job monitoring Boulder, CO25 June 2004WRF/MM5 Users Workshop Job monitoring Boulder, CO25 June 2004WRF/MM5 Users Workshop Job monitoring Boulder, CO25 June 2004WRF/MM5 Users Workshop Workflow applied to storm modeling Courtesy Shawn Hampton & Al Rossi, NCSA Boulder, CO25 June 2004WRF/MM5 Users Workshop What just happened? Local (laptop) copy of ant starts OGRE OGRE java code on another machine (here, a linux box at NCSA) is copied (GridFTP) to the compute machine (here, a teragrid machine at NCSA) The local interface, using GLOBUS, submits to the job queue on the compute machine at NCSA (could be SDSC or elsewhere) Boulder, CO25 June 2004WRF/MM5 Users Workshop What just happened? A WRF job is started on the compute machine. In this case - ideal.exe is run, followed by WRF. Status info is relayed to the local interface as these steps are completed. When finished, WRF output files are copied (GridFTP) to another machine (here, linux box at NCSA) Local interface reports the job is complete. Boulder, CO25 June 2004WRF/MM5 Users Workshop Details Bad news: Grid services do not work uniformly on all platforms. Grid services arent (yet) heavily used, and system changes at remote sites may break them (bleeding edge) Good news: What you have seen here isnt a toy code or demo package. The underpinnings have been developed/refined first; more on the way. Boulder, CO25 June 2004WRF/MM5 Users Workshop Current Status/Future Work Currently have: Backend that can be programmed to, using hand-coded XML workflow descriptions (like Ant build scripts) Parts of user desktop which both executes and uses the backend engine. Courtesy Jay Alameda, NCSA Boulder, CO25 June 2004WRF/MM5 Users Workshop Current Status/Future Work Currently have: Backend that can be programmed to, using hand-coded XML workflow descriptions (like Ant build scripts) Parts of user desktop which both executes and uses the backend engine. Projects for immediate future: Broker (client and service) Graphical Integrated Development Environment Courtesy Jay Alameda, NCSA Boulder, CO25 June 2004WRF/MM5 Users Workshop Acknowledgements Work derived from Modeling Environments for Atmospheric Discovery (MEAD) (Alliance Expedition, leads Bob Wilhelmson and Sara Graves) Linked Environments for Atmospheric Discovery (LEAD) (ITR, lead Kelvin Droegemier) Alliance Science Portal Expedition (Alliance Expedition, leads Dennis Gannon and Beth Plale) Open Grid Computing Environments (OGCE) Consortium (NMI, lead Marlon Pierce) New: Consortium of Universities for the Advancement of Hydrologic Sciences, Inc (CUAHSI) Greg Daues, Shawn Hampton, Al Rossi Boulder, CO25 June 2004WRF/MM5 Users Workshop Additional Information OGRE:MEAD-PI:OGCE Web site:OGCE Demo Portal:Scientific portals (IU): Portal architecture design document: LEAD: lead.ou.edulead.ou.edu Contact: Boulder, CO25 June 2004WRF/MM5 Users Workshop Boulder, CO25 June 2004WRF/MM5 Users Workshop Additional slides Boulder, CO25 June 2004WRF/MM5 Users Workshop How does this work? Client workstation Grid Compute gatekeeper Gridftp server Grid Data Gridftp server gram gridftp WS Event channel Boulder, CO25 June 2004WRF/MM5 Users Workshop How does this work? Client workstation Grid Compute gatekeeper Gridftp server Grid Data Gridftp server Xml E Xml A Xml D Xml CXml B Launch Delegate Stage Delegate Configure XML through GUI Types of XML: Application, resource, parameter, infrastructure, workflow Event channel gram gridftp WS Boulder, CO25 June 2004WRF/MM5 Users Workshop How does this work? Client workstation Grid Compute gatekeeper Gridftp server Grid Data Gridftp server Xml E Xml A Xml D Xml CXml B Launch Delegate Stage Delegate Stage infrastructure (ant.tar.gz, launch script, build xml) ant.tar.gz, launch script build xml Event channel gram gridftp WS Boulder, CO25 June 2004WRF/MM5 Users Workshop How does this work? Client workstation Grid Compute gatekeeper Gridftp server Grid Data Gridftp server Xml E Xml A Xml D Xml CXml B Launch Delegate Stage Delegate Submit job (use resource/application/host xml) Event channel gram gridftp WS Boulder, CO25 June 2004WRF/MM5 Users Workshop How does this work? Client workstation Grid Compute gatekeeper Gridftp server Grid Data Gridftp server Xml E Xml A Xml D Xml CXml B Launch Delegate Stage Delegate Execute workflow: Ideal WRF Post process/analysis Ide al Param, soundings WRF Post- proces s Event channel gram gridftp WS Boulder, CO25 June 2004WRF/MM5 Users Workshop How does this work? Client workstation Grid Compute gatekeeper Gridftp server Grid Data Gridftp server Xml E Xml A Xml D Xml CXml B Launch Delegate Stage Delegate Periodically store data Ide al Param, soundings WRF Post- proces s Event channel gram gridftp WS Boulder, CO25 June 2004WRF/MM5 Users Workshop Service Architecture Jay Alameda - NCSA Boulder, CO25 June 2004WRF/MM5 Users Workshop User-Oriented Data Management in Scientific Portal: Building on PACI and Teragrid Foundation Jay Alameda, Dennis Gannon, Shawn Hampton Beth Plale, Al Rossi, Bob Wilhelmson Create job description Store and record results Generate and store derived products monitoring myGridContext data management service Broker Service Teragrid data service Teragrid Globus Toolkit 3.0 GridFTP MetaCatalog Service(MCS) NEESgrid Meta- Catalog service Storage Resource Broker (SRB) Gather required data files, resources Execute application description Need x put to l by time y:00 Need to store y and z Provides context to resolve x, y, z Retrieve customized run description template from data management services Connection of monitor to application is control flow issue. Monitoring results is captured in event services persistent channel, so myGridContext only records address of service and channel Capable of delayed invocation of GridFTP not provided elsewhere Teragrid Data Service- single point of contact to server-side metadata servers myGridContext- rich query interface to session metadata including output and derived products Courtesy Jay Alameda Boulder, CO25 June 2004WRF/MM5 Users Workshop April 1996 Tornado Outbreak UIUC team filming tornadoes Most IL tornadoes in a single day (36) Average IL tornadoes in a year: 27 Tornadoes hit Jacksonville, Bloomington, Decatur, Champaign-Urbana, Ogden Boulder, CO25 June 2004WRF/MM5 Users Workshop