OGCE Workflow and LEAD Overview
21
OGCE Workflow and LEAD Overview Suresh Marru, Marlon Pierce September 2009
description
OGCE Workflow and LEAD Overview. Suresh Marru, Marlon Pierce. September 2009. The LEAD Gateway. LEAD Pioneering Technology. LEAD Dynamic Adaptive Infrastructure . Storms Forming. Forecast Model. Streaming Observations. Data Mining. Instrument Steering. Refine forecast grid. - PowerPoint PPT Presentation
Transcript of OGCE Workflow and LEAD Overview
OGCE Workflow and LEAD Overview
Suresh Marru, Marlon Pierce
September 2009
Suresh Marru
The LEAD Gateway
Suresh Marru
LEAD Pioneering Technology
Suresh Marru
StreamingObservations
Storms Forming
Forecast Model
Data Mining
Refine forecast grid
Instrument Steering
LEAD Dynamic Adaptive Infrastructure
Suresh Marru
LEAD Scientists and Educational Interactions
DevelopersResearcherss
End UsersLowering the barrier for using complex end-to-end weather
technologies
Democratize
Empower
Facilitate
Suresh Marru
Suresh Marru
Generalize, Harden, Build
Test
Open Grid Computing Environments
Suresh Marru
Middleware Services
Compute & Data Resources
User Interactions
Computational Cloud
GFac Services
Local Lab Resources
Other ClientsXBaya GUI
XBaya Core
Web Portal
Computational Grids
Event Bus
Workflow Engine (ODE) XRegistry
XMCCat Metadata Catalog
Flexible Layered Service Oriented Architecture
Suresh Marru
Application Wrapper Framework• Scientific Applications are wrapped
into web services by filling in a web-form.
• The Application Factory generates a web service for each application with I/O interfaces.
• Registers WSDL for the service with a registry
• Each service generates a stream of notifications that log the service actions back to the XMCCat Metadata Catalog, user monitoring, and provenance tracking tools
cc
Application Factory
Run programRun program& publish events& publish events
App Service
Suresh Marru
Application Wrapper Architecture
Suresh Marru
Service Monitoring via Events• The service output is a stream of
events• I am running your request• I have started to move your input files• I have all the files• I am running your application• The application is finished• I am moving the output to you file
space• I am done
• These are automatically generated by the service using a distributed event system(WS-Eventing / WS-Notification)• Topic based pub-sub system with
a well known “channel”
ApplicationServiceInstance
12
34
56
NotificationChannel
listener publisher
xx
Suresh Marru
Workflow Suite Architecture
Suresh Marru
Composition and
Monitoring Abstract DAG Model
BPEL 1.1
BPEL 2.0
SCUFL
Python
Dynamic Enactor/Interpreter
Jython Based Enactor
GPEL Engine
Apache ODE Engine
Taverna
Python Runtime
Message Bus
Interoperable Xbaya Architecture
Suresh Marru1414
OVP/RST/ MIG
Fault Tolerance/Recovery Service
Application Performance
Models
Resource Reliability
Models
NWS, MDS
BQP
Deadline &Success
Probability
PortalBPEL
Workflow Engine
Application Service
Notification Service
Simple Recovery Architecture
Suresh Marru
Workflow Composer
Suresh Marru
Example: Assimilation Workflow
Suresh Marru
• Domain webs of large genomes– Input list of amino acid sequences– Identify all known domains– Construct webs
Parallel processingCapability-type computing
Ensemble-type processing(minimal network reqs)
Capacity-type computing
Science Independence Example
Suresh Marru
Biology Gateway
Suresh Marru
Polar Grid Example
Suresh Marru
LEADLower entry barrier to using weather analysis toolsImprove detection, analysis & prediction of mesoscale weather
Motif-NetworkTransformation of sequenced genomes to “domain-space”
Cyber-Infrastructure EvaluationPerformance evaluation of future supercomputer architectures
ADAMAlgorithms for feature extraction, data normalization, classification
and normalization
GridChemMolecular Chemistry Grid helping researchers run chemistry
applications on Grid Environment
Example Gateways using OGCE Workflow Suite
Suresh Marru
Live Demo & Questions?
