Bioinformatics Ontology for Automatic Workflow Generation on Web/Grid Services
A Concept of a Monitoring Infrastructure for Workflow-Based Grid Applications
description
Transcript of A Concept of a Monitoring Infrastructure for Workflow-Based Grid Applications
Cracow Grid Workshop 2003
Institute of Computer
Science AGH
A Concept of a Monitoring Infrastructure for Workflow-Based Grid Applications
Bartosz Baliś, Marian Bubak, Włodzimierz Funika, Tomasz
Szepieniec, Marcin Radecki, Roland Wismüller, Tomasz Arodź, Marcin
Kurdziel
Cracow Grid Workshop 2003
Institute of Computer
Science AGH
Plan
Workflow-based applications Application Monitoring Architecture of the monitoring
infrastructure Performance analysis Monitoring of multilingual
applications Summary
Cracow Grid Workshop 2003
Institute of Computer
Science AGH
Workflow-based Grid Applications
Applications composed of a workflow of components
Components are independent Grid services
Cracow Grid Workshop 2003
Institute of Computer
Science AGH
Application Monitoring
Goal: provide monitoring services to • Obtain debug / performance-related
information• Possibly enable manipulations on the
target application• Detect events and program actions to be
executed when they occur
Consumers• Tools for performance analysis / debugging• Other tools and systems, e.g., for fault-
tolerance, load balancing, etc.
Cracow Grid Workshop 2003
Institute of Computer
Science AGH
Monitoring infrastructure for Wf-based apps
Additional monitoring interface for each grid service (component) to be monitored• Obtain monitoring info for that component
Global monitoring service• Separate grid service• One per application• Obtain collective monitoring info
Cracow Grid Workshop 2003
Institute of Computer
Science AGH
Monitoring Wf-based Applications
Client
Monitoring service
Additional monitoring interface (monitoring info related to the component)
Global monitoring (grid) service. Collective
monitoring information.
Cracow Grid Workshop 2003
Institute of Computer
Science AGH
Startup
User submits a workflow application and requests monitoring, possibly subsequently
Monitoring service is created• It must discover the workflow components• Workflow subsystem must provide a
mechanism for this• Workflow registry?
Cracow Grid Workshop 2003
Institute of Computer
Science AGH
Component monitoring interface – how?
Monitoring functionality inherent part of each component
How to provide the additional monitoring interface?• Take monitoring interface into account at
the design stage• Component developers involved?
This approach (hopefully) enables instrumentation of grid services
Cracow Grid Workshop 2003
Institute of Computer
Science AGH
Performance analysis service
Combined with the monitoring service
Supports performance analysis of•Grid infrastructure•Application
•Intra-component•Inter-component
A B C D E F G HSEL ECT ED
O N-LINE
Infrastructuremonitoring
Applicationmonitoring
Inter-component
Inter-
component
Inter-
component
Cracow Grid Workshop 2003
Institute of Computer
Science AGH
Infrastructure and intra-component analysis
Grid infrastructure • Uses existing tools to monitor the status of
grid environment of the components
Intra-component operations• Measure some quantities describing the
status and performance of a service used in the application workflow
• Depend on the monitoring interface defined by the component developers
• Reuse existing approaches to monitoring and analysis
Cracow Grid Workshop 2003
Institute of Computer
Science AGH
Inter-component performance monitoring
Adds a new level to the performance analysis of grid applications
Accounts for the concept of application as a workflow of services
Monitors the state and performance of the cooperation of the services to capture the state and performance of the whole application
Two modes:• Structural, semantic monitoring mode • Activity monitoring mode
Cracow Grid Workshop 2003
Institute of Computer
Science AGH
Structural, semantic monitoring Captures information on workflow components:
• Component status• Component usage• Custom component properties
Uses the intra-component monitoring data on:• Functions• Code regions• Synchronisation objects
Combines the information on various components to describe the status of the whole workflow based-application
Proposed techniques:• Source/byte code instrumentation• Dynamic instrumentation
Cracow Grid Workshop 2003
Institute of Computer
Science AGH
Activity monitoring
Captures information on the behaviour of the workflow
Analysed activity properties:• Caller/callee relationships• End-to-end response time for activity invocation• Success rate of invocation• Synchronisation time in activity• Volume of data exchanged in activity
Proposed techniques:• Distributed instrumentation of workflow components• Insertion of probes
Cracow Grid Workshop 2003
Institute of Computer
Science AGH
Event Service
Automatically sends notifications to other grid services when some performance conditions are met.
Allows for a reaction on undesired changes of grid environment status as well as in the activity of the application.
Will be integrated with performance data provider in a uniform framework.
This integration will enable the notified service to inspect a wide range of performance information before deciding on the actions that should be taken.
Cracow Grid Workshop 2003
Institute of Computer
Science AGH
Two levels of performance information A proven concept form the G-PM tool. Used in both application performance data
provider and event service. Lower level performance information:
• a consistent, well-defined set of metrics,• used, by system administrators, application
developers, users and other parties. Higher level performance information:
• more abstract, numerical, quantitative metrics,• defined on the basis of the lower-level metrics
with a special definition language,• allowing for creation of various metrics suitable for
a particular purpose.
Cracow Grid Workshop 2003
Institute of Computer
Science AGH
Monitoring of multi-lingual applications Services can be implemented using multi-lingual
approach e.g. as a computational kernels written in Fortran and wrapped by Java code.
A higher level abstraction of metrics must be worked out which could handle different programming approaches and languages.
The low-level monitoring system must be designed in a way that enables a simultaneous monitoring of Java, C, C++ and other language-based service parts using an uniform approach:• the specification of monitoring services must be the same
for all languages,• the implementations of the monitoring services may vary.
Cracow Grid Workshop 2003
Institute of Computer
Science AGH
Summary
Workflow-based approach to applications still immature
Architecture for a monitoring infrastructure proposed• Integration of monitoring services into
components• Global grid monitoring service
Performance analysis• Resuse of existing approaches for
infrastructure and intra-component analysis
• New level of inter-component analysis