Höchstleistungsrechenzentrum Stuttgart SEGL Parameter Study Slide 1 Science Experimental Grid...

17
SEGL Parameter Study Slide 1 Höchstleistungsrechenzentrum Stuttgart Science Experimental Grid Laboratory (SEGL) Dynamical Parameter Study in Distributed Systems Natalia Currle-Linde [email protected] University of Stuttgart High-Performance Computing-Center Stuttgart (HLRS) www.hlrs.de/people/linde

Transcript of Höchstleistungsrechenzentrum Stuttgart SEGL Parameter Study Slide 1 Science Experimental Grid...

Page 1: Höchstleistungsrechenzentrum Stuttgart SEGL Parameter Study Slide 1 Science Experimental Grid Laboratory (SEGL) Dynamical Parameter Study in Distributed.

SEGL Parameter StudySlide 1 Höchstleistungsrechenzentrum Stuttgart

Science Experimental Grid Laboratory (SEGL)

Dynamical Parameter Study in Distributed Systems

Natalia [email protected]

University of Stuttgart

High-Performance Computing-Center Stuttgart (HLRS)

www.hlrs.de/people/linde

Page 2: Höchstleistungsrechenzentrum Stuttgart SEGL Parameter Study Slide 1 Science Experimental Grid Laboratory (SEGL) Dynamical Parameter Study in Distributed.

Natalia Currle-LindeSEGL Parameter StudySlide 2 Höchstleistungsrechenzentrum Stuttgart

Overview

Background & Motivation Related Work Architecture & implementation details of SEGL Usage Example Future Work

Page 3: Höchstleistungsrechenzentrum Stuttgart SEGL Parameter Study Slide 1 Science Experimental Grid Laboratory (SEGL) Dynamical Parameter Study in Distributed.

Natalia Currle-LindeSEGL Parameter StudySlide 3 Höchstleistungsrechenzentrum Stuttgart

Background &Motivation

Parameter studies - a great challenge Parameter studies easy to parallelize

Grid Technology enables integration of resources provides a new technical basis for complex parametric

investigations

Problem: Administration of jobs, parameters, results….

MotivationMotivation: automatically start, execute, monitor applications enable efficientefficient execution of experiments

User doesn`t need to have knowledge of specific programming language knowledge of Grid structure.

Page 4: Höchstleistungsrechenzentrum Stuttgart SEGL Parameter Study Slide 1 Science Experimental Grid Laboratory (SEGL) Dynamical Parameter Study in Distributed.

Natalia Currle-LindeSEGL Parameter StudySlide 4 Höchstleistungsrechenzentrum Stuttgart

Tools for parameter investigation studies

NIMROD (Monash University, Australia)http://www.csse.monash.edu.au/~davida/nimrod

Can be used to manage the execution of parameter studies across distributed computers

ILAB (NASA Ames Research Center )http://www.nas.nasa.gov/ILab/

Allows the generation of multi-parametric models and adds workflow management

do not support dynamic parameterizations

Page 5: Höchstleistungsrechenzentrum Stuttgart SEGL Parameter Study Slide 1 Science Experimental Grid Laboratory (SEGL) Dynamical Parameter Study in Distributed.

Natalia Currle-LindeSEGL Parameter StudySlide 5 Höchstleistungsrechenzentrum Stuttgart

Workflow

Support:• multiphysics applications, preprocessing steps, postprocessing

filters,visualization, iterative search in the parameter space for optimum solutions

Require:• use of Grid

Workflow

TRIANAhttp://www.trianacode.org/index.html

UNICOREhttp://www.unicore.org/

BPEL4WS http://www-128.ibm.com/developerworks/library/specification/ws-bpel

•specification of loops•criteria•synchronisation points•communication via message

Page 6: Höchstleistungsrechenzentrum Stuttgart SEGL Parameter Study Slide 1 Science Experimental Grid Laboratory (SEGL) Dynamical Parameter Study in Distributed.

Natalia Currle-LindeSEGL Parameter StudySlide 6 Höchstleistungsrechenzentrum Stuttgart

Dynamic parameterization

SEGL enables

• dynamical selection of parameter sets on the basis of previous and intermediate results

SEGL supports

• creation of complex processes, which involves

– several levels of parameterization

– repeated processing

– data archiving

– conclusions and branches during the processing

– synchronization of parallel branches and processes

Page 7: Höchstleistungsrechenzentrum Stuttgart SEGL Parameter Study Slide 1 Science Experimental Grid Laboratory (SEGL) Dynamical Parameter Study in Distributed.

Natalia Currle-LindeSEGL Parameter StudySlide 7 Höchstleistungsrechenzentrum Stuttgart

Requirement – Hide complexity from the user

Users are very sensitive to the level of automation of application preparation

They must be able to• define a fine - grained logical execution process• formulate the parameterization rules• identify the position in the input area of the parameters

which are to

be changed in the course of the experiment

All other details should be hidden from the user.

Page 8: Höchstleistungsrechenzentrum Stuttgart SEGL Parameter Study Slide 1 Science Experimental Grid Laboratory (SEGL) Dynamical Parameter Study in Distributed.

Natalia Currle-LindeSEGL Parameter StudySlide 8 Höchstleistungsrechenzentrum Stuttgart

Science Experimental Grid Laboratory

System Architecture

Page 9: Höchstleistungsrechenzentrum Stuttgart SEGL Parameter Study Slide 1 Science Experimental Grid Laboratory (SEGL) Dynamical Parameter Study in Distributed.

Natalia Currle-LindeSEGL Parameter StudySlide 9 Höchstleistungsrechenzentrum Stuttgart

System Architecture

J2EEJBOSS Appication ServerJDO (OODB)UNICORE -Adapter

User Workstation

ExpDesigner

ExpMonitor VIS

ExpMonitor

Supervisor

Exp Engine

Resource MonitorTask

Exp DB

Server

Sub Server

TargetMachine A

Sub Server

Target Machine K

. . . . . . .

RB

Sub Server

File Server

Exp Application Server

Data Job

Job

Job

Data, Parameter

I/O Data

Grid Adapter

Page 10: Höchstleistungsrechenzentrum Stuttgart SEGL Parameter Study Slide 1 Science Experimental Grid Laboratory (SEGL) Dynamical Parameter Study in Distributed.

Natalia Currle-LindeSEGL Parameter StudySlide 10 Höchstleistungsrechenzentrum Stuttgart

Graphic language

Experiment is described at three levelsthree levels:

control flowcontrol flow, , data flowdata flow, , data repositorydata repository

Control flowControl flow: description of logical schema of experiments• direction, condition, sequence of execution

Data flowData flow: local description of interblock computation processes

• standard/user-specific computation module • direction of input/output data between

repositoryand computation module• parameterization rules

Data repositoryData repository: aggregation of data

container application ->application serverQL description ->server data base

Page 11: Höchstleistungsrechenzentrum Stuttgart SEGL Parameter Study Slide 1 Science Experimental Grid Laboratory (SEGL) Dynamical Parameter Study in Distributed.

Natalia Currle-LindeSEGL Parameter StudySlide 11 Höchstleistungsrechenzentrum Stuttgart

User Workstation

ExpDesigner

ExpMonitor VIS

ExpMonitor

Supervisor

Exp Engine

Resource MonitorTask

Exp DB

Server

Sub Server

TargetMachine A

Sub Server

Target Machine K

. . . . . . .

RB

Sub Server

File Server

Exp Application Server

Data Job

Job

Job

Data, Parameter

I/O Data

Grid Adapter

Page 12: Höchstleistungsrechenzentrum Stuttgart SEGL Parameter Study Slide 1 Science Experimental Grid Laboratory (SEGL) Dynamical Parameter Study in Distributed.

Natalia Currle-LindeSEGL Parameter StudySlide 12 Höchstleistungsrechenzentrum Stuttgart

Control Flow

Task

Block 1.1

Solver

Block 1.2

Block 2.1

Block 2.2

Block 3.1

BranchSolver

Solver

Block 1.3

Solver

Condition

Block 2.3

Solver

Block 2.4

Solver

Block 2.5

Solver

Block 3.2

Solver

WaitBlock 4.1

WaitBlock 4.2

Block 5.1

Solver

End

User defines the sequence of execution of experiment blocks

Solver blockSolver blocksimple parameter sweep

Control blockprogram object:

allows changing sequence of execution according to specified criteria

Page 13: Höchstleistungsrechenzentrum Stuttgart SEGL Parameter Study Slide 1 Science Experimental Grid Laboratory (SEGL) Dynamical Parameter Study in Distributed.

Natalia Currle-LindeSEGL Parameter StudySlide 13 Höchstleistungsrechenzentrum Stuttgart

Data Flow

is dynamic

Manipulation of data in a very fine-grained way

Solver Block:computation module Creplacemant module Rparameterization module Pdata base

Each module: Java objecthas standard structure consists of several sections

Computation module : organizes preparation of input datagenerates jobinitializes/controls record of results in DBcontrols execution of module operation

P1

P2

P3

#F0

#F1

P4

P5

P2

P1

module 1.1.1

module 1.1.2

module 1.1.3

R1[p1,15,20] [p2,40,50]

I 1

P3

I 2I 3

O 1

O1

O 1

O 1

I 1 I 2 I 3 I 4

O 1

module 1.1.5

Data Base

C 1

module 1.1.4

Page 14: Höchstleistungsrechenzentrum Stuttgart SEGL Parameter Study Slide 1 Science Experimental Grid Laboratory (SEGL) Dynamical Parameter Study in Distributed.

Natalia Currle-LindeSEGL Parameter StudySlide 14 Höchstleistungsrechenzentrum Stuttgart

P1

P2

P3

#F0

#F1

P4

P5

P2

P1

module 1.1.1

module 1.1.2

module 1.1.3

R1[p1,15,20] [p2,40,50]

I 1

P3

I 2I 3

O 1

O1

O 1

O 1

I 1 I 2 I 3 I 4

O 1

module 1.1.5

Data Base

C 1

module 1.1.4

Data Flow (variants of parameterization)

ExpData Base

TM1 TM2 TM3

Task

Application Server

job

data

DPA

File Server

Page 15: Höchstleistungsrechenzentrum Stuttgart SEGL Parameter Study Slide 1 Science Experimental Grid Laboratory (SEGL) Dynamical Parameter Study in Distributed.

Natalia Currle-LindeSEGL Parameter StudySlide 15 Höchstleistungsrechenzentrum Stuttgart

Control Flow

Task

Block 1.1

Solver

Block 1.2

Block 2.1

Block 2.2

Block 3.1

BranchSolver

Solver

Block 1.3

Solver

Condition

Block 2.3

Solver

Block 2.4

Solver

Block 2.5

Solver

Block 3.2

Solver

WaitBlock 4.1

WaitBlock 4.2

Block 5.1

Solver

End

Page 16: Höchstleistungsrechenzentrum Stuttgart SEGL Parameter Study Slide 1 Science Experimental Grid Laboratory (SEGL) Dynamical Parameter Study in Distributed.

Natalia Currle-LindeSEGL Parameter StudySlide 16 Höchstleistungsrechenzentrum Stuttgart

Power plant simulation

AirOil/Air

separate OFA

Oil/Air

Oil/AirCoalAirCoalOil/AirAir

CoalAirCoal

AirAir

CCOFA

Unit:Output Power 170 MWelFiring System Tangential, Windbox

Bituminous CoalOFA retrofit for NOx-reduction in 1991 Optimized Operation Parameters requiredTarget: Minimizing NOx and Unburned Carbon

Parameters:Damper Setting CCOFADamper Setting sep. OFATilting Angle sep. OFA

Page 17: Höchstleistungsrechenzentrum Stuttgart SEGL Parameter Study Slide 1 Science Experimental Grid Laboratory (SEGL) Dynamical Parameter Study in Distributed.

Natalia Currle-LindeSEGL Parameter StudySlide 17 Höchstleistungsrechenzentrum Stuttgart

Conclusion & Future Work

SEGL• allows end-user programming of complex, computation-

intensive simulation and modeling for science and engineering

• offers efficient way to execute scientific experiments

Future work:

– Globus Adapter

– Investigation of Unicore Resource Broker