Runtime Fault-Handling for Job-Flow Management In Grid Environments
Integration of Multidiscipline Applications in Grid-computing Environments
description
Transcript of Integration of Multidiscipline Applications in Grid-computing Environments
![Page 1: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/1.jpg)
Integration of Multidiscipline Applications Integration of Multidiscipline Applications in Grid-computing Environmentsin Grid-computing Environments
NGUYEN G.T., J. BLACHON, C. PLUMEJEAUD
PARA’02, Espoo, June 16th, 2002
« OPALE » PROJECT
Conference on Applied Parallel Computing
![Page 2: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/2.jpg)
• Topics
• Located Sophia-Antipolis & Grenoble
• Follow up SINUS project
• New INRIA project since January 1st, 2002
OPALEOPALE
NUMERIC OPTIMISATION (genetic, hybrid, …)
MODEL REDUCTION (hierarchic, multi-grids, …)
INTEGRATION PLATFORMS
Coupling, distribution, parallelism, grids, clusters, ...
APPLICATIONS : aerospace, electromagnetics, …
![Page 3: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/3.jpg)
• APPLICATION DEVELOPER ’S POINT OF VIEW
• SOFTWARE ENGINEERING POINT OF VIEW
• USER POINT OF VIEW
• CONCLUSIONS
OVERVIEWOVERVIEW
• THEORETICAL POINT OF VIEW
![Page 4: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/4.jpg)
• DEVELOPER ’S POINT OF VIEW
• SOFTWARE ENGINEERING POINT OF VIEW
• USER POINT OF VIEW
• CONCLUSIONS
OVERVIEWOVERVIEW
• THEORETICAL POINT OF VIEW
![Page 5: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/5.jpg)
WHERE WE ARE TODAYWHERE WE ARE TODAY
• 1980 : one year CPU time
• 1992 : one month « »
• 1997 : four days « »
• 2002 : one hour « »
• ASCI White (LLNL) : 8192 IBM SP procs
• ASCI Red (Sandia) : 9632 Intel procs
• ASCI Blue Mountain (LANL) : 6144 SGI procs
Bits and pieces….
![Page 6: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/6.jpg)
TEST CASETEST CASEWING PROFILE OPTIMISATION
![Page 7: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/7.jpg)
TEST CASETEST CASE
• SHOCK-WAVE INDUCED DRAG REDUCTION
• WING PROFILE OPTIMISATION (RAE2822)
• Euler eqns (0,84 Mach, i = 2°) + BCGA (100 gen.)
• 2D MESH : 14747 nodes, 29054 triangles
• 4.5 hours CPU time (SUN Micro SPARC 5, Solaris 2.5)
• 2.5 minutes CPU time (PC cluster 40 bi-procs PIII, Linux)
![Page 8: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/8.jpg)
““CAST” INTEGRATION PLATFORMCAST” INTEGRATION PLATFORM
GOALS
• TEST CASES IMPLEMENTATION
• “DECISION” CORBA INTEGRATION PLATFORM
• DESIGN FUTURE HPCN OPTIMISATION PLATFORMS
COLLABORATIVE MULTI-DISCIPLINE OPTIMISATION
GENETIC & PARALLEL OPTIMISATION ALGORITHMS
CODE COUPLING FOR CFD, CSM SOLVERS & OPTIMISERS
COLLABORATIVE APPLICATIONS SPECIFICATION TOOL
![Page 9: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/9.jpg)
The front stage….
![Page 10: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/10.jpg)
CAST DISTRIBUTED INTEGRATION PLATFORMCAST DISTRIBUTED INTEGRATION PLATFORMUser interface
![Page 11: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/11.jpg)
• DEVELOPER ’S POINT OF VIEW
• SOFTWARE ENGINEERING POINT OF VIEW
• USER POINT OF VIEW
• CONCLUSIONS
OVERVIEWOVERVIEW
• THEORETICAL POINT OF VIEW
![Page 12: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/12.jpg)
• DISTRIBUTED : LAN, WAN, HSN...
• CODE-COUPLING FOR HETEROGENEOUS SOFTWARE
• COLLABORATIVE APPLICATIONS
• COMMON DEFINITION, CONFIGURATION, DEPLOYMENT, EXECUTION & MONITORING ENVIRONMENTS
• TARGET HARDWARE : NOW, COW, PC-clusters, grids, ...
• TARGET APPLICATIONS : multidiscipline engineering, ...
INTEGRATIONINTEGRATION PLATFORMSPLATFORMS
Distributed tasks interacting dynamically in controlled and formally provable way
What they are...
![Page 13: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/13.jpg)
DISTRIBUTED SIMULATION
• MULTI-DISCIPLINE PROBLEM SOLVING ENVIRONMENTS
• HIGH-PERFORMANCE & TRANSPARENT DISTRIBUTION
• USING CURRENT COMMUNICATION STANDARDS
• USING CURRENT PROGRAMMING STANDARDS
• WEB LEVEL USER INTERFACES
• OPTIMIZED LOAD BALANCING & COMMUNICATION FLOW
What is required...
![Page 14: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/14.jpg)
DESIGN ALTERNATIVESDESIGN ALTERNATIVES
• HARWARE & SOFTWARE ENVIRONMENTS
• EXISTING PLATFORMS
• LEGACY APPLICATION SOFTWARE
• PROBLEM REQUIREMENTS
Optimize specific pbs & solutions : ReMAP
System evolution & development : PARIS
Globus, Condor, NetSOLVE, Legion, ….
How to integrate them into new PSE ?
![Page 15: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/15.jpg)
INRIA PROJECTS ALTERNATIVESINRIA PROJECTS ALTERNATIVES
• HARWARE & SOFTWARE ENVIRONMENTS
• OTHER EXISTING PLATFORMS
• LEGACY & NEW APPLICATION SOFTWARE
• PROBLEM REQUIREMENTS
Optimize specific pbs : ReMAP, ATHAPASCAN
System development : PARIS, OASIS
Globus, Condor, NetSOLVE, Legion, ….
How to integrate them into new PSE ?
![Page 16: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/16.jpg)
• VISUAL PROGRAMMING
• COMPONENTS PROGRAMMING
• OBJECT-ORIENTED TECHNOLOGY
ADVANCES IN SOFTWAREADVANCES IN SOFTWARE
PROGRAMMING : C++, JAVA, C#, ...
APPLICATION MODELING : UML
REUSABILITY
MODULARITYINTEROPERABILITY
![Page 17: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/17.jpg)
DISTRIBUTED OBJECT ARCHITECTUREDISTRIBUTED OBJECT ARCHITECTURE
• TRANSPARENT DISTRIBUTED OBJECT COMPUTING
• CORBA COMPLIANT
• SIMPLE SOFTWARE MODEL
• COMPONENTS PLUG-IN (e.g., optimizers, solvers)
- COMPONENTS
- CONNECTORS
![Page 18: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/18.jpg)
DISTRIBUTED OBJECTS ARCHITECTUREDISTRIBUTED OBJECTS ARCHITECTURE
SOFTWARE COMPONENTS
• COMPONENTS ARE DISTRIBUTED OBJECTS
• WRAPPERS AUTOMATICALLY (?) GENERATED
• COMPONENTS ENCAPSULATE CODES
• DISTRIBUTED PLUG & PLAY
![Page 19: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/19.jpg)
CAST PROTOTYPECAST PROTOTYPE
CAST OPTIMIZERS
CORBA
SOLVERS
Server Wrapper Wrapper
Modules Modules
![Page 20: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/20.jpg)
SOFTWARE COMPONENTSSOFTWARE COMPONENTS
• BUSINESS COMPONENTS
LEGACY SOFTWARE
• OBJECT-ORIENTED COMPONENTS
• DISTRIBUTED OBJECTS COMPONENTS
• METACOMPUTING COMPONENTS ?
C++, PACKAGES, ...
Java RMI, EJB, CCM, ...
![Page 21: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/21.jpg)
DISTRIBUTED OBJECT ARCHITECTUREDISTRIBUTED OBJECT ARCHITECTURESOFTWARE CONNECTORS
• CONNECTORS ARE SYNCHRONISATION CHANNELS
• SEVERAL PROTOCOLS
• CONNECTORS = DATA COMMUNICATION CHANNELS
- SYNCHRONOUS METHOD INVOCATION
- ASYNCHRONOUS EVENT BROADCAST
• COMPONENTS COMMUNICATE THROUGH SOFTWARE CONNECTORS
![Page 22: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/22.jpg)
• APPLICATION DEVELOPER POINT OF VIEW
• SOFTWARE ENGINEERING POINT OF VIEW
• USER POINT OF VIEW
• CONCLUSIONS
OVERVIEWOVERVIEW
• THEORETICAL POINT OF VIEW
![Page 23: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/23.jpg)
• SUPPORT FOR NEW APPROACHES
• // SOFTWARE LIBRARIES : MPI, PVM, SciLab //, ...
• PARALLEL and/or DISTRIBUTED HARDWARE
• SUPPORT SEVERAL DEGREES PARALLELISM
PARALLEL APPLICATIONSPARALLEL APPLICATIONS
DOMAIN DECOMPOSITION
GENETIC ALGORITHMS
GAME THEORY
HIERARCHIC MULTI-GRIDS
The good news….
![Page 24: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/24.jpg)
• Lays the ground for GRID and METACOMPUTING
• PC & Multiprocs CLUSTERS : thousands GHz procs...
• HIGH-SPEED NETWORKS : ATM, FIBER OPTICS...
ADVANCES IN HARDWAREADVANCES IN HARDWARE
GLOBUS, LEGION
CONDOR, NETSOLVE
Gigabits/sec networks available (2.5, 10, …)
The best news….
![Page 25: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/25.jpg)
CLUSTER COMPUTINGCLUSTER COMPUTINGPC-cluster at INRIA Rhône-Alpes (216 Linux Pentium III procs.)
![Page 26: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/26.jpg)
CLUSTER COMPUTINGCLUSTER COMPUTINGPC-cluster at INRIA Rhône-Alpes (216 Pentium III procs.)
![Page 27: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/27.jpg)
-300
-200
-100
0
100
200
300
-200-1000100200300400500600
Volet sortiBec sorti
Corps principal
AIRFOIL OPTIMISATIONAIRFOIL OPTIMISATION
![Page 28: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/28.jpg)
AIRFOIL OPTIMISATIONAIRFOIL OPTIMISATION
![Page 29: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/29.jpg)
The results...
![Page 30: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/30.jpg)
CLUSTER COMPUTINGCLUSTER COMPUTINGPC-cluster at INRIA Rhône-Alpes
Multi-airfoil optimization : game theory + multi-grids hierarchic algo.
![Page 31: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/31.jpg)
CAST DISTRIBUTED INTEGRATION PLATFORMCAST DISTRIBUTED INTEGRATION PLATFORM
NICE
RENNES
GRENOBLE
PC cluster
PC clustern CFD solvers
CAST
GA optimiser
PC clustersoftware
VTHD Gbits/s network
GRID computing...
July 2001...
![Page 32: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/32.jpg)
Check for syntaxe of request
NSD
CORBA
Event channell,i1, i2, i3, ….
IRD
Algogen.idl
AlgoGeni1,i2, i3, …, in
CAST
CfdSolvercfd1
CfdSolver cfd2
CAST DISTRIBUTED INTEGRATION PLATFORMCAST DISTRIBUTED INTEGRATION PLATFORMBehind the stage, again...
![Page 33: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/33.jpg)
Event channel,i1, i2, i3, …, in
CfdSolverCfd1
ProcessorP0
ProcessorP1
ProcessorP3
ProcessorP2
i1
CfdSolverCfd2
ProcessorP0
ProcessorP1
ProcessorP3
ProcessorP2
i2
CfdSolverCfd3
ProcessorP0
ProcessorP1
ProcessorP3
ProcessorP2
i3
Genetic Algorithm
i1, i2 ,i3, …, in
Parallelized with MPI on p processors
Genetic algorithm based on selection, mutation, crossover
CORBA server implemented in C++
CORBA client implemented in C++
THREE LEVELS of PARALLELISMTHREE LEVELS of PARALLELISM
![Page 34: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/34.jpg)
Ag2DWithCorba
0100200300400500600700800
1 2 3 4 5 6 7
Nb CfdSolvers
Tim
e (
s) SOPHIA
RENNES
GRENOBLE
CfdSolvers at Sophia, CAST at Grenoble
0 200 400 600 800 1000
Ag atGrenoble
Ag atSophia
Ag atRennes
Time (s)
6 Cfd
5 Cfd
4 Cfd
3 Cfd
2 Cfd
1 Cfd
* Curves quasi-parallels
=> same speed up, whatever the place.
* Join an horizontal asymptote:
time = 200 s
CAST DISTRIBUTED INTEGRATION PLATFORMCAST DISTRIBUTED INTEGRATION PLATFORM
The game : load balancing,...
![Page 35: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/35.jpg)
• DEVELOPER ’S POINT OF VIEW
• SOFTWARE ENGINEERING POINT OF VIEW
• USER POINT OF VIEW
• CONCLUSIONS
OVERVIEWOVERVIEW
• THEORETICAL POINT OF VIEW
![Page 36: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/36.jpg)
BCGA FUN
END
InitB HYBRID
PROCESS FORMULAEPROCESS FORMULAEMILNER ’S SCCS PROCESS ALGEBRA
InitBCGA:InitHybrid:BGGA:(TRUE:(END)+FALSE:(FUN:(TRUE:(HYBRID:
(TRUE: (=>InitHybrid)+FALSE:(=>FUN)))+FALSE:(=>BGGA))))
InitH
![Page 37: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/37.jpg)
OPERATORSOPERATORS
• SYNCHRONIZATION
• PARALLEL EXECUTION
• SERIAL EXECUTION
• ITERATIONS
• COMPLEX EXPRESSIONS : process formulae
• CHOICE
IC simulation : several coupled models
![Page 38: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/38.jpg)
STRONG POINTSSTRONG POINTS
• STRONG THEORETICAL FOUNDATIONS
• SPECIFICATION & VERIFICATION OF COMPLEX APPS
Process algebra for asynchronous systems
• FORMAL SPECIFICATION SYSTEM
• EASY TO USE
Intuitive interface : simple component modelNo theoretical background knowledge requiredTransparent distribution using CORBA
Milner ’s SCCS algebra
![Page 39: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/39.jpg)
• DEVELOPER ’S POINT OF VIEW
• SOFTWARE ENGINEERING POINT OF VIEW
• USER POINT OF VIEW
• CONCLUSIONS
OVERVIEWOVERVIEW
• THEORETICAL POINT OF VIEW
![Page 40: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/40.jpg)
• GRID COMPUTING
• DISTRIBUTED INTEGRATION PLATFORMS
• MULTIDISCIPLINE SIMULATION
TODAY ’S FUTURETODAY ’S FUTURE
e.g., DIGITAL DYNAMIC AIRCRAFT
CAST, JACO3, CCAT, ProACTIVE ...
GLOBUS, LEGION, CONDOR, ...
![Page 41: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/41.jpg)
• DYNAMIC LOAD BALANCING & RESSOURCE ALLOC
• « COTS » PROGRAMMING
• METACOMPUTING
TOMORROW’S FUTURETOMORROW’S FUTURE
COMPONENTS OFF THE SHELF
POWER SUPPLY PARADIGM APPLIED TOCOMPUTING RESOURCES WORLDWIDE
Behind the stage, again...
OBSERVE, START, SUSPEND, RESUME, STOP, MIGRATE
REMOTE PROCESSES DYNAMICALLY
![Page 42: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/42.jpg)
CONCLUSIONCONCLUSION
• INTEGRATION PLATFORMS PROVIDE
• GRID COMPUTING
• FULLY CORBA COMPLIANT
• ALSO ALLOWS CORBA & non-CORBA COMPONENTS
• SMOOTH TRANSITION FROM EXISTING CODE-COUPLING ENVIRONMENTS
DEFINE, CONFIGURE, DEPLOY, EXECUTE & MONITORCOLLABORATIVE APPLICATIONS
• ALLOWS SEQUENTIAL & PARALLEL COMPONENTS
![Page 43: Integration of Multidiscipline Applications in Grid-computing Environments](https://reader035.fdocuments.in/reader035/viewer/2022062322/568150f0550346895dbf0976/html5/thumbnails/43.jpg)
DOCUMENTATIONDOCUMENTATION
• http://www.inrialpes.fr/opale
• http://cast.sourceforge.net/manuel