Post on 18-Dec-2015
Denis Caromel1
Denis Caromel3
Clouds
Denis Caromel4 www.devoxx.com
FrancaPerrina(Italy)
FlorinAlexandru
Bratu(Romania)
DenisCaromel(France,
Nice)
3 Speakers/Demos
Denis Caromel5
3 Strong Faiths
1.Parallel Computing will make it mainstream with Java
1.A need for a unified Parallel Abstraction:Multi-Core + Distributed
3. Java will make it possible to connect Enterprise Grids and
Clouds
Denis Caromel6
Agenda
Effective SOA + GRIDs with
1. Background: INRIA, OASIS, ActiveEon2. Programming, Optimizing3. Scheduling4. SOA: WS, WSDL, BPEL (Franca Perrina)
5. Enterprise Grids, Clouds: Amazon EC26. Java EE EJB (Florin Alexandru Bratu)
Denis Caromel7
1. Background
Denis Caromel8
OASIS Team & INRIA
Computer Science and Control
• 8 Centers all over France
• Workforce: 3 800
•Strong in standardization committees:– IETF, W3C, ETSI, …
• Strong Industrial Partnerships
• Foster company foundation:
90 startups so far
- Ilog (Nasdaq, Euronext)
- …
- ActiveEon
A joint team between: INRIA, Nice Univ. CNRS
•Now about 40 persons
•2004: First ProActive User Group
•2008: 5th one, Acad./Indus. User Presentations
•ProActive 4.0.1: Distributed and Parallel:
From Multi-cores to Enterprise GRIDs
Denis Caromel9
Startup Company Born of INRIACo-developing, Providing support for Open Source ProActive Parallel Suite
Worldwide Customers (EU, Boston USA, etc.)
Denis Caromel10
2. Programming Optimizing
Parallel Acceleration Toolkit in Java:Parallelism: Multi-Core+Distributed
Used in production by industry
Denis Caromel11
Denis Caromel12
Denis Caromel13
ProActive Parallel Suite
Denis Caromel14
ProActive Parallel Suite
Denis Caromel15 15
Distributed and Parallel
Active Objects
Denis Caromel16 16
A
ProActive : Active objects
Proxy
Java Object
A ag = newActive (“A”, […], VirtualNode)V v1 = ag.foo (param);V v2 = ag.bar (param);...v1.bar(); //Wait-By-Necessity
V
Wait-By-Necessity
is a
Dataflow
Synchronization
JVM
A
JVM
Active Object
Future Object Request
Req. Queue
Thread
v1v2 ag
WBN!
Denis Caromel17 17
Standard system at Runtime:No Sharing
NoC: Network On ChipProofs of Determinism
Denis Caromel18 18
TYPED
ASYNCHRONOUS
GROUPS
Denis Caromel19 19
A
Creating AO and Groups
Typed Group Java or Active Object
A ag = newActiveGroup (“A”, […], VirtualNode)V v = ag.foo(param);...v.bar(); //Wait-by-necessity
V
Group, Type, and Asynchrony
are crucial for Composition
JVM
Denis Caromel20 20
Broadcast and Scatter
JVM
JVM
JVM
JVM
agcg
ag.bar(cg); // broadcast cgProActive.setScatterGroup(cg);ag.bar(cg); // scatter cg
c1 c2c3c1 c2c3
c1 c2c3c1 c2c3
c1 c2c3
c1 c2c3
s
c1 c2c3
s
Broadcast is the default behavior Use a group as parameter, Scattered depends on rankings
Denis Caromel21 21
Optimizing
Denis Caromel22
Denis Caromel23
Denis Caromel24
IC2D
Denis Caromel25
ChartIt
Denis Caromel26
Pies for Analysis and Optimization
Denis Caromel27
Video 1: IC2D OptimizingMonitoring, Debugging, Optimizing
Denis Caromel28 28
3. Scheduling
Denis Caromel29
Denis Caromel30
Scheduler and Resource Manager:User Interface
Denis Caromel31
Scheduler: User Interface
Denis Caromel32
Video 2:Scheduler, Resource Manager
Denis Caromel33
4. SOA Integration:Web Services, BPEL Workflow
Franca PerrinaOASIS Team - INRIA
Denis Caromel34
Active Objects as Web Services
Why ?
Access Active Objects from any language
How ?
HTTP Server
SOAP Engine (Axis)
Usage:
ProActive.exposeAsWebService();
ProActive.unExposeAsWebService();
JVM
Web Service Client
Web Services
Denis Caromel35
ProActive + Services + Workflows
Principles:3 kinds of Parallel Services
3. Domain Specific Parallel Services (e.g. Monte Carlo Pricing)
2. Typical Parallel Computing Services (Parameter Sweeping, D&C, …)
1. Basic Job Scheduling Services (parallel execution on the Grid)
Denis Caromel36
3 kinds of Parallel Services
3. Domain Specific ParallelServices: providing business functionalities executed inparallel
2. Parallelization services:typical parallel computingpatterns (ParameterSweeping, D&C, …)
1. Job Scheduling service: Schedule and Run jobs in parallel on the Grid.
Parallel ServicesParallel Services Operational ServicesOperational Services…
1. Job Scheduling
Service
2. ParameterSweeping
Service
2. Divide & Conquer Service
OtherOperational
Service
…
Domain Specific Parallel Service
…Other
Basic Service
High level Business Process
GridGrid
3. Domain Specific Parallel
Service
Denis Caromel37
A sample pattern: Parameter Sweeping
I1I2
…In
O1O2
…On
ExecLogic
parameter sweeping
I1I2…In
O1O2…On
Parameter SweepingService, customized with an Exec logic X
ParamSweeping
Service
Process using parameter sweeping service
…
All the running instances of the Exec logic X are executed on the grid as a whole
PA Scheduler &PA Scheduler &PA Resource ManagerPA Resource Manager
Denis Caromel38
Demo
SOA Integration:Web Services, BPEL Workflow
Denis Caromel39 39
5. Enterprise Grids, Clouds:
Standards &Amazon EC2
Denis Caromel40
Deploy on Various Kinds of Infrastructures
Internet
Clusters
ParallelMachine
LargeEquipment
Internet
Job management forembarrassingly parallel application (e.g. SETI)
Internet
Internet
Servlets EJBs Databases
Denis Caromel41
GCM Fractal StandardizationFractal Based Grid Component Model
Overall, the standardization is supported by industrials: BT, FT-Orange, Nokia-Siemens, Telefonica,
NEC, Alcatel-Lucent, Huawei …
Denis Caromel42
Protocols and Scheduler inGCM Deployment Standard
Protocols:Rsh, sshOarsh, Gsissh
Scheduler, and Grids:GroupSSH, GroupRSH, GroupOARSHARC (NorduGrid), CGSP China Grid, EEGE gLITE, Fura/InnerGrid (GridSystem Inc.)GLOBUS, GridBusIBM Load Leveler, LSF, Microsoft CCS (Windows HPC Server 2008)Sun Grid Engine, OAR, PBS / Torque, PRUN
Soon available in stable release:Java EE
Amazon EC2
Denis Caromel43
6. J2EE IntegrationFlorin Alexandru Bratu
OASIS Team - INRIA
Denis Caromel44
J2EE Integration with Parallelism + Grids/Clouds
Performing Grid & Cloud Computing
From & In
an Application Servers
1. Delegating heavy computations outsides J2EE Applications
2. Using Deployed J2EE Nodes as Computational Resources
Denis Caromel45
ProActive – J2EE Integration (1)
1. Delegating heavy computations outsides J2EE Applications
Denis Caromel46
ProActive – J2EE Integration (2)
2. Using Deployed J2EE Nodes as Computational Resources
Objective Being able to deploy active objects inside the JVMs of application servers
Implementation Based on a Sun standard – Java Connector Architecture JSR112 Deployment module: resource adapter (RAR) Works with all J2EE-compliant Application Servers
Integration(2)
Denis Caromel48
Grids & Clouds:
Amazon EC2 Deployment
Denis Caromel49
Big Picture + Clouds
Denis Caromel50
Clouds: ProActive Amazon EC2 Deployment
Principles & Achievements:
ProActive Amazon Images (AMI) on EC2
So far up to 128 EC2 Instances
(Indeed the maximum on the EC2 platform, … ready to try 4 000 AMI)
Seamless Deployment:
no application change, no scripting, no pain
Open the road to : In house Enterprise Cluster and Grid + Scale out on EC2
Denis Caromel51
ProActive Deployment onAmazon EC2
Video
Denis Caromel52
On Going
AGOSGrid Architecture for SOA
Denis Caromel53
AGOSGrid Architecture for SOA
AGOS Solutions
Building a Platform for Agile SOA with Grid
In Open Source with Professional Support
Denis Caromel54
AGOS Generic Architecture for SOA with GRIDs
OS, HW
OS Virtualization Grid Utility interface
ESBEnterprise Service Bus
SCA Service Component Architecture
Resource Manager
Task & Services Scheduling
Parallel ProgrammingSPMD, workflow
Agent, Master/WorkerFork and Join
In memory db cache(JSR / JPI / javaspaces) SOA BPEL Exec
Repository, Registry, Orchestration
SOA MonitoringReporting, Notifications,
alarms
Business Intelligence BI Monitoring
SLM SLM
SLM
SLM
SLMSLM
SLMSLM
SLM SLM
Denis Caromel55
Summary
Denis Caromel56
Conclusion: Effective SOA + GRIDs in Java with
An Acceleration Toolkit :Concurrency+ParallelismMulti-Core+Distributed
Grid & SOA:J2EE, WS,BEPL, EG +Amazon EC2
Q&A