Denis Caromel1

Denis Caromel2 www.devoxx.com

Denis Caromel3

Clouds

Denis Caromel4 www.devoxx.com

FrancaPerrina(Italy)

FlorinAlexandru

Bratu(Romania)

DenisCaromel(France,

Nice)

3 Speakers/Demos

Denis Caromel5

3 Strong Faiths

1.Parallel Computing will make it mainstream with Java

1.A need for a unified Parallel Abstraction:Multi-Core + Distributed

3. Java will make it possible to connect Enterprise Grids and

Clouds

Denis Caromel6

Agenda

Effective SOA + GRIDs with

1. Background: INRIA, OASIS, ActiveEon2. Programming, Optimizing3. Scheduling4. SOA: WS, WSDL, BPEL (Franca Perrina)

5. Enterprise Grids, Clouds: Amazon EC26. Java EE EJB (Florin Alexandru Bratu)

Denis Caromel7

1. Background

Denis Caromel8

OASIS Team & INRIA

Computer Science and Control

• 8 Centers all over France

• Workforce: 3 800

•Strong in standardization committees:– IETF, W3C, ETSI, …

• Strong Industrial Partnerships

• Foster company foundation:

90 startups so far

- Ilog (Nasdaq, Euronext)

- …

- ActiveEon

A joint team between: INRIA, Nice Univ. CNRS

•Now about 40 persons

•2004: First ProActive User Group

•2008: 5th one, Acad./Indus. User Presentations

•ProActive 4.0.1: Distributed and Parallel:

From Multi-cores to Enterprise GRIDs

Denis Caromel9

Startup Company Born of INRIACo-developing, Providing support for Open Source ProActive Parallel Suite

Worldwide Customers (EU, Boston USA, etc.)

Denis Caromel10

2. Programming Optimizing

Parallel Acceleration Toolkit in Java:Parallelism: Multi-Core+Distributed

Used in production by industry

Denis Caromel11

Denis Caromel12

Denis Caromel13

ProActive Parallel Suite

Denis Caromel14

ProActive Parallel Suite

Denis Caromel15 15

Distributed and Parallel

Active Objects

Denis Caromel16 16

A

ProActive : Active objects

Proxy

Java Object

A ag = newActive (“A”, […], VirtualNode)V v1 = ag.foo (param);V v2 = ag.bar (param);...v1.bar(); //Wait-By-Necessity

V

Wait-By-Necessity

is a

Dataflow

Synchronization

JVM

A

JVM

Active Object

Future Object Request

Req. Queue

Thread

v1v2 ag

WBN!

Denis Caromel17 17

Standard system at Runtime:No Sharing

NoC: Network On ChipProofs of Determinism

Denis Caromel18 18

TYPED

ASYNCHRONOUS

GROUPS

Denis Caromel19 19

A

Creating AO and Groups

Typed Group Java or Active Object

A ag = newActiveGroup (“A”, […], VirtualNode)V v = ag.foo(param);...v.bar(); //Wait-by-necessity

V

Group, Type, and Asynchrony

are crucial for Composition

JVM

Denis Caromel20 20

Broadcast and Scatter

JVM

JVM

JVM

JVM

agcg

ag.bar(cg); // broadcast cgProActive.setScatterGroup(cg);ag.bar(cg); // scatter cg

c1 c2c3c1 c2c3

c1 c2c3c1 c2c3

c1 c2c3

c1 c2c3

s

c1 c2c3

s

Broadcast is the default behavior Use a group as parameter, Scattered depends on rankings

Denis Caromel21 21

Optimizing

Denis Caromel22

Denis Caromel23

Denis Caromel24

IC2D

Denis Caromel25

ChartIt

Denis Caromel26

Pies for Analysis and Optimization

Denis Caromel27

Video 1: IC2D OptimizingMonitoring, Debugging, Optimizing

Denis Caromel28 28

3. Scheduling

Denis Caromel29

Denis Caromel30

Scheduler and Resource Manager:User Interface

Denis Caromel31

Scheduler: User Interface

Denis Caromel32

Video 2:Scheduler, Resource Manager

Denis Caromel33

4. SOA Integration:Web Services, BPEL Workflow

Franca PerrinaOASIS Team - INRIA

Denis Caromel34

Active Objects as Web Services

Why ?

Access Active Objects from any language

How ?

HTTP Server

SOAP Engine (Axis)

Usage:

ProActive.exposeAsWebService();

ProActive.unExposeAsWebService();

JVM

Web Service Client

Web Services

Denis Caromel35

ProActive + Services + Workflows

Principles:3 kinds of Parallel Services

3. Domain Specific Parallel Services (e.g. Monte Carlo Pricing)

2. Typical Parallel Computing Services (Parameter Sweeping, D&C, …)

1. Basic Job Scheduling Services (parallel execution on the Grid)

Denis Caromel36

3 kinds of Parallel Services

3. Domain Specific ParallelServices: providing business functionalities executed inparallel

2. Parallelization services:typical parallel computingpatterns (ParameterSweeping, D&C, …)

1. Job Scheduling service: Schedule and Run jobs in parallel on the Grid.

Parallel ServicesParallel Services Operational ServicesOperational Services…

1. Job Scheduling

Service

2. ParameterSweeping

Service

2. Divide & Conquer Service

OtherOperational

Service

…

Domain Specific Parallel Service

…Other

Basic Service

High level Business Process

GridGrid

3. Domain Specific Parallel

Service

Denis Caromel37

A sample pattern: Parameter Sweeping

I1I2

…In

O1O2

…On

ExecLogic

parameter sweeping

I1I2…In

O1O2…On

Parameter SweepingService, customized with an Exec logic X

ParamSweeping

Service

Process using parameter sweeping service

…

All the running instances of the Exec logic X are executed on the grid as a whole

PA Scheduler &PA Scheduler &PA Resource ManagerPA Resource Manager

Denis Caromel38

Demo

SOA Integration:Web Services, BPEL Workflow

Denis Caromel39 39

5. Enterprise Grids, Clouds:

Standards &Amazon EC2

Denis Caromel40

Deploy on Various Kinds of Infrastructures

Internet

Clusters

ParallelMachine

LargeEquipment

Internet

Job management forembarrassingly parallel application (e.g. SETI)

Internet

Internet

Servlets EJBs Databases

Denis Caromel41

GCM Fractal StandardizationFractal Based Grid Component Model

Overall, the standardization is supported by industrials: BT, FT-Orange, Nokia-Siemens, Telefonica,

NEC, Alcatel-Lucent, Huawei …

Denis Caromel42

Protocols and Scheduler inGCM Deployment Standard

Protocols:Rsh, sshOarsh, Gsissh

Scheduler, and Grids:GroupSSH, GroupRSH, GroupOARSHARC (NorduGrid), CGSP China Grid, EEGE gLITE, Fura/InnerGrid (GridSystem Inc.)GLOBUS, GridBusIBM Load Leveler, LSF, Microsoft CCS (Windows HPC Server 2008)Sun Grid Engine, OAR, PBS / Torque, PRUN

Soon available in stable release:Java EE

Amazon EC2

Denis Caromel43

6. J2EE IntegrationFlorin Alexandru Bratu

OASIS Team - INRIA

Denis Caromel44

J2EE Integration with Parallelism + Grids/Clouds

Performing Grid & Cloud Computing

From & In

an Application Servers

1. Delegating heavy computations outsides J2EE Applications

2. Using Deployed J2EE Nodes as Computational Resources

Denis Caromel45

ProActive – J2EE Integration (1)

1. Delegating heavy computations outsides J2EE Applications

Denis Caromel46

ProActive – J2EE Integration (2)

2. Using Deployed J2EE Nodes as Computational Resources

Objective Being able to deploy active objects inside the JVMs of application servers

Implementation Based on a Sun standard – Java Connector Architecture JSR112 Deployment module: resource adapter (RAR) Works with all J2EE-compliant Application Servers

Integration(2)

Denis Caromel48

Grids & Clouds:

Amazon EC2 Deployment

Denis Caromel49

Big Picture + Clouds

Denis Caromel50

Clouds: ProActive Amazon EC2 Deployment

Principles & Achievements:

ProActive Amazon Images (AMI) on EC2

So far up to 128 EC2 Instances

(Indeed the maximum on the EC2 platform, … ready to try 4 000 AMI)

Seamless Deployment:

no application change, no scripting, no pain

Open the road to : In house Enterprise Cluster and Grid + Scale out on EC2

Denis Caromel51

ProActive Deployment onAmazon EC2

Video

Denis Caromel52

On Going

AGOSGrid Architecture for SOA

Denis Caromel53

AGOSGrid Architecture for SOA

AGOS Solutions

Building a Platform for Agile SOA with Grid

In Open Source with Professional Support

Denis Caromel54

AGOS Generic Architecture for SOA with GRIDs

OS, HW

OS Virtualization Grid Utility interface

ESBEnterprise Service Bus

SCA Service Component Architecture

Resource Manager

Task & Services Scheduling

Parallel ProgrammingSPMD, workflow

Agent, Master/WorkerFork and Join

In memory db cache(JSR / JPI / javaspaces) SOA BPEL Exec

Repository, Registry, Orchestration

SOA MonitoringReporting, Notifications,

alarms

Business Intelligence BI Monitoring

SLM SLM

SLM

SLM

SLMSLM

SLMSLM

SLM SLM

Denis Caromel55

Summary

Denis Caromel56

Conclusion: Effective SOA + GRIDs in Java with

An Acceleration Toolkit :Concurrency+ParallelismMulti-Core+Distributed

Grid & SOA:J2EE, WS,BEPL, EG +Amazon EC2

Q&A

Denis Caromel58 www.devoxx.com