COMP Superscalar:Bringing GRID superscalar

and GCM together

Enric TejedorUniversitat Politècnica de

Catalunyaetejedor@ac.upc.edu

V ProActive and GCM User Group, GridCOMP conferenceOctober 21, 2008. INRIA, Sophia Antipolis, France

Rosa M. BadiaBarcelona Supercomputing

Centerrosa.m.badia@bsc.es

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Outline

• Introduction

• Grid Component Model

• GRID superscalar

• Programming model

• Runtime operation

• Initialisation

• Task processing

• Finalisation and error handling

• Conclusions

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

1. Introduction

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Basic concepts

•The Grid Component Model (GCM)

• CoreGRID NoE, GridCOMP

• Distributed, based on Fractal, intended for the Grid

•GRID superscalar

• Framework to ease the development of Grid-unaware applications

• Simple programming model: Grid as transparent as possible

• Runtime that optimises the performance of the application (exploiting

possible concurrency)

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

A componentised Grid framework

• GRID superscalar is suitable to benefit from the GCM features

• Componentisation process: identify the inner functionalities of the runtime,

and assign each one to a separate component

• Result: COMP Superscalar, whose runtime gains in:

• Reusability

• Flexibility

• Deployability

• Separation of concerns

• Ease of development

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Componentised runtime

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

2. Programming model

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

...for (i=0; i<N; i++){ T1 (data1, data2); T2 (data4, data5); T3 (data2, data5, data6); T4 (data7, data8); T5 (data6, data8, data9);}...

Sequential Application

T10 T20

T30

T40

T50

T11 T21

T31

T41

T51

T12

…

Resource 1

Resource 2

Resource 3

Resource N

.

.

.

Task graph creation

based on data

precedence

Task selection +

parameters direction

(input, output, inout)

Scheduling,

data transfer,

task execution

Synchronization,

results transfer

Parallel Resources(multicore,SMP, cluster, grid)

Programming model

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

initialize(f1);

for (int i = 0; i < 2; i++) {

genRandom(f2);

add(f1, f2);

}

print(f2);

Java application

COMPSs programming model

public interface SumItf {

@ClassName(“example.Sum")

@MethodConstraints(OSType = "Linux")

void genRandom(

@ParamMetadata(type = Type.FILE, direction = Direction.OUT)

String f

);

@ClassName(“example.Sum")

...

}

Task constraints

Parametermetadata

Implementation

Java interface

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Custom Java Class Loader

Java app code

COMPSs runtime

Annotated interface

Javassist

insertscalls to

Custom Loader

uses

input

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

3. Runtime operation

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Implementation details

•Base technologies:

• Java as programming language

• ProActive: implementation of the GCM model, used to build the components

• JavaGAT: used for job submission and file transfer

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Initialization

•Multicast invocation

• Forwarded to all subcomponents

TA TS JM

FM

FIP FTM

init()

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Initialization

•Multicast invocation

• Reduction of return values

TA TS JM

FM

FIP FTMReduction

Reduction

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Task processing (1)

•Application submits task

• Task Analyser receives the request

TA TS JM

FM

FIP FTM

executeTask(…)

T1genRd

T3genRd

T2add

T4add

f1 f1

f2

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Task processing (2)

TA JM

FM

FIP FTM

TS

•File Information Provider is contacted

• File accesses of the task are registered

executeTask(…)

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Task processing (3)

•Task Analyser discovers dependencies

• Dependency-free tasks are sent for scheduling

TA TS JM

FM

FIP FTM

executeTask(…)

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Task processing (4)

•Task Scheduler decides where to execute tasks

• Job Manager is informed of this decision

TA TS JM

FM

FIP FTM

executeTask(…) Scheduling

algorithm

Resourcecapabilitie

s

Taskconstraint

s

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Task processing (5)

• Job Manager checks the necessary file transfers

• File Transfer Manager actually performs them

TA TS JM

FM

FIP FTM

executeTask(…)

GAT

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Task processing (6)

•FTM informs of the end of transfers for a task

• Task Grid job, submitted for execution

TA TS JM

FM

FIP FTM

executeTask(…)

GAT

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Task processing (7)

•Callback for the end of a job is received

• The notification is forwarded to the Task Analyser

TA TS JM

FM

FIP FTM

executeTask(…)

GAT

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Safe stopping process for the runtime

•COMPSs subcomponents have data dependencies

•Modification of the Life Cycle Controller

• Components stopped in a safe order: TS, TA, JM, FTM, FIP

• Otherwise, a deadlock could happen

TATA TS JM

FM

FIP FTM

LCC

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

4. Conclusions

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Conclusions

•COMP Superscalar: componentised version of GRID superscalar

• GCM-based

• Hierarchical runtime: separation of concerns

• Task scheduling, file and resource management

•Straightforward programming model for Grid-unaware Java

applications

•Feedback about GCM-ProActive

• Component distribution

• Performance: no noticeable overhead for a medium number of tasks

• Component composition and dynamic reconfiguration: Grid IDE

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Thank you!

Questions?