COMP Superscalar: Bringing GRID superscalar and GCM together

COMP Superscalar:Bringing GRID superscalar

and GCM together

Enric TejedorUniversitat Politècnica de

[email protected]

V ProActive and GCM User Group, GridCOMP conferenceOctober 21, 2008. INRIA, Sophia Antipolis, France

Rosa M. BadiaBarcelona Supercomputing

[email protected]

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Outline

• Introduction

• Grid Component Model

• GRID superscalar

• Programming model

• Runtime operation

• Initialisation

• Task processing

• Finalisation and error handling

• Conclusions


1. Introduction


Basic concepts

•The Grid Component Model (GCM)

• CoreGRID NoE, GridCOMP

• Distributed, based on Fractal, intended for the Grid

•GRID superscalar

• Framework to ease the development of Grid-unaware applications

• Simple programming model: Grid as transparent as possible

• Runtime that optimises the performance of the application (exploiting

possible concurrency)


A componentised Grid framework

• GRID superscalar is suitable to benefit from the GCM features

• Componentisation process: identify the inner functionalities of the runtime,

and assign each one to a separate component

• Result: COMP Superscalar, whose runtime gains in:

• Reusability

• Flexibility

• Deployability

• Separation of concerns

• Ease of development


Componentised runtime


2. Programming model


...for (i=0; i<N; i++){ T1 (data1, data2); T2 (data4, data5); T3 (data2, data5, data6); T4 (data7, data8); T5 (data6, data8, data9);}...

Sequential Application

T10 T20

T30

T40

T50

T11 T21

T31

T41

T51

T12

…

Resource 1

Resource 2

Resource 3

Resource N

.

.

.

Task graph creation

based on data

precedence

Task selection +

parameters direction

(input, output, inout)

Scheduling,

data transfer,

task execution

Synchronization,

results transfer

Parallel Resources(multicore,SMP, cluster, grid)

Programming model


initialize(f1);

for (int i = 0; i < 2; i++) {

genRandom(f2);

add(f1, f2);

}

print(f2);

Java application

COMPSs programming model

public interface SumItf {

@ClassName(“example.Sum")

@MethodConstraints(OSType = "Linux")

void genRandom(

@ParamMetadata(type = Type.FILE, direction = Direction.OUT)

String f

);

@ClassName(“example.Sum")

...

}

Task constraints

Parametermetadata

Implementation

Java interface


Custom Java Class Loader

Java app code

COMPSs runtime

Annotated interface

Javassist

insertscalls to

Custom Loader

uses

input


3. Runtime operation


Implementation details

•Base technologies:

• Java as programming language

• ProActive: implementation of the GCM model, used to build the components

• JavaGAT: used for job submission and file transfer


Initialization

•Multicast invocation

• Forwarded to all subcomponents

TA TS JM

FM

FIP FTM

init()


Initialization

•Multicast invocation

• Reduction of return values

TA TS JM

FM

FIP FTMReduction

Reduction


Task processing (1)

•Application submits task

• Task Analyser receives the request

TA TS JM

FM

FIP FTM

executeTask(…)

T1genRd

T3genRd

T2add

T4add

f1 f1

f2


Task processing (2)

TA JM

FM

FIP FTM

TS

•File Information Provider is contacted

• File accesses of the task are registered

executeTask(…)


Task processing (3)

•Task Analyser discovers dependencies

• Dependency-free tasks are sent for scheduling

TA TS JM

FM

FIP FTM

executeTask(…)


Task processing (4)

•Task Scheduler decides where to execute tasks

• Job Manager is informed of this decision

TA TS JM

FM

FIP FTM

executeTask(…) Scheduling

algorithm

Resourcecapabilitie

s

Taskconstraint

s


Task processing (5)

• Job Manager checks the necessary file transfers

• File Transfer Manager actually performs them

TA TS JM

FM

FIP FTM

executeTask(…)

GAT


Task processing (6)

•FTM informs of the end of transfers for a task

• Task Grid job, submitted for execution

TA TS JM

FM

FIP FTM

executeTask(…)

GAT


Task processing (7)

•Callback for the end of a job is received

• The notification is forwarded to the Task Analyser

TA TS JM

FM

FIP FTM

executeTask(…)

GAT


Safe stopping process for the runtime

•COMPSs subcomponents have data dependencies

•Modification of the Life Cycle Controller

• Components stopped in a safe order: TS, TA, JM, FTM, FIP

• Otherwise, a deadlock could happen

TATA TS JM

FM

FIP FTM

LCC


4. Conclusions


Conclusions

•COMP Superscalar: componentised version of GRID superscalar

• GCM-based

• Hierarchical runtime: separation of concerns

• Task scheduling, file and resource management

•Straightforward programming model for Grid-unaware Java

applications

•Feedback about GCM-ProActive

• Component distribution

• Performance: no noticeable overhead for a medium number of tasks

• Component composition and dynamic reconfiguration: Grid IDE


Thank you!

Questions?

COMP Superscalar: Bringing GRID superscalar and GCM together

Documents

Transcript of COMP Superscalar: Bringing GRID superscalar and GCM together