1CHEP 2000, 10.02.2000 Roberto Barbera

Roberto Barbera(*)

GENIUS: a Web Portal for DataGRID

ACAT 2002, Moscow, 24-28.06.2002

(*)work in collaboration with A. Andronico, A. Falzone and A. Rodolico

2Roberto Barbera

OutlineThe Grid Vision: the concept of Collaboratory

Grid activities in Europe: the DataGrid Project

The main “actors” of the DataGrid Project

“Easy” ed “ubiquitous” access to the Grid: the GENIUS web portal

Conclusions and perspectives

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

ACAT 2002, Moscow, 24-28.06.2002

3Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

ACAT 2002, Moscow, 24-28.06.2002

4

GRID Projects – EU IST (~37 M€)An integrated approach

ScienceIndustry / business

ApplicationsApplications

MiddlewareMiddleware& Tools& Tools

Underlying Underlying InfrastructuresInfrastructures

GR

IDS

TA

RT

CROSSGRID

DATAGRID

DATATAG

GRIDLAB

EGSO

GRIA

GRIP EUROGRID

DAMIEN

iVDGL

ACAT 2002, Moscow, 24-28.06.2002

5Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

CERN – International (Switzerland/France)

CNRS - France

ESA/ESRIN – International (Italy)

INFN - Italy

NIKHEF – The Netherlands

PPARC - UK

DataGRID main contractors

ACAT 2002, Moscow, 24-28.06.2002

6Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

Research and Academic Institutes•CESNET (Czech Republic)•Commissariat à l'énergie atomique (CEA) – France•Computer and Automation Research Institute,  Hungarian Academy of Sciences (MTA SZTAKI)•Consiglio Nazionale delle Ricerche (Italy)•Helsinki Institute of Physics – Finland•Institut de Fisica d'Altes Energies (IFAE) - Spain•Istituto Trentino di Cultura (IRST) – Italy•Konrad-Zuse-Zentrum für Informationstechnik Berlin - Germany•Royal Netherlands Meteorological Institute (KNMI)•Ruprecht-Karls-Universität Heidelberg - Germany•Stichting Academisch Rekencentrum Amsterdam (SARA) – Netherlands•Swedish Research Council - Sweden

Industrial Partners•Datamat (Italy)•IBM-UK (UK)•CS-SI (France)

Other DataGRID partners

ACAT 2002, Moscow, 24-28.06.2002

7

EDG overview : work packagesEDG Project is structured in 12 Work Packages: WP1: Work Load Management System WP2: Data Management WP3: Grid Monitoring / Grid Information Systems WP4: Fabric Management WP5: Mass Storage Management WP6: Testbed and demonstrators WP7: Network Monitoring WP8: High Energy Physics Applications WP9: Earth Observation WP10: Biology WP11: Dissemination WP12: Management

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

}}MiddlewareMiddleware

}}ApplicationsApplications

Roberto BarberaACAT 2002, Moscow, 24-28.06.2002

8Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

Computational biology

ACAT 2002, Moscow, 24-28.06.2002

9Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

Medical Diagnostic Imaging

ACAT 2002, Moscow, 24-28.06.2002

Earth Observation Community GRID interactive scenario

Common access to EO missions cataloguesAcquisition plan, order, delivery

Parametric data fusion and models integration

Collaborative publishing of results

On demand high level products generation

Roberto BarberaACAT 2002, Moscow, 24-28.06.2002

11

ENVISAT (launched 01.03.2002 !)• 3500 MEuro programme cost3500 MEuro programme cost

• 10 instruments on board10 instruments on board• 200 Mbps data rate to ground200 Mbps data rate to ground• 400 Tbytes data archived/year400 Tbytes data archived/year• ~100 “standard” products~100 “standard” products• 10+ dedicated facilities in Europe10+ dedicated facilities in Europe

• ~700 approved science user projects~700 approved science user projects

• 3500 MEuro programme cost3500 MEuro programme cost

• 10 instruments on board10 instruments on board• 200 Mbps data rate to ground200 Mbps data rate to ground• 400 Tbytes data archived/year400 Tbytes data archived/year• ~100 “standard” products~100 “standard” products• 10+ dedicated facilities in Europe10+ dedicated facilities in Europe

• ~700 approved science user projects~700 approved science user projects

Earth Observation

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

Roberto Barbera

12

http://www.cern.ch

~9 km

LHC

SPS

CERN

Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

High Energy Physics

ACAT 2002, Moscow, 24-28.06.2002

13Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

High Energy Physics

ACAT 2002, Moscow, 24-28.06.2002

14

ATLAS CMS

LHCb

Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

~6-8 PetaBytes / year~O(108) events/year

~O(103) batch and interactive users

High Energy Physics

ACAT 2002, Moscow, 24-28.06.2002

15Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

1/100 of a Pb+Pb @ LHC !

Simulation and reconstruction of a “full” (central) Pb+Pb collision at LHC (about 84000 primary tracks!) takes about 24 hours of a top-PC and produces an output bigger than 2 GB.

ACAT 2002, Moscow, 24-28.06.2002

16Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

5 times the Eiffel Tower

˜1500 m

6-8 Petabytes ˜10.000.000 CD-ROM

ACAT 2002, Moscow, 24-28.06.2002

17Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

Yerevan

CERN

Saclay

Lyon

Dubna

Capetown, ZA

Birmingham

Cagliari

NIKHEF

GSI

Catania

BolognaTorino

Padova

IRB

Kolkata, India

OSU/OSCLBL/NERSC

Merida

Bari

Nantes

ALICE “grid” sites

ACAT 2002, Moscow, 24-28.06.2002

18Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

Europe: 267 institutes, 4603 usersElsewhere: 208 institutes, 1632 users

The LHC “web” in the world

ACAT 2002, Moscow, 24-28.06.2002

19Roberto BarberaACAT 2002, Moscow, 24-28.06.2002

Job Submission work-flow

UIJDL

Logging &Logging &Book-keepingBook-keeping

ResourceResourceBrokerBroker

Job SubmissionJob SubmissionServiceService

StorageStorageElementElement

ComputeComputeElementElement

Information Information ServiceService

Job Status

ReplicaReplicaCatalogueCatalogue

DataSets info

Author.&Authen.

Job S

ub

mit

Even

t

Job

Qu

ery

Job

Stat

us

Input “sandbox”

Input “sandbox” + Broker InfoGlobus RSL

Output “sandbox”

Output “sandbox”

Job Status

Pu

blis

h

grid

-pro

xy-in

it

Exp

and

ed J

DL

SE & CE info

Roberto BarberaACAT 2002, Moscow, 24-28.06.2002

21Roberto Barbera

EDG m/w has been released but…EDG software (Globus, UI, JDL, WP2, WP3, etc.) contains tens of commands/switches which also have their own logical sequences (“B” after “A”, “C” before “D” and so on).

Browsing Grid VO “directories” (users, RC’s, DB’s, etc.) requires LDAP “speaking” and tomorrow could require SQL “speaking”.

“User gridification” is a tough task for a “rookie” this does not fit with the claim that “grids” are for everybody and that grid computing will be as easy as surfing the Internet ?

Furthermore, all this holds for DataGrid. What will happen when other grids’ software (especially UI’s) will come up (PPDG, iVDGL, etc.) ? Will users have to learn tens of “grid dialects” ?

Today “grid computing” is a rather complicated experience only possible at selected machines (UI’s) this does not fit with the claim that one could do “grid computing” even from a PDA ?

Is there any way to set-up a “user-friendly” grid ?

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

ACAT 2002, Moscow, 24-28.06.2002

22Roberto Barbera

A web portal: why and how ?It can be accessed from everywhere and by “everything” (desktop, laptop, PDA, WAP phone).It can keep the same user interface to several back-ends (grid “dialects” command-line UI’s).It must be redundantly “secure” at all levels: 1) secure for web transactions, 2) secure for user credentials, 3) secure for user authentication, 4) secure at VO level.All available grid services must be incorporated in a logic way, just “one mouse click away”.Its layout must be easily understandable and user friendly.

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

ACAT 2002, Moscow, 24-28.06.2002

23Roberto Barbera

GENIUS®

(Grid Enabled web eNvironment for

site Independent User job Submission)

[https://genius.ct.infn.it]

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

OS & Net services

Basic Services

High level GRID middleware

ALICE ATLAS CMS LHCbApplications’specific layer

Other apps

GLOBUS

toolkit

DataGRID architectur

e

GENIUS web portal

ACAT 2002, Moscow, 24-28.06.2002

24Roberto Barbera

GENIUS: how it works

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

Apache

EnginFrame

GENIUShttps+java/xml+rfb

WEB Browser

EDGUI

Local

WS

the GridEDG+GSI

3-tier model

25Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

GENIUS show: the main page

26Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

GENIUS show: the authentication

27Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

GENIUS show: file services

28Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

GENIUS show: the authorization

29Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

GENIUS show: security services

30Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

GENIUS show: job submission

31Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

GENIUS show: job submission

32Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

GENIUS show: job queue

33Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

GENIUS show: job output

34Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

GENIUS show: personal spooler

35Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

GENIUS show: job data

36Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

GENIUS show: interactive analysis

37Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

GENIUS show: the VO services

38Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

GENIUS show: the VO services

39Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

GENIUS show: monitoring services

40Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

GENIUS show: monitoring services

41Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

GENIUS show: monitoring services

42Roberto Barbera

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

GENIUS show: monitoring services

43Roberto Barbera

Present status and perspectivesCurrent implementation of GENIUS already includes: secure web transactions, user authentication and

authorization; remote interaction with the user’s file system; interfaces for job submission/control, to VO servers

(users’ and catalogues), and to monitoring systems; persistent (user’s) book-keeping and spooler system; interactive analysis ! rpm available !

Todo: multi-jobs (parallel and sequential); interface to data management and other grid services; more application-specific customizations; web-guided creation of a work flow system.

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

44Roberto Barbera

GENIUS vs. other grid portalsIt is not a toolkit. It is a complete production-ready environment which combine the concepts of “user portal” and “science portal”.No client software needs to be installed apart from the web browser. GENIUS can be accessed from everywhere.No security delegation (“à la MyProxy”) is needed. Access passwords are securely “streamed” only when needed.Interactive analysis (via VNC) and web access to personal spooling areas are possible.User file system is not limited to input and output files.EnginFrame modularity makes different customizations easy to implement. Already available for EDG m/w, GLOBUS, and LSF. Under definition for CONDOR (hungarian grids). It is compatible with the Tomcat open source java servlet container available from SUN.

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

45Roberto Barbera

ConclusionsComputational grids could represent the “natural” environment for next generation high energy physics experiments, computational bio-medecine, Earth observation and many other inter-disciplinary applications. “Grid” could be the Internet “new age” where users can seamlessly and ubiquitously access not only information but also huge computing resources and mass storage systems distributed worldwide with their own applications.However, in order to turn dreams into reality, grid access must be easy and intuitive especially for the vast majority of non-expert users and this is the mandate of the GENIUS team. Live demos of GENIUS at work could be made for interested people during breaks in the e-mail reading area.

Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy

ALICE Collaboration

ACAT 2002, Moscow, 24-28.06.2002