LCG LCG-1 Deployment and usage experience Lev Shamardin SINP MSU, Moscow...

LCG

LCG-1 Deployment LCG-1 Deployment and usage and usage experienceexperience

Lev ShamardinLev Shamardin

SINP MSU, MoscowSINP MSU, Moscow

[email protected]@theory.sinp.msu.ru

18.09.2003 2Lev Shamardin

LCG

LCG GeographyLCG Geography

LCG covers a LCG covers a number of sites number of sites both in Europe both in Europe and USand US

LCG-1 testbed LCG-1 testbed covers 13 sites covers 13 sites including SINP including SINP MSUMSU


LCG

LCG-1LCG-1

LCG-1 is the production version of LCG-1 is the production version of the LCG software which is now the LCG software which is now installed on the 13 LCG-1 sites and installed on the 13 LCG-1 sites and running in production moderunning in production mode

New LCG-2 is expected in production New LCG-2 is expected in production in Novemberin November


LCG

LCG-1 ArchitectureLCG-1 Architecture Minimal LCG-1 site must Minimal LCG-1 site must

have a Computing have a Computing Element with worker Element with worker nodes and a Storage nodes and a Storage ElementElement

There are no limitations There are no limitations to the number of other to the number of other components installed on components installed on each siteeach site

These components do These components do not have to be not have to be registered in LCG-1registered in LCG-1 LCG-1

site

LCG-1 site

MDS west

MDS eastRB

LCG-1 site

LCG-1 site

LCG-1 site

LCG-1 site


LCG

LCG-1 ArchitectureLCG-1 Architecture

Current version LCG-1 is based on Current version LCG-1 is based on the EDG middleware with MDS the EDG middleware with MDS information systeminformation system

R-GMA is not yet stable enough for R-GMA is not yet stable enough for the production usagethe production usage

LCG-2 will be using new R-GMA LCG-2 will be using new R-GMA information systeminformation system


LCGLCFGng and LCG-1 LCFGng and LCG-1 deploymentdeployment The base middleware The base middleware

LCFGng configuration LCFGng configuration profiles are stored in profiles are stored in the central LCG CVS the central LCG CVS repositoryrepository

Sites create their own Sites create their own site-specific profiles site-specific profiles based on the CVS based on the CVS configurationconfiguration

No manual installation No manual installation supported yetsupported yet

LCG CVS

Generic LCG-1 configuration

LCG-1 Deploymentgroup

LCG-1 Site LCG-1 Site LCG-1 Site…LCG-1 Site

Site-specific configuration


LCG

SINP MSU PC FarmSINP MSU PC Farm

20 dual-CPU PIII 20 dual-CPU PIII nodesnodes

Two 1.2 TB Two 1.2 TB fileserversfileservers

Gigabit Ethernet Gigabit Ethernet uplinks to the uplinks to the fileserversfileservers

Fast Ethernet links to Fast Ethernet links to the nodesthe nodes

Sto

rage

Sto

rage

switch

non-interactivenodes

PC Farmbatch master

node


LCGSoftware installation Software installation system on SINP MSU PC system on SINP MSU PC FarmFarm Based on Etherboot Based on Etherboot

network boot packagenetwork boot package Nodes are installed Nodes are installed

with anaconda kickstartwith anaconda kickstart Nodes without a boot Nodes without a boot

ROM are installed using ROM are installed using a fake linux “kernel” or a fake linux “kernel” or a boot diska boot disk

Supports both Supports both completely unattended completely unattended automatic installation automatic installation and manual installationand manual installation

DHCP

TFTP

configurationserver

NFS

node

file server


LCGLCFGng support on SINP LCFGng support on SINP MSU PC FarmMSU PC Farm LCFGng server was LCFGng server was

installed for configuring installed for configuring and installing LCG-1 and installing LCG-1 nodesnodes

LCFGng enabled nodes LCFGng enabled nodes configuration is configuration is controlled from the controlled from the LCFGng serverLCFGng server

Nodes configuration and Nodes configuration and health status information health status information provided with LCFGng provided with LCFGng components can be components can be observed on the webobserved on the web

DHCP

TFTP

configurationserver

NFS

node

file server LCFGng server

LCFG,HTTP


LCGCurrent LCG-1 status in Current LCG-1 status in SINP MSUSINP MSU For the LCG-1 testbed part For the LCG-1 testbed part

of the PC Farm was of the PC Farm was configured as LCG-1 nodesconfigured as LCG-1 nodes

These nodes are logically These nodes are logically disconnected from the disconnected from the main farmmain farm

Installed LCG-1 Installed LCG-1 components are: components are: Computing Element with Computing Element with several Worker Nodes, several Worker Nodes, Storage Element, Resource Storage Element, Resource Broker, BDII and User Broker, BDII and User InterfaceInterface

MyProxy and MDS servers MyProxy and MDS servers are coming soonare coming soon Sto

rage

Sto

rage

switch

non-interactivenodes

PC Farmbatch master

node

LCG-1


LCG

MSU PC ClustersMSU PC Clusters LCG middleware installation on the SRCC MSU LCG middleware installation on the SRCC MSU

(Scientific Research Computing Center) parallel (Scientific Research Computing Center) parallel clustercluster Site specifics:Site specifics:

Exotic batch system. Interface for the globus package is Exotic batch system. Interface for the globus package is ready but was not completely testedready but was not completely tested

Only manual node configuration can be doneOnly manual node configuration can be done Middleware for the worker nodes must be installed on the Middleware for the worker nodes must be installed on the

shared filesystemshared filesystem Connect the Physical faculty and the faculty of Connect the Physical faculty and the faculty of

Computing Mathematics and Cybernetics clusters Computing Mathematics and Cybernetics clusters to the SINP MSU Resource Brokerto the SINP MSU Resource Broker Experience with SRCC is required, similar site-specific Experience with SRCC is required, similar site-specific

limitationslimitations


LCG

ConclusionConclusion

What difficulties will emerge if a site What difficulties will emerge if a site wants to install the LCG middlewarewants to install the LCG middleware

What LCG can give you right nowWhat LCG can give you right now Future plansFuture plans


LCG

DifficultiesDifficulties At the moment the only documented way to install LCG At the moment the only documented way to install LCG

software requires using the LCFGng configuration serversoftware requires using the LCFGng configuration server Using LCFGng is not possible in a number of cases, in general Using LCFGng is not possible in a number of cases, in general

due to the administrative reasonsdue to the administrative reasons Minimal stand-alone site must be running quite a big Minimal stand-alone site must be running quite a big

number of nodes supporting the infrastructurenumber of nodes supporting the infrastructure The minimal set is:The minimal set is:

Resource BrokerResource Broker BDII & MDSBDII & MDS Computing Element with Worker NodesComputing Element with Worker Nodes Storage Element(s)Storage Element(s) MyProxy if long-time jobs support is requiredMyProxy if long-time jobs support is required

This gives us at least 5 nodes will be supporting the This gives us at least 5 nodes will be supporting the infrastructureinfrastructure


LCGWhat LCG-1 can give you What LCG-1 can give you right nowright now Convenient way for job balancing Convenient way for job balancing

between several sitesbetween several sites Common way of user authentication Common way of user authentication

and authorization for job submissionand authorization for job submission Some basic accountingSome basic accounting Data replicationData replication


LCG

Future plansFuture plans

SINP MSU Site is one of the sites SINP MSU Site is one of the sites participating in the LCG-1participating in the LCG-1

New Russian sites will be connected New Russian sites will be connected to the SINP MSU Resource Broker in to the SINP MSU Resource Broker in the nearest futurethe nearest future IHEP Protvino will be connected soonIHEP Protvino will be connected soon ITEP and JINR Dubna in futureITEP and JINR Dubna in future Internal MSU sites (SRCC and others) as Internal MSU sites (SRCC and others) as

soon as manual installation is possiblesoon as manual installation is possible

LCG LCG-1 Deployment and usage experience Lev Shamardin SINP MSU, Moscow...

Documents

Transcript of LCG LCG-1 Deployment and usage experience Lev Shamardin SINP MSU, Moscow...