Status of analysis & MC farms
description
Transcript of Status of analysis & MC farms
Status of analysis Status of analysis & MC & MC farmsfarms
30/08/2002
Concezio BozziINFN Ferrara
30/08/0230/08/02 Concezio Bozzi (Ferrara)Concezio Bozzi (Ferrara)
The name of the gameThe name of the game
Three farms in Italy today:Three farms in Italy today:– ““MC”: for official Simulation Production MC”: for official Simulation Production
(formerly at Caspur, now at INFN Rome)(formerly at Caspur, now at INFN Rome)– ““Analysis”: a Tier B with Kanga data located Analysis”: a Tier B with Kanga data located
at Caspur (Rome)at Caspur (Rome)– ““Repro”: located in Padova, will reprocess all Repro”: located in Padova, will reprocess all
Babar Run1+Run2 data starting this summer Babar Run1+Run2 data starting this summer
30/08/0230/08/02 Concezio Bozzi (Ferrara)Concezio Bozzi (Ferrara)
INFN Farms in RomeINFN Farms in RomeMonteCarlo FarmMonteCarlo Farm
- TodayToday- UpgradeUpgrade
Analysis FarmAnalysis Farm - TodayToday- - UpgradeUpgrade
MonteCarlo Farm has been moved June 15MonteCarlo Farm has been moved June 15thth from Caspur to INFN (Rome) from Caspur to INFN (Rome)The Analysis Farm is still at CaspurThe Analysis Farm is still at Caspur
30/08/0230/08/02 Concezio Bozzi (Ferrara)Concezio Bozzi (Ferrara)
MonteCarlo Farm todayMonteCarlo Farm today- 1 server Linux, dual cpu, PIII 800 MHz, 512 MB1 server Linux, dual cpu, PIII 800 MHz, 512 MB (NFS and lock servers, batch queues, MocaEspresso)(NFS and lock servers, batch queues, MocaEspresso)
- 14 Linux clients, dual cpu, PIII 800 MHz, 768 MB14 Linux clients, dual cpu, PIII 800 MHz, 768 MB
- 2 Array RAID EIDE : 2x0,5 = 1 TB2 Array RAID EIDE : 2x0,5 = 1 TB
Event rate: 3.5 M events/monthEvent rate: 3.5 M events/month
30 M events in 1 year30 M events in 1 year
MonteCarlo Farm: jobsMonteCarlo Farm: jobs
30/08/0230/08/02 Concezio Bozzi (Ferrara)Concezio Bozzi (Ferrara)
Farm MonteCarlo: Farm MonteCarlo: comparisons …comparisons …
30/08/0230/08/02 Concezio Bozzi (Ferrara)Concezio Bozzi (Ferrara)
Farm MonteCarlo: Farm MonteCarlo: comparisons …comparisons …
19.4 M events from August 1519.4 M events from August 15thth 2001 to June 15 2001 to June 15thth 2002 2002
Farm MonteCarlo: Farm MonteCarlo: comparisons …comparisons …
we are here !
30/08/0230/08/02 Concezio Bozzi (Ferrara)Concezio Bozzi (Ferrara)
MonteCarlo Farm: CPUMonteCarlo Farm: CPU
SCSI problems stop/start productionre-validation
30/08/0230/08/02 Concezio Bozzi (Ferrara)Concezio Bozzi (Ferrara)
MonteCarlo Farm: disk & MonteCarlo Farm: disk & netnet
30/08/0230/08/02 Concezio Bozzi (Ferrara)Concezio Bozzi (Ferrara)
MonteCarlo Farm: UpgradesMonteCarlo Farm: Upgrades2002: increase production to 6-8 Mevents/month2002: increase production to 6-8 Mevents/month
- clone Linux farmclone Linux farm- 17 2-CPUs, 1.26GHz, 1 GB RAM: installing now17 2-CPUs, 1.26GHz, 1 GB RAM: installing now- Server and disk (1TB) orderedServer and disk (1TB) ordered
- use Analysis farm resources when availableuse Analysis farm resources when available-Problems with a Sun disk (1TB), recovering…Problems with a Sun disk (1TB), recovering…
2003: increase production to 20 M events/month2003: increase production to 20 M events/month
- more CPUs (150*30SpecInt95 ~ 35 1.6GHz more CPUs (150*30SpecInt95 ~ 35 1.6GHz 2xCPUs)2xCPUs)- use repro farm whenever possible use repro farm whenever possible
30/08/0230/08/02 Concezio Bozzi (Ferrara)Concezio Bozzi (Ferrara)
Analysis Farm: TodayAnalysis Farm: Today- 5 Sun E450, 4x400 MHz, 2GB (NFS data server)5 Sun E450, 4x400 MHz, 2GB (NFS data server)- 15 Linux clients, dual cpu, PIII 1 GHz, 1 GB: 15 Linux clients, dual cpu, PIII 1 GHz, 1 GB: - 1 Sun Ultra 10: lock server1 Sun Ultra 10: lock server- 1 Sun Ultra 10: monitoring (to be dismissed soon)- 1 Sun Ultra 10: monitoring (to be dismissed soon)- Disk space: 9 TB for Kanga Disk space: 9 TB for Kanga - Data: AllEvents + 8 streams. The complete (99-02) Data: AllEvents + 8 streams. The complete (99-02)
dataset is availabledataset is available- MC: ~650M events availableMC: ~650M events available- Private & Toy MC Private & Toy MC - SP MC production as “infn2” (lower priority wrt SP MC production as “infn2” (lower priority wrt
analysis)analysis)
30/08/0230/08/02 Concezio Bozzi (Ferrara)Concezio Bozzi (Ferrara)
Analysis Farm: usersAnalysis Farm: usersSU
NSU
NLin
uxLin
ux
30/08/0230/08/02 Concezio Bozzi (Ferrara)Concezio Bozzi (Ferrara)
Analysis Farm: jobsAnalysis Farm: jobsLin
uLin
uxx
CPU efficiencyCPU efficiency
Running jobsRunning jobs
Pending jobsPending jobs
30/08/0230/08/02 Concezio Bozzi (Ferrara)Concezio Bozzi (Ferrara)
Analysis farm: UpgradeAnalysis farm: Upgrade2002: 2002: - 17 Linux clients, dual cpu (in hand)17 Linux clients, dual cpu (in hand)- 3TB disk space (ordered): 15TB total3TB disk space (ordered): 15TB total
This will be enough until March 2003This will be enough until March 20032003:2003:- cpu power x3: 210 30SpecInts95 (50 1.6GHz 2xCPUs)cpu power x3: 210 30SpecInts95 (50 1.6GHz 2xCPUs)- Disk: from Padova (?)Disk: from Padova (?)- maintain Kanga until end of reprocessing, (spring 03)maintain Kanga until end of reprocessing, (spring 03)- Next: Computing Model Working GroupNext: Computing Model Working Group
OBJY vs. non-OBJY Event Store, MINI vs. MICRO, etc…OBJY vs. non-OBJY Event Store, MINI vs. MICRO, etc…
30/08/0230/08/02 Concezio Bozzi (Ferrara)Concezio Bozzi (Ferrara)
Babar-GridBabar-Grid- Some activity going on in ItalySome activity going on in Italy- Define and implement a Grid-scheme for MC Define and implement a Grid-scheme for MC
productionproduction- Installing/testing on the INFN-Grid Ferrara Installing/testing on the INFN-Grid Ferrara
testbedtestbed
30/08/0230/08/02 Concezio Bozzi (Ferrara)Concezio Bozzi (Ferrara)
Farmers ….Farmers ….- Ferrara:Ferrara:
D. Andreotti, V. Azzolini, C. Bozzi, E. Luppi + CS studentsD. Andreotti, V. Azzolini, C. Bozzi, E. Luppi + CS students- Padova:Padova:
A. Ceseracciu, A. Crescente, A. Dorigo, F. Galeazzi, M. A. Ceseracciu, A. Crescente, A. Dorigo, F. Galeazzi, M. Marzolla, M. Morandin, R. Stroili, G. Tiozzo, G. VedovatoMarzolla, M. Morandin, R. Stroili, G. Tiozzo, G. Vedovato
- Rome:Rome:C. Bulfon, F. Safai TehraniC. Bulfon, F. Safai Tehrani
- Trieste:Trieste:G. Della RiccaG. Della Ricca