Post on 13-Jan-2016
Your university or experiment logo here
LHCb Development
Glenn PatrickRaja Nandakumar
GridPP18, 20 March 2007
2
LHCb December 2006
Muon Calorimeters RICH2Trackers
Magnet
RICH1VELO
p p
3
2 kHz@30 kB/event60MB/s
LHCb Computing Model
40 MHz
Level-0Hardware
1 MHz
Level-1Software
HLTSoftware
40 kHz
4
DIRAC Production and Analysis
DIRAC JobManagement
Service
DIRAC JobManagement
Service
DIRAC CEDIRAC CEDIRAC CEDIRAC CE
DIRAC CEDIRAC CE
LCGLCGResourceBroker
ResourceBroker
CE 1CE 1
DIRAC SitesDIRAC Sites
AgentAgent AgentAgent AgentAgent
CE 2CE 2
CE 3CE 3
Productionmanager
Productionmanager GANGA UIGANGA UI User CLI User CLI
JobMonitorSvcJobMonitorSvc
JobAccountingSvcJobAccountingSvc
AccountingDB
Job monitorJob monitor
InformationSvcInformationSvc
FileCatalogSvcFileCatalogSvc
MonitoringSvcMonitoringSvc
BookkeepingSvcBookkeepingSvc
BK query webpage BK query webpage
FileCatalogbrowser
FileCatalogbrowser
Userinterfaces
DIRACservices
DIRACresources
DIRAC StorageDIRAC Storage
DiskFileDiskFile
gridftpgridftpbbftpbbftp
rfiorfio
Next talk
5
DIRAC Workload Management
Job Receiver
Job Receiver
JobJDL
Sandbox
JobInput
JobDB
Job Receiver
Job Receiver
Job Receiver
Job Receiver
DataOptimizer
DataOptimizer
TaskQueue
LFCLFC
checkData
AgentDirectorAgent
Director
checkJob
RBRBRBRBRBRB
PilotJob
CECE
WNWN
PilotAgentPilot
Agent
JobWrapper
JobWrapper
execute(glexec)
UserApplication
UserApplicationfork
MatcherMatcher
CEJDL
JobJDL
getReplicasWMSAdminWMS
Admin
getProxySE
uploadData
VO-boxVO-boxputRequest
AgentMonitorAgent
Monitor
checkPilot
getSandbox
JobMonitor
JobMonitor
DIRACservicesDIRAC
services
LCGservices
LCGservices
WorkloadOn WN
WorkloadOn WN
6
DIRAC3 Revision and Roadmap
Dec 2006. Brainstorming meeting at CERNamongst developers.
Jan 2007. Barcelona workshop.
Feb – April 2007. Re-implementation of the code baseaccording to new design.
May 2007. Integration of DIRAC3 system andthorough testing.
June 2007. Release of DIRAC 3.
• Operation in multiplatform environment - various Linux flavours, 32 bit/64 bit, Windows(!)
• Need to separate generic and LHCb behaviour.• Need for new functionality affecting multiple components (e.g. job
state machinery). • DIRAC3 will be the result of this major code revision and
reorganisation.
Gennady Kuznetsov (RAL)
7
Oracle DBOracle DB
JDB
C D
river
BK Service
Bookk eeping Svc B
oo kkeep ingQuer y
Tomcat Serv
let
Web
Browser
Read
Read
Jython Server
XM
L-RPC
GANGA
applicationRead
Read
volhcb01
Write
Read/Write
AM
GA
Client
GANGA
application
volhcb01
Write
AM
GA
AMGA Client
Re
ad
R/WR/W
AMGA-Bookkeeping Architecture
Carmine Cioffi (Oxford):AMGA now used in Production – old system retired.New production machine (volhcb01) for bookkeeping.
8
ReconstructionBrunel
SimulationGauss
DigitisationBoole
Strippingand Analysis
DaVinci
MC Truth Raw Data rDSTEvent TagCollection
DST+RAW
Software Modules and Data Flow
SoftwareInstallation moduleSoftwareInstallation module
GaussApplication moduleGaussApplication module
BookkeepingUpdate moduleBookkeepingUpdate module
Gauss Step
SoftwareInstallation moduleSoftwareInstallation module
BooleApplication moduleBooleApplication module
BookkeepingUpdate moduleBookkeepingUpdate module
Boole Step
Monte-Carlo Production Job
9
Last Month Activity
Bugs found, stop and restart the
production
Record of running jobs
9715
CNAF GRIDKA
IN2P3 NIKHEF
PIC
RAL
ALL
CERN
• Average of 7.5K running jobs in the last month• Temporary problems at PIC and RAL Raja Nandakumar (RAL)
10
CPU Use since Dec. 2006
COUNTRYCPU USE
(%)
UK 41.1
CERN 12.1
Italy 9.6
German 8.1
France 7.7
Spain 6.6
Greece 3.8
Netherlands 3.1
Poland 2.4
Russia 2.0
Hungary 0.9
UK
CER
N
German
SpainFrance
Italy
11
CPU Use since Dec. 2006
Main sitesCPU Time
(%)
Manchester 17
CERN 11
QMUL 8
CNAF 7
GRIDKA 7
IN2P3 4
Brunel(UK) 4
RAL 3
Glasgow 3
NIKHEF 3
USC 3
Lancashire 2
HG-06 (Greece) 2
Barcelona 2
PIC 2
Other sites 20
CPU Use - 40% @ T1s
Manchester
CERN
CNAF
QMUL
GRIDKA
12
Reconstruction since Dec. 2006
PIC32%
CNAF20%
CERN19%
IN2P317%
RAL10%
GRIDKA2%
NIKHEF0%
Tier 1 EventsReconstructed
PIC 32.4%
CNAF 20.1%
CERN 18.6%
IN2P3 16.5%
RAL 10.4%
GRIDKA 5.3%
NIKHEF 0.1%
Data access problems the main cause of delays to reconstruction.RAL dCache unstable since December.Problems with file staging through SRM.Some GridFTP problems.
New staging component in DIRAC
13
Data Transfer 1
Problems with transfers:When a job fails to transfer data to one or more T1, the transfer request is queued through VO box. Storage is not always available at T1s and number of pending transfer requests increase.
Failed
14
Data Transfer 2
Improvements:Temporary replication to a fail-over SE (all Tier 1s).Replication to final destination queued in VO box.VO box retries until transfer succeeds.Extremely reliable (multi-threaded transfer agent required).
Success
15
Next Steps
Castor-2
Castor Migration.LHCb tests progressing at RAL (Raja).Once jobs run and are stable aim to switch and replicate existing data from dCache to Castor.End June deadline for Castor approaching fast!
Data Stripping.Delayed to ~June because of late availability of high performance pre-selection algorithms.• Stripped DSTs to be shipped to all Tier 1 centres.• Analysis using Ganga.• Output used for LHCb “Physics Book”.
16
Alignment Challenge
First release of alignment framework – March.
First Alignment Challenge using tracking detectors – end April for production of datasets. ~June for alignment demonstration.
Second Alignment Challenge using all sub-detectors – September?VELO is most precise device in LHCb, but it
moves!
Retracted by ~3cm in between fills.
21 tracking stations. 4 sensors per station (r/ )
Different Configurations:Magnet OFF, VELO Open Magnet OFF, VELO Closed Magnet ON, VELO Open Magnet ON, VELO ClosedGrid test of Conditions Database – streaming of data constants andrunning of LHCb applications.
17
2007 Timetable
December 2007
January 2007
From March April: DAQ -Tier 0 throughput tests
June/July: Full chain DAQ -Tier 0 - Tier 1 tests
September: Re-reconstruction of b and Min Bias events
November: First data!
Jan - March: DC06 Production Phase
June: Release of DIRAC3end April - June: First Alignment Challenge
September: Second Alignment Challenge