Computing in CMSComputing in CMS

May 24, 2002 NorduGrid Helsinki workshop

Veikko Karimäki/HIP

Nor

dUG

rid,

Hel

sink

i May

23-

24, 2

002

Slide 2

OutlineOutline

CMS software overview SIMULATION

RECONSTRUCTION

VISUALISATION

ANALYSIS

Production activities

Schedules of CMS Data Challenge

Nor

dUG

rid,

Hel

sink

i May

23-

24, 2

002

Slide 3

CMSIM (soon OSCAR)

FZsignal

HEPEVTNtuple

ORCA

FZsignal

FZminbias

OODBminbias

OODBsignal

OODBDigis

Ntuple

G3Reader SimReader RecReader

MC generator

CMKIN

Production

Production User

Simulation-Reconstruction-Analysis Chain (now)Simulation-Reconstruction-Analysis Chain (now)

OODBTags

Generation Simulation

Digitization Reconstruction

Nor

dUG

rid,

Hel

sink

i May

23-

24, 2

002

Slide 4

Software ProjectsSoftware Projects

CMSIM (The original GEANT3 Simulation of CMS) CARF (CMS Analysis and Reconstruction Framework

(deprecated) In 2001 was split out of ORCA repository to become:

COBRA (Coherent Object-oriented Base for simulation, Reconstruction and Analysis)

OSCAR (Object oriented Simulation for CMS Analysis and Reconstruction)

The GEANT4 Simulation framework for CMS ORCA (Object Reconstruction for CMS Analysis)

The OO reconstruction program IGUANA (Interactive Graphical User ANAlysis)

Toolkits for Interactive Analysis FAMOS (Fast Monte-Carlo Simulation)

“Smearing” Monte-Carlo DDD (needs an Acronym!)

The Detector Description Database

See PRS talks for use of CMSIM, CARF/COBRA,

ORCA

See PRS talks for use of CMSIM, CARF/COBRA,

ORCA

New Packages, IGUANA, OSCAR, FAMOS, DDD, not

yet in Physicist use, see following slides

New Packages, IGUANA, OSCAR, FAMOS, DDD, not

yet in Physicist use, see following slides

Nor

dUG

rid,

Hel

sink

i May

23-

24, 2

002

Slide 5

Software Process ComponentsSoftware Process Components

SCRAMToolBox Product specification and versioning, Product break-down structure Configuration definition and versioning

SCRAM Configuration specification Assembly break-down Mapping of local resources onto

configuration requirements Build, Source-code distribution

DAR Self-contained binary distribution (prod)

cvs Source code management

CVSpm Standardized directory structures and

repositories, responsibility and access right

information and control.BugsRS

Error reporting

Insure, workshop Memory leaks performance monitoring, etc..

CodeWizard Coding rule checking

Ignominy General dependencies some project metrics

McCabe,and risk pages Measurement

(OO and procedural metrics) Risk evaluation

DepUty Dependencies style checking UML package diagrams

Oval Regression testing

unit, cluster, acceptance, verification, validation

Nor

dUG

rid,

Hel

sink

i May

23-

24, 2

002

Slide 6

ORCA Project RelationsORCA Project Relations

Objectivity, Anaphe, Geant4, ...

COBRA

OSCAR

FAMOS

FrameworkVisualisation

Reconstruction

Simulation

Fast Reco/Simu

(replaces CMSIM soon)

Nor

dUG

rid,

Hel

sink

i May

23-

24, 2

002

Slide 7

Tracker: Track ReconstructionTracker: Track Reconstruction

Generation of seeds (Seed Generator)

Construction of trajectories for a given seed (Trajectory Builder) Ambiguity resolution (Trajectory Cleaner) Final fit of trajectories (Trajectory Smoother)

Each component has one or more implementation. Three different algorithms are currently fully implemented: ( Combinatorial Track

Finding, Connection Machine, Deterministic Annealing Filter)

Nor

dUG

rid,

Hel

sink

i May

23-

24, 2

002

Slide 8

IGUANA Detector and Event DisplayIGUANA Detector and Event Display

Nor

dUG

rid,

Hel

sink

i May

23-

24, 2

002

Slide 9

Tracker - analysis exampleTracker - analysis example

Resolutions for 100 GeV muons Number of hits used

Nor

dUG

rid,

Hel

sink

i May

23-

24, 2

002

Slide 10

CMS Distributed Production (~2001)CMS Distributed Production (~2001)

~1100 available

CPU

>30 involved persons

23 Sites11 RC

(Regional Centres)

Location

?21Wisconsin

4011UCSD

12021Florida

8051FNAL

34023Caltech

USA

6024MoscowRussia

5012Bristol/RAL

150107INFN

9631IN2P3

1021Helsinki

20041CERN

Europe

Nor

dUG

rid,

Hel

sink

i May

23-

24, 2

002

Slide 11

Production 2002, ComplexityProduction 2002, Complexity

File Transfer by GDMP and by perl Scripts over scp/bbcp

15 TBData Size (Not including fz files from Simulation)

~11,000Number of Files

6-8Number of Production Passes for each Dataset(including analysis group processing done by production)

176 CPUsLargest Local Center

~1000Number of CPU’s

21Number of Computing Centers

11Number of Regional Centers

Nor

dUG

rid,

Hel

sink

i May

23-

24, 2

002

Slide 12

TYPICAL EVENT SIZES

Simulated 1 CMSIM event

= 1 OOHit event= 1.4 MB

Reconstructed 1 “1033” event

= 1.2 MB 1 “2x1033” event

= 1.6 MB1 “1034” event

= 5.6 MB

CMS Produced Data in 2001CMS Produced Data in 2001

0.05MFlorida

0.06MUCSD

0.07MWisconsin

0.13MHelsinki

0.31MIN2P3

0.43MMoscow

0.76MINFN

1.10MCERN

1.27MBristol/RAL

1.65MFNAL

2.5MCaltechSimulated Events

0.05TBWisconsin

0.08TBFlorida

0.10TBIN2P3

0.20TBUCSD

0.22TBBristol/RAL

0.40TBINFN

0.45TBMoscow

0.60TBCaltech

12TBFNAL

14TBCERN

Reconstructed with pileup

Nor

dUG

rid,

Hel

sink

i May

23-

24, 2

002

Slide 13

May 10. Production Status

0

1,000,000

2,000,000

3,000,000

4,000,000

5,000,000

6,000,000

7,000,000

Simulated Hit No PU 2x10^33 10^34 Filtered

Eve

nts

Requested

Produced

Production Status 2002Production Status 2002

On schedule for June 1 deadline

On schedule for June 1 deadline

Imperial College

Bristol/RAL

Wisconsin

UCSD

Moscow

IN2P3

INFN

FNAL

Florida

CERN

Caltech

Nor

dUG

rid,

Hel

sink

i May

23-

24, 2

002

Slide 14

Data Transfers Data Transfers

INFN

CERN

FNAL

Bristol/RALCaltech

Moscow

IN2P3

UFL

HIP

Wisconsin UCSDMin.Bias Objy/DB

.fz files

Objy/DB

RC archiving data

RC publishing data

Nor

dUG

rid,

Hel

sink

i May

23-

24, 2

002

Slide 15

CMS and the GRIDCMS and the GRID

CMS Grid Implementation plan for 2002 published (CMS NOTE-2002/015) Close collaboration with EDG and Griphyn/iVDGL,PPDG

Upcoming CMS GRID/Production Workshop (June CMSweek) File Transfers

Production File Transfer Software Experiences

Production File Transfer Hardware Status & Reports

Future Evolution of File Transfer Tools

Production Tools Monte Carlo Production System Architecture

Experiences with Tools

Monitoring / Deployment Planning Experiences with Grid Monitoring Tools

Towards a Rational System for Tool Deployment

Nor

dUG

rid,

Hel

sink

i May

23-

24, 2

002

Slide 16

CMS - Schedule for Challenge Ramp UpCMS - Schedule for Challenge Ramp Up

All CMS work to date with Objectivity,

Now being phased out to be replaced with LCG Software

Enforced lull in production challenges

No point to do work to optimize a solution being replaced

(But much learnt in past challenges to influence new design)

Use Challenge time in 2002 to benchmark current performance

Aim to start testing new system as it becomes available

Target early 2003 for first realistic tests

Thereafter return to roughly exponential complexity ramp up to reach 50%

complexity in 2005

20% Data Challenge

Nor

dUG

rid,

Hel

sink

i May

23-

24, 2

002

Slide 17

Objectivity IssuesObjectivity Issues

Bleak CERN has not renewed the Objectivity Maintenance

Old licenses are still applicable, but cannot be migrated to new hardware Our understanding is that we can continue to use the product as before,

clearly without support any longer But cannot be used on newer RedHat OS’s (7…) (or other Linux OS’s)

Will become increasingly difficult during this year to find sufficient resources correctly configured for our Objectivity usage.

We are preparing for the demise of our Objectivity-based code by the end of this year CMS already contributing to the new LCG Software Aiming to have first prototypes for catalog layer by July Initial release of CMS prototype ROOT+LCG, September

Nor

dUG

rid,

Hel

sink

i May

23-

24, 2

002

Slide 18

Planning - CMS ComputingPlanning - CMS Computing

2002: DAQ Technical Design Report 2003: GEANT4 validation, 5% Data Challenge start 2004 beg: 5% Data Challenge Complete 2004 end: Computing and Core Software (CCS) TDR submitted 2004-2005: Physics TDR 2005: 20% Data Challenge 2006 beg: 20% Data Challenge Complete 2006: CCS commissioning 2007 beg: fully operational computing systems (20% capacity) 2007-2008: CCS systems ramp-up 2009 beg: CCS systems 100% operational

Note: The new LHC schdule caused 9-15 months adjustements in CMS computing planning

Nor

dUG

rid,

Hel

sink

i May

23-

24, 2

002

Slide 19

SummarySummary

CMSIM/Geant3 (Fortran) to be replaced by OSCAR/Geant Then the full chain will be in C++ ODBS: Objectivity --> customized ROOT under work CMS simulation mass productions well under way

11 Regional Centres, >1000 CPU’s being used ~30 TB of data in 2001, 15 TB in 2002 so far ~8 M events in 2001 + ~25 M MinBias events for pile-up

Active participation in LCG 5% Data Challenge planned for beg of 2004 20% Data Challenge for 2006 From 9 to 15 months delays due to new LHC schedule