ATLAS and GridPPATLAS and GridPP

GridPP Collaboration Meeting, Edinburgh, 5GridPP Collaboration Meeting, Edinburgh, 5thth November 2001November 2001

RWL Jones, Lancaster UniversityRWL Jones, Lancaster University

RWL Jones, Lancaster University

ATLAS NeedsATLAS Needs

Long term, ATLAS needs a fully Grid-enabled Long term, ATLAS needs a fully Grid-enabled Reconstruction, Analysis and Simulation environmentReconstruction, Analysis and Simulation environment

Short-term, the first ATLAS priority is a Monte Carlo Short-term, the first ATLAS priority is a Monte Carlo production system, building towards the full systemproduction system, building towards the full system

ATLAS has an agreed program of Data Challenges ATLAS has an agreed program of Data Challenges (based in MC data) to develop and test the computing (based in MC data) to develop and test the computing modelmodel

RWL Jones, Lancaster University

Data Challenge 0Data Challenge 0

Runs from October-December 2001 Runs from October-December 2001 Continuity test of MC code chain. Continuity test of MC code chain. Only modest samples 10Only modest samples 1055 event samples, and event samples, and

essentially all in flat file format. essentially all in flat file format. All the Data Challenges will be run on Linux systems All the Data Challenges will be run on Linux systems compilers distributed with the code if not already compilers distributed with the code if not already

installed locally in the correct version.installed locally in the correct version.

RWL Jones, Lancaster University

Data Challenge 1Data Challenge 1

Runs in the first half of 2002 Runs in the first half of 2002 Several sets of 10Several sets of 1077 events (high level trigger studies, events (high level trigger studies,

physics analysis).physics analysis). Intend to generate and store 8Tbytes in the UK, Intend to generate and store 8Tbytes in the UK, 1-2Tbytes in Objectivity. 1-2Tbytes in Objectivity. Will use of M9 DataGrid deliverables and as many Will use of M9 DataGrid deliverables and as many

other Grid tools as time permits.other Grid tools as time permits. Tests of distributed reconstruction and analysisTests of distributed reconstruction and analysis Test of database technologiesTest of database technologies

RWL Jones, Lancaster University

Data Challenge 2Data Challenge 2

Runs for the first half of 2003Runs for the first half of 2003 Will generate several samples of 10Will generate several samples of 1088 events events Mainly in OO-databasesMainly in OO-databases Full use of the Testbed 1 and Grid toolsFull use of the Testbed 1 and Grid tools Complexity and scalability tests of the distributed Complexity and scalability tests of the distributed

computing systemcomputing system Large-scale distributed physics analysis using Grid Large-scale distributed physics analysis using Grid

tools, calibration and alignement tools, calibration and alignement 

RWL Jones, Lancaster University

LHC Computing Model (Cloud)LHC Computing Model (Cloud)

CERNTier2

Lab a

Uni a

Lab c

Uni n

Lab m

Lab b

Uni bUni y

Uni x

PhysicsDepartment

Desktop

Germany

Tier 1

USAFermiLab

UK

France

Italy

NL

USABrookhaven

……….

The LHC Computing

Centre

RWL Jones, Lancaster University

Implications of Cloud ModelImplications of Cloud Model

Internal: need cost sharing between global regions Internal: need cost sharing between global regions within collaborationwithin collaboration

External (on Grid services): Need External (on Grid services): Need authentication/accounting/priority on the basis of authentication/accounting/priority on the basis of experiment/region/team/local region/userexperiment/region/team/local region/user

Note: The NW believes this is a good model for tier-2 Note: The NW believes this is a good model for tier-2 resources as well.resources as well.

RWL Jones, Lancaster University

ATLAS SoftwareATLAS Software

Late in moving to OO as physics TDR etc given a high Late in moving to OO as physics TDR etc given a high prioritypriority

Generations and reconstruction now done in C++/OO Generations and reconstruction now done in C++/OO Athena frameworkAthena framework

Detector simulation still in transition to Detector simulation still in transition to OO/C++/Geant4; DC1 will still use G3OO/C++/Geant4; DC1 will still use G3

Athena common framework with LHCb GaudiAthena common framework with LHCb Gaudi

RWL Jones, Lancaster University

Simulation software for DC1.Simulation software for DC1.

HepMc

Detector simulation

Dice: slug+geant3 fortranproduce GENZ+KINE bank

ZEBRA

ATHENA

Fast det.simulation

ReconstructionC++reads GENZ +kineconvert to HepMcproduce Ntuples

ATHENA

Particle lev. simulation

GeneratorModulesC++, linux----------------Py6+code dedicated toB-physics----------------PYJETS->HepMc--------------- EvtGen BaBar package ( later).

Atlfast++reads HepMcproduce

Ntuples

RWL Jones, Lancaster University

Requirement CaptureRequirement Capture

Extensive use case studies:“ATLAS Grid Use Cases Extensive use case studies:“ATLAS Grid Use Cases and Requirements” 15/X/01and Requirements” 15/X/01

Many more could be developed, especially in the Many more could be developed, especially in the monitoring areasmonitoring areas

Short-term use case centred on immediate MC Short-term use case centred on immediate MC production needsproduction needs

Obvious overlaps with LHCb – joint projectsObvious overlaps with LHCb – joint projects Three main projects defined, “Three main projects defined, “Proposed ATLAS UK Proposed ATLAS UK

Grid Projects” 26/X/01Grid Projects” 26/X/01

RWL Jones, Lancaster University

Grid User interface for AthenaGrid User interface for Athena

Completely common project with LHCbCompletely common project with LHCb Obtains resource estimates and applies quota and

security policies Query installation tools

Correct software installed? Install if not Job submission guided by resource broker Run-time monitoring and job deletion Output to MSS and bookkeeping update

RWL Jones, Lancaster University

Installation ToolsInstallation Tools

Tools to automatically generate installation kits, Tools to automatically generate installation kits, deploy using Grid tools and install at remote sites via deploy using Grid tools and install at remote sites via Grid jobGrid job

Should be integrated with a remote autodetection Should be integrated with a remote autodetection service for installed softwareservice for installed software

Initial versions should cope with pre-built libraries Initial versions should cope with pre-built libraries and executablesand executables

Should later deploy development environmentShould later deploy development environment ATLAS and LHCb build environments converging on ATLAS and LHCb build environments converging on

CMT – some commonality hereCMT – some commonality here

RWL Jones, Lancaster University

MC Production SystemMC Production System

For DC1, will use existing MC production system (G3), For DC1, will use existing MC production system (G3), integrated with M9 toolsintegrated with M9 tools

(Aside: M9/WP8 validation and DC kit development in (Aside: M9/WP8 validation and DC kit development in parallel)parallel)

Decomposition of MC system into components: Decomposition of MC system into components: Monte Carlo Monte Carlo job submissionjob submission, bookkeeping services, metadata catalogue , bookkeeping services, metadata catalogue services, monitoring and quality-control toolsservices, monitoring and quality-control tools

Bookkeeping and data-management projects already Bookkeeping and data-management projects already ongoing – will work in close collaboration, good link with ongoing – will work in close collaboration, good link with US projectsUS projects

Close link with Ganga developmentsClose link with Ganga developments

RWL Jones, Lancaster University

Allow regional management of large productionsAllow regional management of large productions Job script and steering generatedJob script and steering generated Remote installation as requiredRemote installation as required Production site chosen by resource broker.Production site chosen by resource broker. Generate events and store locallyGenerate events and store locally Write log to webWrite log to web Copy data to local/regional store through interface with Copy data to local/regional store through interface with

Magda (data management).Magda (data management). Copy data from local storage to remote MSSCopy data from local storage to remote MSS Update book-keeping databaseUpdate book-keeping database

RWL Jones, Lancaster University

Work Area PMB Allocation

(FTE)

Previously Allocated

(FTE)

Total Allocation

(FTE)

ATLAS/LHCb 2.0 0.0 2.0

ATLAS 1.0 1.5 2.5

LHCb 1.0 1.0 2.0

This will just allow us to cover the three projectsAdditional manpower must be found for monitoring tasks, testing the computing model in DC2, and the simple running of the Data Challenges

RWL Jones, Lancaster University

WP8 M9 Validation WP8 M9 Validation

WP8 M9 Validation now beginningWP8 M9 Validation now beginning GlasgowGlasgow, Lancaster(, RAL?) involved in the ATLAS , Lancaster(, RAL?) involved in the ATLAS

M9 validationM9 validation Validation is exercises the tools using the ATLAS kitValidation is exercises the tools using the ATLAS kit

The software used is behind the current version This is likely to be the case in all future tests

(decouples software changes from tool tests)

Previous test of MC production using Grid tools a Previous test of MC production using Grid tools a successsuccess

DC1 validation (essentially of ATLAS code); Glasgow, DC1 validation (essentially of ATLAS code); Glasgow, Lancaster (Lancaster is working on tests of standard Lancaster (Lancaster is working on tests of standard generation and reconstruction quantities to be generation and reconstruction quantities to be deployed as part of kit) Cambridge to contributedeployed as part of kit) Cambridge to contribute