GridPP Building a UK Computing Grid for Particle Physics Professor Steve Lloyd, Queen Mary,...

GridPPBuilding a UK Computing Grid for Particle Physics

Professor Steve Lloyd, Queen Mary, University of London

Chair of the GridPP Collaboration Board

Steve Lloyd Public Service Summit - 22 September 2004 Slide 2

Outline

• Why? – The CERN LHC and the Data Deluge• What? - GridPP and the Grid

– What is the Grid?– Applications and Middleware– Tier-1 and Tier-2 Regional Centres

• How? - GridPP Management• Summary and Challenges

What is GridPP?

19 UK Universities, CCLRC (RAL & Daresbury) and CERN

Funded by the Particle Physics and Astronomy Research Council (PPARC)

GridPP1 - 2001-2004 £17m "From Web to Grid"

GridPP2 - 2004-2007 £15m "From Prototype to Production"

The CERN LHC

4 Large Experiments

The world’s most powerful particle accelerator - 2007

LHC Experiments

• > 108 electronic channels• 8x108 proton-proton collisions/sec• 2x10-4 Higgs per sec• 10 Petabytes of data a year • (10 Million GBytes = 14 Million CDs)

Searching for the Higgs Particle and exciting new Physics

Starting from this event

Looking for this ‘signature’

e.g. ATLAS

What is the Grid?

MIDDLEWARE

CPUDisks, CPU etc

PROGRAMS

OPERATING SYSTEM

Word/Excel

Email/Web

Your Progra

mGames

CPUCluste

UserInterfac

eMachine

CPUCluste

Resource Broker

Information Service

Single PC

DiskCluste

Your Progra

Middleware is the Operating System of a distributed computing system

What is the Grid?

From this:

To this:

International Collaboration

• EU DataGrid (EDG) 2001-2004– Middleware Development

Project

• US and other Grid projects– Interoperability

• LHC Computing Grid (LCG)– Grid Deployment Project for

LHC • EU Enabling Grids for e-Science

in Europe (EGEE) 2004-2006– Grid Deployment Project for all

disciplines

The LCG Grid

Grid Snapshot

GridPP1 Areas

6/Feb/2004

£3.57m

£5.67m

£3.74m

£2.08m£1.84m

DataGrid

Tier - 1/A

ApplicationsOperations

LHC Computing Grid Project (LCG)Applications, Fabrics, Technology and Deployment

European DataGrid (EDG)Middleware Development

UK Tier-1/A Regional CentreHardware and Manpower

Grid Application DevelopmentLHC and US Experiments + Lattice QCD

Management Travel etc

GridPP2 Areas

July 2004

£0.75m

£2.71m

£2.88m

£0.87m

£0.69m

£2.71m

£2.89m

£1.00m

£2.40m

Tier-1/AHardware

Tier-2Operations

Applications

MgrTravel

Tier-1/AOperations

Management Travel etc

UK Tier-1/A Regional Centre Hardware

UK Tier-2 Regional Centres Manpower

UK Tier-1/A Manpower

LHC Computing Grid Project (LCG)Manpower

Middleware, Security and Networking Manpower

Grid Application Development LHC and US Experiments + Lattice QCD, Phenomenology and Generic Portal

Application Development

Fabric

TapeStorage

Elements

RequestFormulator and

Planner

Client Applications

ComputeElements

Indicates component that w ill be replaced

DiskStorage

Elements

LANs andWANs

Resource andServices Catalog

ReplicaCatalog

Meta-dataCatalog

Authentication and SecurityGSISAM-specific user, group , node, st at ion regis tration B bftp ‘cookie’

Connectivity and Resource

CORBA UDP File transfer protocol s - ftp, b bftp, rcp GridFTP

Mass Storage s ystems protocol se.g. encp, hp ss

Collective Services

C atalogproto co ls

Signi fi cant Event Log ger Naming Service Database ManagerC atalog Manager

SAM R es ource M an ag em entB atch Sys tems - LSF, FB S, PB S,

C ondorData Mov erJob Services

Storage ManagerJob ManagerCache ManagerRequest Manager

“Dataset Editor” “File Storage Server”“Project Master” “Station M aster” “Station M aster”

Web Python codes, Java codesCom mand line D0 Fram ework C++ codes

“Stager”“Optim iser”

CodeRepostory

Name in “quotes” is SAM-given software component name

or addedenhanced using PPDG and Grid tools

SAMGridLattice QCD

AliEn → ARDA

Middleware Development

Configuration Management

Storage Interfaces

Network Monitoring

Security

Information Services

Grid Data Management

UK Tier-1/A Centre

• High quality data services• National and International

Role• UK focus for International

Grid development

•700 Dual CPU•80 TB Disk•60 TB Tape (Capacity 1PB)

Grid Operations Centre

UK Tier-2 Centres

ScotGridDurham, Edinburgh, Glasgow NorthGridDaresbury, Lancaster, Liverpool,Manchester, Sheffield

SouthGridBirmingham, Bristol, Cambridge,Oxford, RAL PPD, Warwick

LondonGridBrunel, Imperial, QMUL, RHUL, UCL

Mostly funded by HEFCE

UK Core e-Science

Programme

Institutes

Tier-2 Centres

CERNLCG

GridPP

GridPP in Context

Tier-1/A

Middleware, Security,

Networking

Experiments

GridSupportCentre

Not to scale!

Apps Dev

AppsInt

GridPP

Management

Collaboration Board

Project ManagementBoard

Project Leader

Project Manager

DeploymentBoard

UserBoard

Production Manager

Dissemination Officer

EGEE Leader

Tier-1 Board

Project Map

Risk Register

Tier-2 Board

CERN LCG Liaison

Deployment Team

Summary

19 UK Institutes

RAL Computer Centre

CERN ComputerCentre

SAMGrid

BaBarGrid

EDGGANGA

UK PrototypeTier-1/A Centre

CERN PrototypeTier-0 Centre

4 UK Tier-2 Centres

UK Tier-1/ACentre

CERN Tier-0Centre

200720042001

4 UK Prototype Tier-2 Centres

Separate Experiments,

Resources, Multiple Accounts

'One' Production GridPrototype Grids

Challenges

Concorde(15 km)

CD stack with1 year LHC data(~ 20 km)

We are here(1 km)

• Scaling to full size ~10,000 → 100,000 CPUs

• Stability, Robustness etc• Security• Sharing resources (in RAE

environment!)• International Collaboration• Continued funding beyond

start of LHC!

Further Info

http://www.gridpp.ac.uk

GridPP Building a UK Computing Grid for Particle Physics Professor Steve Lloyd, Queen Mary,...

Documents

Transcript of GridPP Building a UK Computing Grid for Particle Physics Professor Steve Lloyd, Queen Mary,...

Grid Deployment & Operations: EGEE, LCG and GridPP Jeremy Coles GridPP Production Manager UK&I Operations Manager for EGEE J.Coles@rl.ac.uk 20 th September.

GridPP Deployment & Operations GridPP has built a Computing Grid of more than 5,000 CPUs, with equipment based at many of the particle physics centres.

Tony Doyle - University of Glasgow 10 January 2005Oversight Committee GridPP: status report Andy Parker (Taken from GridPP report at OSC by Tony Doyle)

Tony Doyle - University of Glasgow 6 September 2005Collaboration Meeting GridPP Overview (emphasis on beyond GridPP) Tony Doyle.

GridPP Deployment Status

Slide David Britton, University of Glasgow IET, Oct 09 1 Prof. David Britton GridPP Project leader University of Glasgow GridPP Oversight Committee Meeting.

S.L.LloydGrid Steering Committee 8 March 2002 Slide 1 Status of GridPP Overview Financial Summary Recruitment Status EU DataGrid UK Grid Status GridPP.

Slide David Britton, University of Glasgow IET, Oct 09 1 Prof. David Britton GridPP Project leader University of Glasgow GridPP delivering The UK Grid.

GridPP Presentation to PPARC e-Science Committee 26 July 2001 Steve Lloyd Tony Doyle John Gordon.

GridPP From Prototype to Production David Britton 21/Sep/06 1.Context – Introduction to GridPP 2.Performance of the GridPP/EGEE/wLCG Grid 3.Some Successes.

GridPP, The Grid & Industry Who we are, what it is and what we can do. Tony Doyle, Project Leader Steve Lloyd, Collaboration Board Chairman Robin Middleton,

Prof. David Britton GridPP Project leader University of Glasgow

GridPP: the UK's contribution to the international collaboration building a worldwide Grid, the LHC Computing Grid GridPP – is the system usable? Tony.

Financing the Kingsdon Chronicle · Lloyd and Sarah’s Wedding The wedding took place on 25th July at Queen Camel Church of Lloyd Attwell and Sarah Messenger. The best man was Simon

CMS Report – GridPP Collaboration Meeting IX Peter Hobson, Brunel University4/2/2004 CMS Status Progress towards GridPP milestones Data management – the.

10.517 cbm - LPG/E Queen Isabella - harpaingas · 10.517 cbm - LPG/E Queen Isabella Name Queen Isabella Flag Liberia Port of registry Monrovia IMO No. 8908569 Class Germanischer Lloyd

The Grid Professor Steve Lloyd Queen Mary, University of London.

Steve LloydGridPP13 Durham July 2005 Slide 1 Using the Grid Steve Lloyd Queen Mary, University of London.

Slide 1 Steve Lloyd London Tier-2 Workshop - 16 Apr 2007 Introduction to Grids and GridPP Steve Lloyd Queen Mary, University of London London Tier-2 Workshop.

Janet Update GridPP Collaboration meeting Sept 2012 David Salmon.