Slide 1 Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 GridPP Steve Lloyd Queen Mary, University of...
-
Upload
chase-ortiz -
Category
Documents
-
view
215 -
download
0
Transcript of Slide 1 Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 GridPP Steve Lloyd Queen Mary, University of...
Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 1
GridPP
Steve LloydQueen Mary, University of LondonGrid Brokering Meeting December
2006
Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 2
19 UK Universities, CERN and
CCLRC (RAL & Daresbury) Funded by PPARC:
GridPP1 2001-2004 “From Web to Grid”
GridPP2 2004-2007 “From Prototype to Production”
GridPP3 2007-2011 (proposed)“From Production to
Exploitation”
Who are GridPP?
Developed a working, highly functional Grid
Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 3
Why? - The CERN Large Hadron Collider - LHC
4 Large Experiments
The world’s most powerful particle accelerator – Starting 2007/8
• ~100,000,000 electronic channels
• 800,000,000 proton-proton interactions per second
• 0.0002 Higgs per second• 10 PBytes of data a year • (10 Million GBytes = 14 Million
CDs)
Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 4
International Context
LHC Computing Grid (LCG)Grid Deployment Project for
the Large Hadron Collider (LHC)
EU Enabling Grids for E-SciencE (EGEE) 2004-2008Grid Deployment Project for all disciplines
GridPP LCG
EGEE
GridPP is part of EGEE and LCG (currently the largest Grid in the world)
UK National Grid ServiceUK’s core production computational and data Grid
Open Science Grid (USA)Science applications from HEP to biochemistry
Nordugrid (Scandinavia)Grid Research and Development collaboration
UK part of LCG
PP part of EGEE
Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 5
What is (gLite) Middleware?
MIDDLEWARE
CPUDisks, CPU etc
PROGRAMS
OPERATING SYSTEM
Word/Excel
Email/Web
Your Progra
mGames
CPUCluste
r
UserInterfac
eMachine
CPUCluste
r
CPUCluste
r
Resource Broker
Information Service
Single PC
Grid
DiskServer
Your Progra
m
Middleware is the Operating System of a distributed computing system
Replica CatalogueBookkeepin
g Service
Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 6
GridPP Middleware Development
Workload Management
Storage Interfaces
Network Monitoring
SecurityInformation Services
Grid Data Management
Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 7
The EGEE Grid Status Worldwide
207 Sites
48 Countries
32,823 CPUs
13.1 PB Disk
6,946 Years of CPU time
UK
21 Sites
6306 CPUs
131 TB Disk
1,996 Years of CPU time
Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 8
UK Tier-1 and Tier-2 SitesMostly funded by HEFCE
Tier-1
CPU (KSI2K) Disk (TB)Brunel 116 1.4Imperial 368 8.4QMUL 1209 18.0RHUL 163 8.8UCL 121 1.1Lancaster 484 72.0Liverpool 592 2.8Manchester 1152 145.0Sheffield 183 2.0Durham 99 4.0Edinburgh 11 60.0Glasgow 47 1.6Birmingham 223 9.3Bristol 12 3.8Cambridge 40 4.4Oxford 150 27.0RAL PPD 320 6.4London 1977 37.7NorthGrid 2411 221.8ScotGrid 157 65.6SouthGrid 745 50.9Total Tier-2 5290 376.0Tier-1 1034 180.0
Normally most recent gLite versions.
Still running the LCG RB and CE (not gLite).
SEs being upgraded to SRM 2.2.
Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 9
What you need to use the Grid
1. Get a digital certificate (UK Certificate Authority)
2. Join a Virtual Organisation (VO)
3. Get access to a local User Interface Machine (UI) and copy your files and certificate there
Authentication – who you are
Authorisation – what you are allowed to do
4. Write some Job Description Language (JDL) and scripts to wrap your programs
############# HelloWorld.jdl #################Executable = "/bin/echo";Arguments = "Hello welcome to the Grid ";StdOutput = "hello.out";StdError = "hello.err";OutputSandbox = {"hello.out","hello.err"};#########################################
Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 10
How it works
Job Descriptio
n
UI Machine
Resource Broker
Input SandboxScript you
want to run
Other files (Job
Options, Source...)
Storage Element
Compute Element
Storage Element
Grid
Proxy Certificate
Output SandboxOutput
files (Plots, Logs...)
Input Data
Output Data
Job
Output Sandbo
x
Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 11
What is it good for?
Problems that are highly parallelizable
Problem
Grid
Solution
Input data is independent e.g.
Images:
A=2B=3 A=3
B=3 A=2B=4
Simulation using different
parameters:
Not so good for closely coupled
problems
These pieces may be independent
These pieces will have to interact
Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 12
GridPP OutreachCurrent GridPP Outreach Team:
0.5 FTE Dissemination Officer
1.0 FTE Events Officer
Proposed GridPP Outreach Team (GridPP3):
1.0 FTE Dissemination Officer0.5 FTE Industrial Liaison Officer0.5 FTE (User) Documentation
GridPP WebsiteEvents and DemonstrationsPress Releases and News ItemsDissemination Awards (e.g. Schools)Posters, Brochures, Papers
Expand Industrial LiaisonTechnology Transfer Steering GroupImproved User DocumentationTraining e.g. Use of Portal
Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 13
GridPP and Industry
• Use of our Grid• Access to the Certificate
Authority• Security tools• GridSite Secure Web Toolkit• R-GMA Information System• APEL Accounting tools• Help getting PPARC funding
What We Have to Offer:
Current Involvement:
• GridPP has a collaboration with IBM through ScotGrid and R-GMA.
• Specific sites also have close relationships with various industrial suppliers.
• One PPARC MiniPPIPS with a startup company.
Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 14
Cost Recovery Survey GridPP attitudes to allowing commercial access to
resourcesBirmingham Keen to share if not too time consuming
CambridgeConcerned about organisational issues but happy for limited access
Glasgow Will allow access for limited time
Oxford Concerned about organisational issues
Queen MaryPrefer to offer full technical and research support rather than ‘dumb’ access
Royal Holloway Concerned about diversion of effort
Generally positive as long as not too time consuming and (University) bureaucracy can be overcome
Steve Lloyd Grid Brokering Meeting - 4 Dec 2006 Slide 15
Further Information …
http://www.gridpp.ac.uk
RSS News feed