Post on 31-Mar-2015
Large-Scale Distributed Computing in the Netherlands
Dutc hG rid
a status overview
What is DutchGrid?
• collaboration of 12 organisations• established in 2000• functions as a platform,
getting manpower & hardware from partner orgs
• major contributors– WTCW Virtual Laboratory Project (ICES/KIS-II)– ASCI (5 combined Dutch CS Faculties)– Foundation NCF (NWO)– NIKHEF
DutchGrid Sites
KNMI
Nijmegen
Delft
Leiden
Amsterdam
VU
WCW
Utrecht
Telin
NIKHEFSARAUvAAMOLFCWI
DutchGrid site map 20021104David Groep, NIKHEF
ASTRON, JIVE
WCW Amsterdam
Resources
• ASCI DAS-2 (VU, UvA, Leiden, TUDelft, Utrecht)– 200 dual P-III 1GHz CPUs– homogeneous clusters, 5 locations
• NIKHEF DataGrid clusters– 75 dual P-III ~ 1GHz – 1Gb/s IPv4 + 1Gb/s IPv6
• NCF Gridnational computer facilities foundation from NWO
– 66 node dual AMD-K7 Fabric Research Cluster (NIKHEF)– 32 node duals “production quality” cluster (SARA)*– 10Gb/s optical “lambda” test bed†
– …
• BioASP – various smaller O(10 node) clusters
Resources (cont.)
SARA – National HPC Centre• Processing
– SGI 1024 processor MPP
• Mass storage– StorageTek NearLine tape robot– currently: 200 TByte– will grow “indefinitely” (with funds from NCF)
• User expertise centre
In Total
Operations/deployment effort ~ 8 FTE available
Research effort O (40 FTE) over 12 organisations
CR1-1 CR1-2
CR2-1 CR2-2BR2
AR5
BR1AMS
10GPOS
2.5GPOS
BR1CHI
622M PO Prot.Tyco/IEEAF
AMSTERDAM
2x2.5Gb
622Mb L2 VPN
2xGbE622Mb “rest”Global Xing
622Mb to “rest”
NY
CR1-1 CR1-2
CR2-1 CR2-2BR2
AR5
BR1AMS
10GPOS
2.5GPOS
BR1CHI
622M PO Prot.Tyco/IEEAF
AMSTERDAM
622Mb L2 VPN
NY
CR1-1 CR1-2
CR2-1 CR2-2BR2
AR5
BR1AMS
10GPOS
2.5GPOS
622M PO Prot.Tyco/IEEAF
622Mb L2 VPN
NY
Networks
IGRID6509
BR4
SP01
OC48cSR
StarLight6509
ON
S15
454
ON
S
StarLightT640
ABILENE NY
ABILENE NY
ARGONNE IL
ARGONNE IL
SingARENTANet2
…
SingARENTANet2
…
StarLightRTR
STARTAP
GbE
GbE
CERNRTR
ONS
10G POS Tyco
10G POS Level3
2xGbE
10GbE
2xGbE
2.5Gb lambda
OC
48c
lam
bda
7xGbE
GbE
GbE’s
CHICAGO, IL
CERN,CH
18xGbEUvANIKHEF“NCSA”“ANL”
DutchGrid Services
• Authentication Services (Certification Authority)– EDG CACG “proper” CA– Student and educational CA– EDG “Tutorial” CA
• Regular coordination and info meetings• Installation support
– Have been distributing EDG 1.2 till now
• User support
Initiatives inside NL
NCFGridstrengthening Grid Infrastructure in NL
• 4 partners (NIKHEF, ASCI, SARA, ASTRON)• 600 kEUR funding• hardware (3 clusters, 10Gb/s networking)
and people (1.5 FTE over 3 years)
• approved August 2002• fabric research + production-quality grids• new apps: bio informatics, astronomy
First cluster assembled and being tested…
Initiatives inside NL
Virtual Laboratory for e-ScienceDutch “knowledge infrastructure” project
• 14 partners• 6 application domains• extensive middleware
research programme• test bed
HPC SecAAA Networks
IntVisual.
Inf. Mngt
Intg ExpEnvs
UIs
D.I.Sc. FoodInf BioInfLargeEquip
Biodiversity
PersonalCare
Test BedScaling
&Validation
in project planning stage withestimated O (25 MEUR) funding
Some Apps
• WTCW Virtual Laboratoryvisual analysis environment for (small-scale) experimental science– Chemo-physical surface analysis (PIXE,FT-IR,SIMS)– BioASP – correlating “omics” databases– Virtual Surgery (by-pass surgery, artery defects)
• DØ – Monte-Carlo and Analysis• EcoGrid
correlating bio-diversity information
• Radio Astronomy and VLBI (JIVE)LOFAR, AVO
Research Areas in NL
• Generic AAA and `security’UvA, NIKHEF
• Adaptive programming environmentsFree University Amsterdam
• Memory-processor co-allocation TU Delft
• High-bandwidth networkingSURFnet, UvA, SARA, NIKHEF
• Federated Information & Content ManagementUvA
• Interactive visualisationCWI, UvA
• Cluster, Storage Management & MonitoringNIKHEF, SARA
Summary
• DutchGrid Platform started informally in 2000• De-facto centre for Grid computing in NL• Tasks supported by partners: CA, web, …
• Mix of people from all layers of the ISO stacknetworks, AAA, programming envs, semantic DB, apps
Plenty of opportunities for collaboration!