Effects of Flow Field on Diffusion Flame Structure - IWR: Home
M.Kunze, NEC2003, Varna The European CrossGrid Project Marcel Kunze Institute for Scientific...
-
date post
21-Dec-2015 -
Category
Documents
-
view
214 -
download
2
Transcript of M.Kunze, NEC2003, Varna The European CrossGrid Project Marcel Kunze Institute for Scientific...
M.Kunze, NEC2003, Varna
The European CrossGrid Project
Marcel Kunze
Institute for Scientific Computing (IWR)
Forschungszentrum Karlsruhe GmbH
www.eu-crossgrid.org
M.Kunze, NEC2003, Varna
CrossGrid Collaboration
Poland:Cyfronet & INP CracowPSNC PoznanICM & IPJ Warsaw
Portugal:LIP Lisbon
Spain:CSIC SantanderValencia & RedIrisUAB BarcelonaUSC Santiago & CESGA
Ireland:TCD Dublin
Italy:DATAMAT
Netherlands:UvA Amsterdam
Germany:FZK KarlsruheTUM MunichUSTU Stuttgart
Slovakia:II SAS Bratislava
Greece:AlgosystemsDemo AthensAuTh Thessaloniki
Cyprus:UCY Nikosia
Austria:U.Linz
21 21 institutinstitutees s
11 11 countriescountries
M.Kunze, NEC2003, Varna
GRIDLAB
GRIA
EGSO
DATATAG
DATAGRIDGRIP EUROGRID
DAMIEN
Science
IST Grid Project Space
Industry/Business Science
Applications
Middleware
Fabric
CROSSGRID
M.Kunze, NEC2003, Varna
Mission Statement
Development of a Grid Environment
for
Interactive Applications
Make the Grid more user friendly, secure and efficient
Timeline: 2002-2005
M.Kunze, NEC2003, Varna
Workpackages
WP1 – Application Development
WP2 – Grid Application Programming Environment
WP3 – New Grid Services and Tools
WP4 – International Testbed Organisation
WP5 – Project Management
M.Kunze, NEC2003, Varna
Applications
Middleware & Tools
Fabric (Infrastructure)
Architecture: Layered Structure
M.Kunze, NEC2003, Varna
WP1 –Application Development1. Interactive simulation and visualization
of a biomedical system2. Flooding crisis team support3. Distributed data analysis in HEP4. Weather forecast and air pollution
modeling
WP2 – Programming Environment
1. MPI code debugging and verification
2. Metrics and benchmarks 3. Interactive performance
evaluation toolsVisualization Kernel
Data Mining
WP3 – New Services
Globus Middleware
WP4 – Testbeds - Fabric Infrastructure
Grid Services:
DataGridGriPhyN
...
1. Portals and roaming access2. Grid resource management3. Grid monitoring4. Optimization of data access
High Level Architecture
Technical Components
M.Kunze, NEC2003, Varna
Detailed Architecture
Supporting Tools
1.4Meteo
Pollution
1.4Meteo
Pollution
3.1 Portal & Migrating Desktop
3.1 Portal & Migrating Desktop
ApplicationsDevelopment
Support
2.4Performance
Analysis
2.4Performance
Analysis
2.2 MPI Verification
2.2 MPI Verification
2.3 Metrics and Benchmarks
2.3 Metrics and Benchmarks
App. Spec Services
1.1 Grid Visualisation
Kernel
1.1 Grid Visualisation
Kernel
1.3 DataMining on Grid (NN)
1.3 DataMining on Grid (NN)
1.3 Interactive Distributed
Data Access
1.3 Interactive Distributed
Data Access
3.1Roaming Access
3.1Roaming Access
3.2Scheduling
Agents
3.2Scheduling
Agents
3.3Grid
Monitoring
3.3Grid
Monitoring
MPICH-GMPICH-G
Fabric
1.1, 1.2 HLA and others
1.1, 1.2 HLA and others
3.4Optimization of
Grid Data Access
3.4Optimization of
Grid Data Access
1.2Flooding
1.2Flooding
1.1BioMed
1.1BioMed
Applications
Generic Services
GRAMGRAM GSIGSIReplica CatalogReplica CatalogGIS / MDSGIS / MDSGridFTPGridFTP Globus-IOGlobus-IO
DataGridReplica
Manager
DataGridReplica
Manager
DataGrid Job Submission
Service
DataGrid Job Submission
Service
Resource Manager
(CE)
Resource Manager
(CE)
CPUCPU
ResourceManagerResourceManager
Resource Manager
(SE)
Resource Manager
(SE)Secondary
StorageSecondary
Storage
ResourceManagerResourceManager
Instruments ( Satelites,
Radars)
Instruments ( Satelites,
Radars)
3.4Optimization of
Local Data Access
3.4Optimization of
Local Data Access
Tertiary StorageTertiary Storage
Replica CatalogReplica Catalog
GlobusReplica
Manager
GlobusReplica
Manager
1.1User Interaction
Services
1.1User Interaction
Services
CrossGridDataGridGlobus
M.Kunze, NEC2003, Varna
Key Features of CrossGrid Applications
Data Data sources and data bases geographically
distributed
To be selected on demand
Processing Large processing capacity required; both HPC & HTC
Interactive
Presentation Complex data requires versatile 3D visualisation
Support for interaction and feedback to other components
M.Kunze, NEC2003, Varna
Biomedical Application
CT / MRI scan
MedicalDB
Segmentation
MedicalDB
LB flowsimulation
VEWDPC
PDA
Visualization
Interaction
HDB
10 simulations/day60 GB/simulation> 20 MB/s
M.Kunze, NEC2003, Varna
Interactive Treatment Planning
The vascular geometry can be modified using a library of models
Draw interactively Computational geometry
M.Kunze, NEC2003, Varna
Flood Simulation
Data sources
Meteorological simulations
Hydraulic simulations
Hydrological simulations
Users
Output visualization
M.Kunze, NEC2003, Varna
Distributed Data Analysis in HEP
Objectives Distributed data analysis
Distributed data mining techniques with neural networks
Issues Typical interactive requests will run on o(TB) distributed data
Transfer/replication times for the whole data about one hour
Data transfers once and in advance of the interactive session
Allocation, installation and set-up of corresponding database servers before the interactive session
Integration of user-friendly interactive access (based on PROOF)
M.Kunze, NEC2003, Varna
Parallel ROOT Facility: PROOF
Local
Remote
Selection
Parameters
Procedure
Proc.C
Proc.C
Proc.C
Proc.C
Proc.C
PROOF
CPU
CPU
CPU
CPU
CPU
CPU
TagDB
RDB
DB1
DB4
DB5
DB6
DB3
DB2
http://root.cern.ch
M.Kunze, NEC2003, Varna
Distributed/parallel codes on the Grid Coupled Ocean/Atmosphere Mesoscale Prediction System
STEM-II Air Pollution Code
Integration of distributed databases
Data mining applied to downscaling weather forecast
Weather Forecast and Air Pollution Modeling
M.Kunze, NEC2003, Varna
Migrating Desktop
Idea Save and resume a user grid session
Look and feel of a windows desktop
Implementation Roaming Access Server and Clients
Java Web Services (Portability)
Integration of Tools Job submission wizard
Job monitoring dialog
GridExplorer dialog
GridCommander dialog
M.Kunze, NEC2003, Varna
CrossGrid Testbeds
Example sites: Valencia GoG farm, Santander (GridWall), FZK
Various instances: Production, Validation, Test
M.Kunze, NEC2003, Varna
Testbed Monitoring
Mapcentergrid monitoring framework.
Mapcenter was developed by DataGrid and adapted to CrossGrid.
http://mapcenter.lip.pt
M.Kunze, NEC2003, Varna
Production Resource Broker Statistics
Total users 61
Submitted 2903
Accepted 2716
Matching 2554
Sent by JSS 2514
Jobs run 2355
Jobs done 2299
Graphics withRB statisticsavailable fromMapcenter
EDG 1.4
M.Kunze, NEC2003, Varna
Production RB Statistics (continued)
Most of the failures are related with: Authorization Matchmaking Testbed sites
Submitted 2903
Accepted 2716
Matching 2554
Sent by JSS 2514
Jobs run 2355
Jobs done 2299
187 Not accepted
162 Matching failures
40 Not submitted
159 Didn’t run
56 Didn’t reach the end
604 Jobs failedEDG 1.4
M.Kunze, NEC2003, Varna
1980s: Internet 1990s: Web 2000s: Grid
Where do we need to get to ?
Applications to support an “e-society” (“Cyber-Infrastructure”)
A Grid infrastructure which hides the complexities from the users (“Invisible Computing”)
A powerful and flexible network infrastructure (GEANT 2)
Where do we need to invest ?Applications targeted at realistic problems in “e-science”
Prototypes of Grid infrastructures
Maintain and improve the GEANT network
EU FP6 proposal: EGEEVision: Create European e-Infrastructure
70 Partners, 31.5 MEuro
Start: April 2004