INFSO-RI-508833 Enabling Grids for E-sciencE FloodGrid application Ladislav Hluchy, Viet D. Tran...
-
Upload
vincent-townsend -
Category
Documents
-
view
216 -
download
0
Transcript of INFSO-RI-508833 Enabling Grids for E-sciencE FloodGrid application Ladislav Hluchy, Viet D. Tran...
INFSO-RI-508833
Enabling Grids for E-sciencE
www.eu-egee.org
FloodGrid application
Ladislav Hluchy, Viet D. Tran
Institute of Informatics, SAS
Slovakia
2
Enabling Grids for E-sciencE
INFSO-RI-508833
History of Flood application
• Flood application is continually developed in – ANFAS: datA fusioN for Flood Analysis and decision Support,(2000-03) IST-
1999-11676 Data fusion, hydraulic modeling Cluster computing Remote processing
– CrossGrid: Development of Grid Environment for Interactive Applications (2002-05) IST-2001-32243
More models (meteorology, hydrology) Grid computing Metadata catalog Portal
– EGEE: Enabling Grids for E-sciencE (2004-2006) INFSO-RI-508833 Porting to gLite Working in Earth Science Research Virtual Organization (ESR VO)
– K-Wf Grid: Knowledge-based Workflow System for Grid Applications (2004-2006) FP6-511385
Porting to web services Building knowledge system for flood application
• Collaboration with Slovak Hydro-meteorological Institute (SHMI) and Slovak Water Research Institute (WRI)
3
Enabling Grids for E-sciencE
INFSO-RI-508833
Flood forecasting problem
Data sources
Hydrological simulation
Hydraulic simulation
Portal
Meteorological simulation
Data sources
Hydrological simulation
Hydraulic simulation
Portal
Meteorological simulation
4
Enabling Grids for E-sciencE
INFSO-RI-508833
Flood forecasting problem
• Many types of simulations– Meteorological models: MM5, ALADIN– Hydrological models: HSPF, NLC, HEC, MIKE-1D, …– Hydraulic models: FESWMS, DaveF, MIKE-2D, TELEMAC, …
• Many types of data– Real-time meteorological and hydrological data from hydro-
meteorological stations– Boundary conditions from ALADIN-LACE (Local Area Central
European) model from Prague– Digital elevation models measured using LIDAR (Light Detection and
Ranging)– River profiles and networks– Input and output data of different simulations
• Many types of users– Experts in meteorology, hydrology and hydraulics– River authorities, water management authorities– Hydropower stations, farms– Governments– Public
5
Enabling Grids for E-sciencE
INFSO-RI-508833
Flood forecasting and Grid
• For critical situations– Needs to run many scenarios at the same time (worst-case
scenarios for risk analysis)– Needs to have results as accurate as possible (higher
resolutions for simulations -> higher computational power)– Needs to have results as soon as possible (every minute is
important)
Needs grid computational power• For collaborations:
– Different user groups: meteorology, hydrology, hydraulics, river authorities, crisis team, …
– Different countries: for international rivers like Danube– Sharing resources: data, computational powers, expertise, …
Grid is the technology for collaborations
6
Enabling Grids for E-sciencE
INFSO-RI-508833
Simplified scenario
User PortalWorkflow
management
Workflow
Storageelement
Meteorology simulation
Hydrology simulation
Hydraulics simulation
Meteorologyvisualization
Hydrologyvisualization
Hydraulicsvisualization
OPERATIONS IN GRID
7
Enabling Grids for E-sciencE
INFSO-RI-508833
Full scenario
8
Enabling Grids for E-sciencE
INFSO-RI-508833
FloodGrid portal
• Portal is accessible from anywhere:– Needs only network connection and web browsers– Users can use portal also from mobile devices (very important
for field work)
• Portal is the “meeting point” for users:– Users need to collaborate on flood forecasting
• Portal hides complexity of Grid computing:– The main users of the portal may be experts in meteorology,
hydrology, … but not in Grid computing
• Portal increases the security– Some data and information from flood forecasting are sensitive– Portal has additional security level– Users cannot run other code than the portal allows
9
Enabling Grids for E-sciencE
INFSO-RI-508833
FloodGrid Portal
Workflow management
Data management
Visualization Collaboration
Portal
Models
ALADIN HSPF
DaveF
Data Computationalresources
Workflow management
Data management
Visualization Collaboration
Portal
Models
ALADIN HSPF
DaveF
DataData Computationalresources
Computationalresources
GRID
10
Enabling Grids for E-sciencE
INFSO-RI-508833
Workflow management
• Manages and executes jobs with data dependences• Cooperates with gLite resource broker to find suitable
computing element for running simulation• Monitors status of jobs• Abilities to use predefined workflow templates,
spawning running workflow, modifying parameters of jobs
11
Enabling Grids for E-sciencE
INFSO-RI-508833
Workflow management module
Workflow template database
Workflow execution service
CrossGrid JSS
Workflow selection portlet
Workflow definition portlet
LCG gLite Web service
Job execution adaptors
Portal
Workflow manager
EGEE testbed
Meteorology Hydrology Hydraulic Visualization
12
Enabling Grids for E-sciencE
INFSO-RI-508833
Workflow management
13
Enabling Grids for E-sciencE
INFSO-RI-508833
Data management
• Many kinds of data in FloodGrid– Meteorological, hydrological, hydraulic– Generated by simulations or obtained from sensors– Permanent or periodically updated– Publicly available or with restricted access
• Using metadata catalog for describing data• Data are stored in storage elements and are accessed
via Grid protocols• Operation: query, adding, modification, deleting
14
Enabling Grids for E-sciencE
INFSO-RI-508833
Data management module
Metadata catalog
Replica Location Service
Metadata query Data access
GridFTP
Portal
Metadata service
Replica 1 Replica 2 Replica 3 Replica 4
Data identifier
(GUID, LFN)
Grid testbed
15
Enabling Grids for E-sciencE
INFSO-RI-508833
Visualization
• Multiple visualization modes according to models and visualization tools– Texts– Pictures– Animations– Virtual reality
16
Enabling Grids for E-sciencE
INFSO-RI-508833
Visualization
17
Enabling Grids for E-sciencE
INFSO-RI-508833
Visualization in Virtual Reality
• In cooperation with GUP, Joh. Kepler University, Linz
18
Enabling Grids for E-sciencE
INFSO-RI-508833
Collaboration
• Different users groups (experts in meteorology, hydrology, hydraulics, crisis team, river authorities) need to collaborate on flood application
• Portal provides different means of communication among users: chats, mailing lists, discussion groups, file sharing
• Collaboration via other shared tools of portal
19
Enabling Grids for E-sciencE
INFSO-RI-508833
Collaboration
20
Enabling Grids for E-sciencE
INFSO-RI-508833
Security
• Two level of securities: Portal level and Grid level• Portal level:
– Secure HTTP access– Authentication by user names/passwords– Access controls for every tools/resources at portal (individually or
by groups)– Only for accessing tools/resources located on portal. For accessing
physical data, models or computing resource, Grid certificates are needed
• Grid level:– Authentication by Grid certificates– Using Grid communication protocols
21
Enabling Grids for E-sciencE
INFSO-RI-508833
Workflow management
Data management
Visualization Collaboration
Portal
Models
ALADIN HSPF
DaveF
Data Computationalresources
Workflow management
Data management
Visualization Collaboration
Portal
Models
ALADIN HSPF
DaveF
DataData Computationalresources
Computationalresources
GRID
Security
PORTAL ACCESS LEVEL
GRID ACCESS LEVEL
22
Enabling Grids for E-sciencE
INFSO-RI-508833
Conclusion
• Flood forecasting is a complex application with many models, data types and user groups
• FloodGrid portal has been developed with:– Workflow management module– Data management module– Visualization– Collaboration tools
• Current and future works– New data management with improved data access controls
(using ACL support in gLite)