IODE/JCOMM/VLIZ/MHI – Virtual lab project
Virtual Lab project
International oceanographic data and information exchange programme of IOC (IODE)
Joint WMO/IOC technical commision for oceanography and marine meteorology (JCOMM)
Marine Hydrophysical Institute (MHI)
Vlaams Instituut voor de Zee (VLIZ)
IODE/JCOMM/VLIZ/MHI – Virtual lab project
What is VLIZ ?
• VLIZ : Flanders Marine Institute
• Oostende, Belgium
• Supports Marine Research in Flanders
• Support IODE, IOC, UNESCO
• Flanders Marine Datacenter
IODE/JCOMM/VLIZ/MHI – Virtual lab project
Located : Oostende
IODE/JCOMM/VLIZ/MHI – Virtual lab project
What is the virtual lab ?
• The Virtual Lab will be a computing environment collecting the data, models and interfaces needed to run hydrodynamic models for training and research.
IODE/JCOMM/VLIZ/MHI – Virtual lab project
Hydrodynamic models
• Hydrodynamic models like POM predict water currents, wave heights and ocean circulations, so basically the movement of watermasses, and all that is carried by it.
• The water movements are caused by wind, temperature and atmospheric pressure. These meteo parameters are called the forcing data because they make the waterccolumn move. These movements are then deviated by land masses and shallow waters.
• The models are used in forecasting, operational mode when using predicted meteorological parameters. Important for safety at sea, stormsurge warning etc.
• They can also be used in hindcasting using corrected meteo parameters. Mainly for research or training.
IODE/JCOMM/VLIZ/MHI – Virtual lab project
Running models
• Modelling software eg. POM • Forcing data eg.NCEP • Bathymetry eg.ETOPO• Powerfull computers• Visualization software eg GRADS• Expertise & training
IODE/JCOMM/VLIZ/MHI – Virtual lab project
Known problems
• Difficult software– Runs on Linux, command line mode– Edit fortran code, and recompile to run the model.– Install viewers, learn to operate them
• Preparing data, lots of work– Download forcing data (>100Mb : bandwidth ?)– Extract bathymetry
• Very computer intensive– Reformatting data– Running the model
• Possible solution : Virtual Lab
IODE/JCOMM/VLIZ/MHI – Virtual lab project
The virtual lab
• Experts will install the models• Install powerful hardware• Make use of GRID computing• Make software user friendly• Develop easier interfaces• Batch jobs download & reformat the data
automatically• Make VL available through internet
IODE/JCOMM/VLIZ/MHI – Virtual lab project
System diagram
Data repositories
GRID network
Data collectors Interfaces
Data providers Clients
IODE/JCOMM/VLIZ/MHI – Virtual lab project
System diagram:providers
Data repositories
GRID network
Data collectors Interfaces
Data providers ClientsData providers are the met-offices, oceanographic datacentres, basically any organisation that has relevant data
IODE/JCOMM/VLIZ/MHI – Virtual lab project
Data repositories
GRID network
Data collectors Interfaces
Data providers Clients•User authentication based on name/password and/or certificates
•Can be Windows, Mac or Unix machines•Software needed : at least an internet browser (IE, firefox,mozilla…)•Optionally SSH , FTP , VNC, Xwindows software
System diagram:clients
IODE/JCOMM/VLIZ/MHI – Virtual lab project
System diagram:GRID
Data repositories
GRID network
Data collectors Interfaces
Data providers Clients
•GRID network is the EGEE computing GRID, specially the BEGRID component of it.•IODE computers from trainingroom
IODE/JCOMM/VLIZ/MHI – Virtual lab project
System diagram:collectors
Data repositories
GRID network
Data collectors Interfaces
Data providers Clients•Scripts written in perl, php, python, csh•Run several times a day, unattended•Download data from providers•Extract and reformat data to the formats needed by the different models.•Where appropriate the jobs can be submitted to run on the GRID network.
IODE/JCOMM/VLIZ/MHI – Virtual lab project
System diagram:repositories
Data repositories
GRID network
Data collectors Interfaces
Data providers Clients•Computers running NFS, FTP, SMB(samba/windows)•Database servers running MySQL•GIS databases•SAN coupled (fibre disks with Terabytes storage capacity)
IODE/JCOMM/VLIZ/MHI – Virtual lab project
System diagram:interfaces
Data repositories
GRID network
Data collectors Interfaces
Data providers Clients
•Linux systems•Access protocols : SSH, (s)FTP, HTTP,VNC,Xwindows•SSH and FTP access will allow full control of the models•WebPages can simplify the process of running the models, and allow to submit jobs on the GRID, and be used to visualise the results.•VNC and Xwindows access provided the network bandwidth is available
IODE/JCOMM/VLIZ/MHI – Virtual lab project
Internet
System diagram:flow
Data repositories
GRID network
Data collectors Interfaces
Data providers Clients
SSH, VNC, Xwindows, FTPHTTP, HTTPS
FTP,HTTP
NFS, SMB, SQL
edg-job-submit
NFS
edg-job-submit
NFS, SMB
IODE/JCOMM/VLIZ/MHI – Virtual lab project
Candidate models
• Selected for the JCOMM – IODE jamboree– Extra tropical storm surge (Ø.Sætra, NMI)
– WAM wave model from (NMI)
– Tropical storm surge (S. Dube, IIT)
– Circulation model for the Black Sea (Korotaev,MHI)
IODE/JCOMM/VLIZ/MHI – Virtual lab project
First steps : Black sea circulation Model
• Project VLIZ – MHI Ukraine• Based on POM• Running high resolution model for Blacksea is very
computer intensive.– 5km resolution, 1 month : hours– 1km resolution, 1 month : days
• Current approach : nested model– Low resolution model for whole BlackSea– Use this as input for high resolution model for smaller regions
• Ideal for GRID computing– 1 job for whole black sea– each region = 1 job, independant of others
IODE/JCOMM/VLIZ/MHI – Virtual lab project
28.6 28.8 29.0 29.2 29.4 29.6 29.8 30.0 30.2
Long itude, E
43.5
43.7
43.9
44.1
44.3
44.5
44.7
44.9
45.1
45.3
45.5
45.7
La
titu
de
, N
B o tto m re lie f, m
Basin-Scale Circulation Model
Regional Atmospheric Model (ALADIN Family)
Sea Surface Elevation
Current Velocity
Sea Water Temperature and Salinity
Regional Models Family
Kalamita BayRegional Model
Nowcasting system
32.6 32.8 33.0 33.2 33.4 33.644.6
44.8
45.0
45.2
45.4K a l a m i t a
B a y
Romanian Coastal ZoneRegional Model
36.5 37 37.5 38 38.5 3944
44.5
45
Russian Coastal ZoneRegional Model
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
8
9
17.6
17.8
18
18.2
18.4
18.6
18.8
19
19.2
19.4
19.6
19.8
20
20.2
20.4
NWSRegional Model
Burgas Bay Georgian Zone
Forecasting System of the Black Sea Circulation
IODE/JCOMM/VLIZ/MHI – Virtual lab project
low resolution Blacksea model
• Lineair scalable till about 10 – 16 CPU
• Will run on 2x4core CPU system with 8Gb in VL
IODE/JCOMM/VLIZ/MHI – Virtual lab project
regional models
• Kalamita Bay• GRID cluster with
5x2x4 CPU will be used for running the regional models in parrallel
IODE/JCOMM/VLIZ/MHI – Virtual lab project
Planning
• mid 2007 – install equipment : Blades with 5x2xQuadCores, 8Gb– install gLite middleware – install existing models
• end 2007– make models ‘GRID’ enabled– investigate MPI – POM
• beginning 2008– run model in ‘semi-operational’ forecasting mode
• in parallel– start same exercise for other models
IODE/JCOMM/VLIZ/MHI – Virtual lab project
Thank you
Top Related