Virtual Lab project

IODE/JCOMM/VLIZ/MHI – Virtual lab project

Virtual Lab project

International oceanographic data and information exchange programme of IOC (IODE)

Joint WMO/IOC technical commision for oceanography and marine meteorology (JCOMM)

Marine Hydrophysical Institute (MHI)

Vlaams Instituut voor de Zee (VLIZ)


What is VLIZ ?

• VLIZ : Flanders Marine Institute

• Oostende, Belgium

• Supports Marine Research in Flanders

• Support IODE, IOC, UNESCO

• Flanders Marine Datacenter


Located : Oostende


What is the virtual lab ?

• The Virtual Lab will be a computing environment collecting the data, models and interfaces needed to run hydrodynamic models for training and research.


Hydrodynamic models

• Hydrodynamic models like POM predict water currents, wave heights and ocean circulations, so basically the movement of watermasses, and all that is carried by it.

• The water movements are caused by wind, temperature and atmospheric pressure. These meteo parameters are called the forcing data because they make the waterccolumn move. These movements are then deviated by land masses and shallow waters.

• The models are used in forecasting, operational mode when using predicted meteorological parameters. Important for safety at sea, stormsurge warning etc.

• They can also be used in hindcasting using corrected meteo parameters. Mainly for research or training.


Running models

• Modelling software eg. POM • Forcing data eg.NCEP • Bathymetry eg.ETOPO• Powerfull computers• Visualization software eg GRADS• Expertise & training


Known problems

• Difficult software– Runs on Linux, command line mode– Edit fortran code, and recompile to run the model.– Install viewers, learn to operate them

• Preparing data, lots of work– Download forcing data (>100Mb : bandwidth ?)– Extract bathymetry

• Very computer intensive– Reformatting data– Running the model

• Possible solution : Virtual Lab


The virtual lab

• Experts will install the models• Install powerful hardware• Make use of GRID computing• Make software user friendly• Develop easier interfaces• Batch jobs download & reformat the data

automatically• Make VL available through internet


System diagram

Data repositories

GRID network

Data collectors Interfaces

Data providers Clients


System diagram:providers

Data repositories

GRID network


Data providers ClientsData providers are the met-offices, oceanographic datacentres, basically any organisation that has relevant data


Data repositories

GRID network


Data providers Clients•User authentication based on name/password and/or certificates

•Can be Windows, Mac or Unix machines•Software needed : at least an internet browser (IE, firefox,mozilla…)•Optionally SSH , FTP , VNC, Xwindows software

System diagram:clients


System diagram:GRID

Data repositories

GRID network



•GRID network is the EGEE computing GRID, specially the BEGRID component of it.•IODE computers from trainingroom


System diagram:collectors

Data repositories

GRID network


Data providers Clients•Scripts written in perl, php, python, csh•Run several times a day, unattended•Download data from providers•Extract and reformat data to the formats needed by the different models.•Where appropriate the jobs can be submitted to run on the GRID network.


System diagram:repositories

Data repositories

GRID network


Data providers Clients•Computers running NFS, FTP, SMB(samba/windows)•Database servers running MySQL•GIS databases•SAN coupled (fibre disks with Terabytes storage capacity)


System diagram:interfaces

Data repositories

GRID network



•Linux systems•Access protocols : SSH, (s)FTP, HTTP,VNC,Xwindows•SSH and FTP access will allow full control of the models•WebPages can simplify the process of running the models, and allow to submit jobs on the GRID, and be used to visualise the results.•VNC and Xwindows access provided the network bandwidth is available


Internet

System diagram:flow

Data repositories

GRID network



SSH, VNC, Xwindows, FTPHTTP, HTTPS

FTP,HTTP

NFS, SMB, SQL

edg-job-submit

NFS

edg-job-submit

NFS, SMB


Candidate models

• Selected for the JCOMM – IODE jamboree– Extra tropical storm surge (Ø.Sætra, NMI)

– WAM wave model from (NMI)

– Tropical storm surge (S. Dube, IIT)

– Circulation model for the Black Sea (Korotaev,MHI)


First steps : Black sea circulation Model

• Project VLIZ – MHI Ukraine• Based on POM• Running high resolution model for Blacksea is very

computer intensive.– 5km resolution, 1 month : hours– 1km resolution, 1 month : days

• Current approach : nested model– Low resolution model for whole BlackSea– Use this as input for high resolution model for smaller regions

• Ideal for GRID computing– 1 job for whole black sea– each region = 1 job, independant of others


28.6 28.8 29.0 29.2 29.4 29.6 29.8 30.0 30.2

Long itude, E

43.5

43.7

43.9

44.1

44.3

44.5

44.7

44.9

45.1

45.3

45.5

45.7

La

titu

de

, N

B o tto m re lie f, m

Basin-Scale Circulation Model

Regional Atmospheric Model (ALADIN Family)

Sea Surface Elevation

Current Velocity

Sea Water Temperature and Salinity

Regional Models Family

Kalamita BayRegional Model

Nowcasting system

32.6 32.8 33.0 33.2 33.4 33.644.6

44.8

45.0

45.2

45.4K a l a m i t a

B a y

Romanian Coastal ZoneRegional Model

36.5 37 37.5 38 38.5 3944

44.5

45

Russian Coastal ZoneRegional Model

0 1 2 3 4 5 6 70

1

2

3

4

5

6

7

8

9

17.6

17.8

18

18.2

18.4

18.6

18.8

19

19.2

19.4

19.6

19.8

20

20.2

20.4

NWSRegional Model

Burgas Bay Georgian Zone

Forecasting System of the Black Sea Circulation


low resolution Blacksea model

• Lineair scalable till about 10 – 16 CPU

• Will run on 2x4core CPU system with 8Gb in VL


regional models

• Kalamita Bay• GRID cluster with

5x2x4 CPU will be used for running the regional models in parrallel


Planning

• mid 2007 – install equipment : Blades with 5x2xQuadCores, 8Gb– install gLite middleware – install existing models

• end 2007– make models ‘GRID’ enabled– investigate MPI – POM

• beginning 2008– run model in ‘semi-operational’ forecasting mode

• in parallel– start same exercise for other models


Thank you

Virtual Lab project

Documents

Transcript of Virtual Lab project