The DRIHM Portal: architecture, design principles and issues.
28
Coordinating person: Eng. Antonio Parodi, PhD The DRIHM Portal: architecture, design principles and issues. Daniele D‘Agostino (CNR-IMATI)
Transcript of The DRIHM Portal: architecture, design principles and issues.
Coordinating person: Eng. Antonio Parodi, PhD
The DRIHM Portal:
architecture, design principles
and issues.
Daniele D‘Agostino (CNR-IMATI)
2
Terminology
Grid Application Developer
• Develops a grid application with gUSE
• Exports it to the local repository of gUSE
End Users, Scientists
• execute a copy of the exported application with
custom input parameters
Grid Portal Developer
•Develops an interface (in general a portlet) that fits
to the application I/O pattern and the end users’
needs and associate it with the role
• Connects the GUI to gUSE using the programming
API of Application Specific Module(ASM)
DRIHM DCI Ideal Architecture
WPS and
critical case
data repository
Complex run for
meteo models HMR Linux-based
Models
HMR Windows
Models Testbed
Citizen Scientists
and proprietary
software
Model SW and
Static data
Other data
sources
3.6.2
The DRIHMIFICATION process
6
The Binary Repository
7
A DRIHM Model Chain
Probability
Probability
Probability
Ensemble Probability
“Large” Scale
Meteorological
“Small” Scale
Meteorological
Impact
Hydraulic
Hydrologic
DRiFt
Ensemble
RainFARM
COSMO-Model
Arome EPS
Ensemble
Meso-NH
WRF-NMM
Deterministic
HYPROM 2D
HYPROM 1D
WRF-NMM
HBV
SOBEK-FLOW
Delft3D-FLOW
TELEMAC-2D
Deterministic
Property Damage
Measured Rainfall
Deterministic
RIBS
Deterministic Ensemble
Bridge
WPS
The Preprocessing of
Meteorological Input data
9
Probability
Probability
Probability
Ensemble Probability
“Large” Scale
Meteorological
“Small” Scale
Meteorological
Impact
Hydraulic
Hydrologic
DRiFt
Ensemble
RainFARM
COSMO-Model
Arome EPS
Ensemble
Meso-NH
WRF-NMM
Deterministic
HYPROM 2D
HYPROM 1D
WRF-NMM
HBV
SOBEK-FLOW
Delft3D-FLOW
TELEMAC-2D
Deterministic
Property Damage
Measured Rainfall
Deterministic
RIBS
Deterministic Ensemble
WPS
Meteorological Data • Our meteo models are nested in global models which
usually run by the main weather forecasting centres
like ECMWF or NOAA
• NOAA databases are accessible for free, not the
ECMWF ones
– For the purpose of the project activities, it was agreed, during
an informal meeting at EGU2013, that the initial and boundary
conditions files of a limited number of cases can be freely
shared on the DDCI
• In every cases WPS extract from these large
databases only the data that will be used for the
simulation
10
What happens on the DDCI
The Meteorological model execution
Probability
Probability
Probability
Ensemble Probability
“Large” Scale
Meteorological
“Small” Scale
Meteorological
Impact
Hydraulic
Hydrologic
DRiFt
Ensemble
RainFARM
COSMO-Model
Arome EPS
Ensemble
Meso-NH
WRF-NMM
Deterministic
HYPROM 2D
HYPROM 1D
WRF-NMM
HBV
SOBEK-FLOW
Delft3D-FLOW
TELEMAC-2D
Deterministic
Property Damage
Measured Rainfall
Deterministic
RIBS
Deterministic Ensemble
WPS
Bridge
OR
WPS-WRF In SHIWA
14
The portlet needs to change…
15
start.sh $modelName $jobId
WRF Job
16
Created by our portlet
Downloaded from the
Binary Repository
Otherwise…
17
But our approach allows the integrated definition
of the whole experiment
(Metadata validity check)
One model one portlet
18
Wish list 1
19
myproxy-init -s myproxy.cnaf.infn.it -p 7512 -l dagostino -m drihm.eu -t 20
It will be supported in a future version
The hydrological model execution
20
Probability
Probability
Probability
Ensemble Probability
“Large” Scale
Meteorological
“Small” Scale
Meteorological
Impact
Hydraulic
Hydrologic
DRiFt
Ensemble
RainFARM
COSMO-Model
Arome EPS
Ensemble
Meso-NH
WRF-NMM
Deterministic
HYPROM 2D
HYPROM 1D
WRF-NMM
HBV
SOBEK-FLOW
Delft3D-FLOW
TELEMAC-2D
Deterministic
Property Damage
Measured Rainfall
Deterministic
RIBS
Deterministic Ensemble
*
Ensemble forecasting is a numerical prediction method
that is used to attempt to generate a representative sample
of the possible future states. Multiple numerical predictions
are conducted using slightly different initial conditions.
Other data
sources
The Hydraulic-Impact
models execution
Probability
Probability
Probability
Ensemble Probability
“Large” Scale
Meteorological
“Small” Scale
Meteorological
Impact
Hydraulic
Hydrologic
DRiFt
Ensemble
RainFARM
COSMO-Model
Arome EPS
Ensemble
Meso-NH
WRF-NMM
Deterministic
HYPROM 2D
HYPROM 1D
WRF-NMM
HBV
SOBEK-FLOW
Delft3D-FLOW
TELEMAC-2D
Deterministic
Property Damage
Measured Rainfall
Deterministic
RIBS
Deterministic Ensemble
23
gUSE is already connected to many cloud types, and the rOCCI
interface of FedCloud will be supported in a future version
If-Then-Else We try to reduce the number of workflows by exploiting
this feature
24
Wish List 2
25 It will be supported in a future version
Other aspects…
One final
(probably stupid) question
27
Meteo
Hydro
?
• It produced 100 GB of data
• Scheduled on PRACE or Grid
• Possible ensemble simulation
• Scheduled on Grid
Which is the best solution?
28
Meteo
Hydro
Out
In
Meteo
Hydro
Out
In
Out
SE
Downloaded via script
Something.txt
Direct link in a future version
via Data Avenue
but via portal so far
