DSD-INT 2016 Calibration and scenario generation of hydrodynamics and water - Aguilar Gómez
-
Upload
delftsoftwaredays -
Category
Software
-
view
43 -
download
0
Transcript of DSD-INT 2016 Calibration and scenario generation of hydrodynamics and water - Aguilar Gómez
Calibration and scenario generation of Hydrodynamics and Water Quality models over a Cloud Computing environment: INDIGO-DataCloud
RIA-653549
Presented by Fernando Aguilar (IFCA-CSIC) [email protected] INDIGO-DataCloud WP2
Delft3D Days – 1st November 2016
Introduction • Framework: Collaboration with SME Ecohydros. Last year ROEM+.
• Reservoir Hydrodinamic and Water Quality modelling. Cuerda del Pozo: water supply, water activities.
• Previous work • Platform takes data from water: physical, chemical, biological, etc. Allows to know water status (data
taken since 2010 aprox.)
• Data visualization tool. Aims to alert authorities when the water quality is under the limits.
• High Resolution Models require a powerful and scalable computing environment.
November 2016 2
Case Study: Algae Bloom in a Water Reservoir
• Research Community: LifeWatch (ESFRI) • Topic/Area: Biodiversity & Ecosystem research
• Objective of the Case Study: • Monitor the evolution of the potential eutrophication of a Water Reservoir.
• Two main objectives: • Scenarios: What if? Water flow, nutrients, etc. Range.
• Calibration: Algae Parameters like Mortality, Respiration, etc.
• Both requires a number of simulations. • Scalability and flexibility needed.
• User Friendly Management: Input, Configuration, N outputs.
• Cloud Computing is the answer! • INDIGO-DataCloud Project is developing all the solution needed.
3
The INDIGO-DataCloud Project • INDIGO = INtegrating Distributed data Infrastructures for Global ExplOitation
• H2020 project, Apr 2015 – Oct 2017, 11M EURO
• 26 partners in 11 European countries
• Objectives: Develop a data/computing platform, targeting various scientific communities and deployable on hybrid (private or public) Cloud infrastructures
• Website: https://www.indigo‐datacloud.eu INDIGO-DataCloud RIA-653549
4
INDIGO-DataCloud RIA-653549
INDIGO Architecture
5
• Enhanced features in • IaaS • PaaS • SaaS
• 1st release MidnightBlue • Data Center Solutions • Data Solutions • Automated Solutions • High-level User Oriented
Solutions
INDIGO Communities
November 2016 6
Life Sciences: ELIXIR, INSTRUCT/WeNMR, EuroBioImaging
Physical Sciences & Astronomy: CTA, LBT, WLCG
Social Sciences & Humanities: DARIAH, DCH-RP
Environmental Science: LifeWatch, EMSO, ENES
Agile Methodology to satisfy User Requirements!
Solution Developed
9
• Distributed Storage Solution • Accessible by users and
Machines • DropBox-like, but online • WebClient, POSIX, etc. • Sharable Environment • Input/Output
Solution Developed
10
• GitHub: Software Repository. Script for running the “job”. • DockerHub: SO, Delft3D Software and all dependencies ready to be
deployed. • Ansible: Recipes for installing requirements (OneData).
Solution Developed
11
• User Interface • Generates TOSCA template
from user configuration, including model params.
• Sends json to Orchestrator. • Everything transparent for
the user.
Solution Developed
12
• PaaS Orchestrator: Deploys the docker and manage the execution. • Deployment over many different IaaS Services. • Using Mesos/Chronos. • Running instances mount OneData, get the input and write the output. • Output available for the user.
Demo description • Testbed resources that will be used: Bari TestBed
• Teams involved: INFN/Bari, PSNC, IFCA/CSIC
• Prerequisites: application in Docker
• Sequence of actions • Connect to IAM to access OneData. Input data upload. • Access to the Graphical User Interface (FutureGateway). Fill the form (OneData,
Access, Sweep Parameter values). Submit. • The TOSCA template edited and sent to the orchestrator. • Check Deployment status. • After finishing, output accessible via OneData. Comparing models using Delft3D
tools.
• NOW WE SWITCH TO THE DEMO SCREEN…
14
INDIGO added value
• Scalable (storage and computing) resources in the cloud to perform o(100-10000) tests…
• …and share directly within the community
• User Friendly interface to use cloud resources: • Final users only need to fill a form to submit a new simulation, avoiding
the script edition or direct contact with the infrastructure (Supercomputer, Grid, Cloud) (very helpful for non IT experts).
• First time we use a flexible and “universal” user authentication (quite relevant to collaborate with SMEs also)
• Transparent access to shared large storage (OneData)
15
Conclusions
• Cloud Computing environment is a recommendable framework to run a number of Delft3D simulations.
• INDIGO-DataCloud solutions integration offers a user-oriented alternative: powerful and easy-to-use.
• IT skills are not needed to use Computing Resources. • Distributed and Sharable Storage system available: no
need to copy files and scientists can collaborate easily. • AAI Solution for accessing. • INDIGO-DataCloud solutions available for using (Open
Source).
16
Thank you
https://www.indigo-datacloud.eu Better Software for Better Science.
17
Fernando Aguilar (IFCA-CSIC) [email protected]