Delft3D Open Source

Workshop

Delft, Nov. 07, 2014

Qinghua Ye

Adri Mourits

Overview

• Introduction

• Code management

• Development environment

Exercises

• General code structure Delft3D-FLOW, -MOR, -WAVE

• Ongoing developments

OR

• Optional topics

Exercises

In main building:

Lunch: 12:30 – 14:00

Drinks: 17:30

Overview

Optional topics (Votes will be counted before the first exercise)

• Linux

• Parallel

• 64-bit

• Debugging

• Domain Decomposition

• OSS site

• Mixed C++/Fortran

• Your suggestions

• …

Overview

Organisation of this workshop

• All information is at

https://svn.oss.deltares.nl/repos/delft3d/branches/research/Deltare

s/20131108_workshop

• You do the work

• Qinghua and Adri give information when needed

• Work on exercises/your own stuff whenever you want

• Switch to other workshops whenever you want

• For the rest:

Introduction

Introduction

• Assumed knowledge

• What is Delft3D?

http://www.deltaressystems.com/hydro/product/621497/delft3d-suite

• NOT in this workshop

• GUI usage

http://oss.deltares.nl/web/delft3d/screen-casts

• Creating a model

http://oss.deltares.nl/web/delft3d/modelling-guidelines

http://oss.deltares.nl/web/delft3d/training-courses

Adri Mourits Qinghua Ye

Introduction

Open Source Software site: www.oss.deltares.nl

• Getting started, download source code

• Screen casts

• Webinars

• Modelling Guidelines

• FAQ, release notes

• Discussion groups

Code management

Code management

Overview:

• SVN

• Setups

• FLA

• Future Open Source plans

Code management - SVN

SVN

• Trunk, main line:

• Fixing bugs

• New developments being merged in

• Run on testbench

=> Possibly not stable

• Tags:

• Copies of stable, fully tested Trunk-revisions

=> Use the latest. Currently

https://svn.oss.deltares.nl/repos/delft3d/tags/3058

• Branches:

• Separate develop versions

• “Your own private version”

Code management - SVN

SVN

• New branch:

• For “big” changes

• When stability is harmed too much

• Private developments

Code management – Setups

Receiving from Deltares:

Setups

• Open source installation (“Install-Shield.exe”):

All User Interfaces but not the FLOW/WAVE executables

• Services installation:

All user interfaces and also FLOW/WAVE executables

(Delft3D service package needed, see

http://oss.deltares.nl/web/delft3d/services)

• Setup variants including WAQ

License File still needed when using UserInterfaces / Delft3D-menu.

Code management – from Deltares

Receiving from Deltares:

GUI

• Quickplot source code:

https://svn.oss.deltares.nl/repos/delft3d/trunk/src/tools_lgpl/matlab/

quickplot

MatLab needed.

Code management - FLA

FLA

When delivering source code changes, an FLA agreement must be

signed.

Code management – future plans

Future Open Source plans

• Flexible mesh: currently in a restricted community, later fully open

• Delta Shell (Framework, partly): 2014

Development environment

Development environment

Overview:

• Windows

• VisualStudio 2008/2010/2012

• Build configuration (Release/Debug)

• Linux

• Automake, valgrind

• Resulting files

• Version numbers

See file “….\src\README “

Development environment

Windows

• Visual Studio 2008: generation as standard version

-> d_hydro_open_source.sln

• Visual Studio 2010: also supported

-> d_hydro_open_source_vs2010.sln

-> d_hydro_open_source_vs2010_intel13.sln

• Visual Studio 2012: not supported yet, but it does compile

-> d_hydro_open_source_vs2010_intel13.sln

• Build configuration “Release” version:

Use by default

• Build configuration “Debug” version:

Use only for tracing/solving bugs

Development environment

Linux

• GNU Autotools/Libtools

• 2 build methods:

1. If all tools are installed on default location: ./autogen.sh

./configure --prefix=`pwd`

make ds-install

2. If some tools are not on default location: Check paths/settings in script “build.sh” and execute build.sh –intel11.1 -64bit

• Debug: Add flag –debug when executing build.sh valgrind $exedir/deltares_hydro.exe $argfile TotalView Write statements

Development environment

Resulting release files (Windows)

• FLOW: ….\bin\win32\flow2d3d\bin: d_hydro.exe flow2d3d.dll mormerge, plugins, datsel, kubint, lint DOL dll’s MPICH dll’s more dll’s (OpenDA, compiler, xml etc.)

• Wave with SWAN: ….\bin\win32\wave\bin: wave.exe ….\bin\win32\swan\bin: swan_4072ABCDE_del_w32_i11_omp.exe

Resulting files FLOW, debug (Windows) ….\src\engines_gpl\d_hydro\bin\Debug

To be started

Small executable

XML input file

Big dll containing

Delft3D-FLOW

Remote Online Visualisation

Parallel calculations

Development environment

Resulting release files (Linux)

• FLOW: ….\bin\lnx\flow2d3d\bin: d_hydro.exe flow2d3d.so mormerge, plugins, datsel, kubint, lint DOL so’s MPICH so’s more so’s (OpenDA, compiler, xml etc.)

• Wave with SWAN: ….\bin\lnx\wave\bin: wave.exe ….\bin\lnx\swan\bin: swan_4072ABCDE_del_l64_i11_omp.exe

Install script called via “post build event (Win)”/ “ds-install rule (Lin)”

To be started

Small executable

XML input file

Big dll containing

Delft3D-FLOW

Remote Online Visualisation

Parallel calculations

Development environment

Version number

6.01.04.3058

Update_version_tools:

• Windows: pre-build commands in projects

• Linux: make rule “BUILT_SOURCES = ./always”

Every module has it’s own version number!

Increased

when not

backwards

compatible

New major

functionality

Minor

changes,

used to

distinct stable

versions

SVN,

Automatically

generated

revision

number

Development environment

Update_version_tools:

• On every compile action, version number, date and time are

generated

• Tri-diag file:

• “C:\Delft3D\w32\lib\what.exe flow2d3d.dll”:

Deltares, DELFTIO Version 1.09.00.4108, Sep 16 2014, 11:22:21

Deltares, DELTARES_COMMON Version 1.00.00.4108, Sep 16 2014, 11:22:06

Intel Fortran RTL Message Catalog V12.1-110 Apr 04 2012

$HeadURL:

https://svn.oss.deltares.nl/repos/delft3d/trunk/src/utils_lg

pl/deltares_common/packages/deltares_common/src/deltares_common_version.

F90.svn

$

Deltares, NEFIS Version 5.07.02.4108, Sep 16 2014, 11:22:10

Deltares, ESMFSM Version 4.06.00.4108, Sep 16 2014, 11:22:02

Deltares, FLOW2D3D Version 6.01.08.4108, Sep 16 2014, 11:26:16

*** FLOW2D3D Version 6.01.08.4108, Sep 16 2014, 11:26:16

*** built from : https://svn.oss.deltares.nl/repos/delft3d/trunk

Vote for optional topics

Optional topics:

• Linux

• Parallel

• 64-bit

• Debugging

• Domain Decomposition

• OSS site

• Mixed C++/Fortran

• Your suggestions

• …

Prepared:

• General code structure Delft3D-FLOW, -MOR, -WAVE

• Ongoing developments

Exercises

General code structure

General code structure

• Directory structure / VS projects

• FLOW

• Main (DD/OLV)

• Trisim (gdp)

• Tricom (time loop)

• Trisol (ADI)

• MOR

• Erosed

• Bott3d

• WAVE

• SWAN

• Doxygen

• NaN check

General code structure

FLOW Directory structure

From

top-level

To

…\src\engines_gpl\flow2d3d\packages\kernel\src\compute\adi.f90

General code structure

Directory structure

General code structure

Directory structure

• Bin: resulting executables

• Examples: Ready-to-run testcases (after compiling release version)

• Src: source code

• Engines_gpl: source code per kernel

• Third_party_open: source code/binaries from outside this source

tree (mainly from outside Deltares)

• Tools: additional programs, Post processing

• Utils: Generic libraries, used by more than one kernel

General code structure

Directory structure

Engines:

• d_hydro: Executable starting FLOW dynamic lib

• flow2d3d: Delft3D-FLOW kernel

• Packages: source code, separated from binaries, doc, scripts, etc.

• wave

General code structure

Directory structure

Generic

package names

Package dependencies:

data manager

kernel

io

plugin

flow2d3d

main

General code structure

Directory structure

Each package:

Separate src

(,include) and

binaries

General code structure

Directory structure

Kernel/src: Traditional

Functional splitting Trisim structure

General code structure

Directory structure

Files!

General code structure

Directory structure

Linux:

• Binaries next to source code (*.o, *.lo, *.la) default

• Libraries/executables copied to “src/lib” and “src/bin” default

• Install script copies to “bin/lnx” Deltares

General code structure

Visual Studio

2010

General code structure

FLOW - Main

…\src\engines_gpl\d_hydro\packages\d_hydro\src\d_hydro.cpp main (argc, argv, envp) … DeltaresHydro * DH = new DeltaresHydro (argc, argv, envp);

DH->Run (); DeltaresHydro::DeltaresHydro (argc, argv, envp)

…

dllhandle = LoadLibrary (“flow2d3d”);

entryPoint = GetProcAddress (dllhandle, “DeltaresHydroEntry”);

DeltaresHydro::~DeltaresHydro (void)

DeltaresHydro::Run (void)

…

this->startComponent->Run ();

exe

General code structure

FLOW - Main

…\src\engines_gpl\flow2d3d\packages\flow2d3d\src\flow2d3d.cpp

DeltaresHydroEntry (DeltaresHydro * DH)

DH->startComponent = new Flow2D3D (DH);

return DeltaresHydroEntry (DH);

Flow2D3D::Flow2D3D (DeltaresHydro * DH)

Flow2D3D::~Flow2D3D (void)

Flow2D3D::Run (void)

if (this->dd != NULL)

this->dd->Run ();

else {

TRISIM (ndom, nmap, id, flags, runid);

}

dll

General code structure

FLOW - Fortran entry for each subdomain:

…\src\engines_gpl\flow2d3d\packages\manager\src\trisim.F90

subroutine trisim …

allocate(gdp)

…

retval = trisim_init

…

retval = trisim_step

…

retval = trisim_finish

gdp: GlobalDataPointer

Structure containing

(pointers to) all

arrays/parameters being

global inside one subdomain.

gdp must be passed through

“everywhere” inside one

subdomain for thread safety.

Fortran

General code structure

FLOW - Subdomain initialization:

…\src\engines_gpl\flow2d3d\packages\manager\src\trisim_mod.F90

function trisim_init

…

call gdp_alloc(gdp)

…

call tdatmain

…

call tripoi

…

call tricom_init

Scan input, write time

dependent data to TMP-files

Allocate arrays

Read complete input

Check input

Initialization

General code structure

FLOW - Subdomain step: …\src\engines_gpl\flow2d3d\packages\manager\src\trisim_mod.F90 function trisim_step

call tricom_step …\src\engines_gpl\flow2d3d\packages\manager\src\tricom_step.F90 subroutine tricom_step

…

do nst = itstrt, itstop - 1, 1 …

call postpr

… update wave info

call trisol (/z_trisol/z_trisol_nhfull)

enddo

Time loop

Post processing

Solver

General code structure

FLOW - Subdomain solver: …\src\engines_gpl\flow2d3d\packages\kernel\src\main\trisol.f90 subroutine trisol

…

call f0isf1

timnow = timnow + 0.5_fp

call incbc

call adi(…stage1) … susp. transport solvers

call f0isf1

timnow = timnow + 0.5_fp

call incbc

call adi(…stage2) … susp. transport solvers

First half timestep:

v implicit, u explicit

Second half timestep:

u implicit, v explicit

General code structure

Morphology

…\src\engines_gpl\flow2d3d\packages\kernel\src\main\trisol.f90 subroutine trisol

… call adi(…stage1) … susp. transport solvers

call erosed

…

call bott3d

… call adi(…stage2) … susp. transport solvers

call erosed

…

call bott3d

Sediment fluxes at bed

Bathymetry update

General code structure

WAVE - main

…\src\engines_gpl\wave\packages\wave\src\wave.f90

program waves_main

…

if (not stand_alone)

do while perform_step

wait for FLOW signal

call swan_tot

enddo

else

call swan_tot

endif

General code structure

WAVE – swan_tot

…\src\engines_gpl\wave\packages\manager\src\swan_tot.f90

subroutine swan_tot

…

do itide = 1, swan_run%nttide

do i_swan = 1, n_swan_grids

… get input (from FLOW)

… write swan input

call run_swan

call read_swan_output

enddo

… write wave output (to FLOW)

enddo

General code structure

Code documentation using Doxygen:

www.oss.deltares.nl -> download -> manuals

Link to preprocessed Doxygen output:

http://apidocs.oss.deltares.nl/delft3d

Or generate it yourself:

• Download and install Doxygen from

• http://www.stack.nl/~dimitri/doxygen/

• Download and install Graphviz from

• http://www.graphviz.org/

• In src directory give command:

doxygen doxyfile_delft3d

• Open the generated “delft3d-apidoc\html\index.html”

• Search for top level flow routines as a starting point: trisim, tricom, trisol, or specifically for morphology erosed.

General code structure

Code documentation using Doxygen

General code structure

Not a Number (NaN) check

By default, FLOW continues calculation with NaNs:

• Full check is too time consuming

• Input is scanned on NaNs

• Waterlevel is scanned on NaNs every half time step

Debugging with full NaN check (Intel compiler):

• Activate 4 lines of code in trisim.f90 (see next slide)

• Recompile

General code structure

NaN check

…\src\engines_gpl\flow2d3d\packages\manager\src\trisim.F90

subroutine trisim

…

use ifcore

…

INTEGER*4 OLD_FPE_FLAGS, NEW_FPE_FLAGS

…

NEW_FPE_FLAGS = FPE_M_TRAP_OVF + FPE_M_TRAP_DIV0 +

FPE_M_TRAP_INV

OLD_FPE_FLAGS = FOR_SET_FPE (NEW_FPE_FLAGS)

Ongoing developments

Ongoing developments

Overview of developments on next 3 slides (for completeness)

• OSS site:

http://oss.deltares.nl/web/delft3d/publications

• Open source branches

16 new branches in 2013 from inside Deltares.

More than 10 new branches in 2014

• Closed source branches

Old (< 2011)

Ongoing developments – OSS site

Ongoing developments – Open source branches

20110201-BwN-mud_dynamics/

20110308_Kleptsova_Pietrzak_Stelling_Coriolis/

20110405_12987_openmi_openda_merge/

20110408_Liang_consolidation/

20110420_OnlineVisualisation/

20110510_cormix/

20110512-Rivermorphology-semi_fixed_layers/

20110630_GrandCanyon/

20110712_14855_flow-wave-timelag/

20110818_nieuwboer_sediment/

20111011_luijendijk/

20111205_bedform_angle/

20120109_k_tau/

20120131-Bank_erosion/

20120217_tangential_boundary_conditions/

20120413_outflow-boundary/

20120606_reorganizing_sediment_transport/

20120817_15922_gnu_optimization_problem_wavef90/

20121029-RMD/

20121113-Semi_fixed_layers/

20121130_workshop/

20130228_16080_xml/

20130416_PLIC_VOF_bankEROSION/

20130603-DAD_uniform_graded/

20130620_reorganizing_sediment_transport/

20130725_SANTOSS_sediment_transport/

20130808_tidal_turbines/

20130912_12819_EC-module/

20131003_quasi3d_Bendflow/

20131008_16582_mass_conservation_with_ztbml/

20131021_Telemac_coupling/

20131108_workshop/

20131129_UNST-389_D-FLOWFM_WAVE

20131218_UNST-386_trachytopes

20140103_Galappatti_FacTsd

20140104_Delwaq_Special_KRWV_UNOPS

20140128_12924_ice_coverage

20140203_QP_for_VOF_bankEROSION

20140319_back_temp

20140322_nefis_variable_cell_dimension

20140326_Flow_netcdf

20140402_quasi_3d_bend_flow

20140416_ShinHanul

20140429_moerman

20140512_Irv

20140626_33406_Subsidence_Uplift

20140716_Delwaq_parallel_testing

20140911_broekema_sed_transport_analysis

https://svn.oss.deltares.nl/repos/delft3d/branches/research/Deltares/

Ongoing developments – Closed source branches

OpenMI

power station capacity steered

TRITON online

NearFarfield tk

OLV DOL

river

slib3D

streams on windows

CIW coolingwater Threetox

bubble screen sigma parallel

tangential BC

OLV DOL

TO semi fixed layers for bedcomposition

xbeach in delft3d-flow

DD multi node

……

Questions & answers / discussion

Questions or comments?