Post on 13-May-2018
CFD Modelling with OpenFOAM
Hrvoje Jasak
Faculty of Mechanical Engineering and Naval Architecture, Uni Zagreb, Croatia
Wikki Ltd. United Kingdom
Oxford University, 13 October 2017
CFD Modelling with OpenFOAM – p. 1
Outline
Objective
• Present the activities of the CFD Research group in Zagreb and Wikki Ltd.
• Review progress on selected topics related to turbomachinery CFD
Topics
• CFD Group at University of Zagreb: Research Activity
• Harmonic Balance Solver for Turbomachinery flows
• Block matrix and pressure-based coupled implicit solver
• Native Overset Mesh in OpenFOAM
• NUMAP-FOAM Summer School
• Summary
CFD Modelling with OpenFOAM – p. 2
Overview of Research Activity
Research Group Members, CFD Group at University of Zagreb
• CFD Research Group attached to the Chair of Turbomachinery
• 2 professors: Prof. Hrvoje Jasak, Prof. Željko Tukovic
• 1 (+ 1) post doctoral researchers: dr. Vuko Vukcevic
• 6 fully funded PhD students
• Larger group of Master Thesis and Diploma Thesis students integrated within thegroup: results of Master Thesis projects directly used in further research
• Regular external (foreign) visitors working with the group: 3-6 months
Communication and Activity
• Leading developer of OpenFOAM: http://foam-extend.fsb.hr/
• Integrated work effort, modern communication and data integration tools
• Private Media-Wiki: http://spirit.local/mediawiki/
• YouTube Channel: 8th Floor CFD@FSB
• Public web site: http://www.fsb.hr/cfd
• Approximately 25 (significant) publications per year
CFD Modelling with OpenFOAM – p. 3
Overview of Research Activity
Open Source Software in Research and Engineering
• Open Source tools are ideal for a research environment: industrial partner gainsaccess not only to physical model equations but also to a working and validatedimplementation by expert users
• Deployment of results of research is faster and more reliable
• Proven track record of model development and delivery
• First-class students with good technical and CFD background (OpenFOAM)
• Strong multi-disciplinary group: fluids, structure multi-phase flow modelling,turbulence, optimisation. Premier source of numerics knowledge in collaborationwith leading world Universities
CFD Modelling with OpenFOAM – p. 4
Overview of Research Activity
Areas of Research Activity, CFD Group at University of Zagreb
• CFD simulations in Turbomachinery: basic validation and verification andpractical industrial simulations
◦ Incompressible flow: pumps and turbines, wind energy devices
◦ Compressible flow: compressors and fans with pressure- and density-basedCFD solution algorithms
◦ Harmonic balance modelling in CFD as a general-purpose tool
• Gradient-based and gradient-free optimisation: continuous and discrete adjoint
• CFD in naval hydrodynamics, wave and off-shore structures
• Fundamental research in numerics: discretisation, solution techniques, HPCperformance, inter-equation coupling
• Complex solid mechanics modelling: large deformation, lubricated contact,conjugate heat transfer
• Fluid-solid interaction and “multi-physics” modelling
• Detailed fuel cell modelling using CFD
• Acoustics modelling using linearised Euler equations and coupling with CFD
CFD Modelling with OpenFOAM – p. 5
Free Surface Flow Simulations
CFD Modelling with OpenFOAM – p. 6
Turbomachinery Simulations
Full Support for Turbomachinery CFD
• Complex rotor-stator interfaces, for all physics models: GGI, mixing plane
• Incompressible and compressible, MRF and transient
• Periodic transient mode: harmonic balance solvers
CFD Modelling with OpenFOAM – p. 7
Complex Physics in Complex Geometry
Simulating Complex Physical Processes in Complex Geometry
CFD Modelling with OpenFOAM – p. 8
Harmonic Balance Solver
Expanding the Horizons: Spectral Modelling of Time-Periodic Flows
• Many CFD problems involve temporally periodic flows◦ Flows induced by periodic boundary condition◦ Flows with periodically moving objects
◦ Wave-like phenomena
• To remove irregular start-up unsteadiness, a number of periods is simulated:expensive, complicated – periodic uncertainty issues
Harmonic Balance Method
• Replacing a transient problem with a set of coupled “steady-state” snapshots byvirtue of using periodicity of the time-signal in the solution
CFD Modelling with OpenFOAM – p. 9
Harmonic Balance Solver
Harmonic Balance Method: Work-Flow
• Variables are developed into Fourier series in time with n-harmonics andsubstituted into transport equation
• Transport equation with n sine and n cosine parts + mean part is obtained andwritten as a set of 2n+ 1 equations in frequency domain
• Equations are transformed back to time domain in order to be able to usetime-domain boundary conditions and time-domain non-linear flow solver
Harmonic Balance in FOAM-Extend
• Harmonic balance decomposition does not relate to a special physics model:implement HB as a choice of “temporal discretisation” scheme
• Geometric aspects of harmonic balance can be tackled without change
• Currently, HB is implemented in a segregated mode: low memory, and explicitinter-mode coupling terms. Analysis shows issues for HB without a dominantmean flow component: working towards coupled implicit HB solver
• Author: Gregor Cvijetic, Uni Zagreb
CFD Modelling with OpenFOAM – p. 10
Harmonic Balance Solver
Rapid Simulation of Non-Linear Periodic Flow: Harmonic Balance
• A variable is presented by a Fourier series, using first n harmonics and the mean:replacing a transient simulation with a set of coupled steady-state problems
• Periodicity is independently developed in each computational point
• Non-linear interaction is captured without simplification
• Example: Harmonic Balance scalar equation set
∇•(uQtj )−∇•(γ∇Qtj ) = −2ω
2n+ 1
(
2n∑
i=1
P(i−j)Qti
)
Pi =
n∑
k=1
k sin(kωi∆t) for i = {1,2n}
• A transient equation is replaced by a set of n coupled quasi-steady coupled
equations of the same type
• Physical justification: if a functional form of temporal variation is known, thefunction and its time derivative can be reproduced from a small number of datapoints by fitting a prescribed harmonic function (spectral time accuracy!)
CFD Modelling with OpenFOAM – p. 11
Harmonic Balance Solver
Harmonic Balance for Navier-Stokes Equations
• Harmonic balance momentum equation
∇•(utjutj )−∇•(γ∇utj ) = −2ω
2n+ 1
(
2n∑
i=1
P(i−j)uti
)
• Harmonic continuity equation∇•utj = 0
• Harmonic scalar transport
∇•(utj Qtj )−∇•(γ∇Qtj ) = −2ω
2n+ 1
(
2n∑
i=1
P(i−j)Qti
)
• Physical justification: each tj instance represents a “single-time-step” solution;time derivative terms couple solution fields to each other
CFD Modelling with OpenFOAM – p. 12
Harmonic Balance Solver
Harmonic Balance Solver: ERCOFTAC Centrifugal Pump
• Validation of harmonic balance in turbulent incompressible periodic flow
• HB simulations performed using 1 and 2 harmonics: rotor and stator blade count
• Results compared against full transient simulation: excellent agreement
◦ Integral properties: typical error of 2%
◦ Local solution features: pressure on surface in time
◦ Mode and nature of flow instability
• Results are significantly better than expected!
• Substantial reduction in simulation time:◦ Intel Core i5-3570K, 3.4 GHz computer with 16 GB memory◦ Transient run needs approx. 50 blade passages to become quasi-periodic
Transient HB, 1 h HB, 2 h
Simulation time 5 hrs/rotation 52 mins 78 mins
Iterations 600, dt = 5e-5 s 3000 24001 rotation = 0.03 s
CFD Modelling with OpenFOAM – p. 13
Harmonic Balance Solver
Harmonic Balance Solver: ERCOFTAC Centrifugal Pump
CFD Modelling with OpenFOAM – p. 14
Harmonic Balance Solver
Harmonic Balance Solver: ERCOFTAC Centrifugal Pump
-0.2 -0.15 -0.1 -0.05 0x-Axis
-1200
-1000
-800
-600
-400
-200
0
Pre
ssu
re,
Pa
TransientHB, 1hHB, 2h
-0.15 -0.1 -0.05 0 0.05x-Axis
-1200
-1000
-800
-600
-400
-200
0
Pre
ssure
, P
a
TransientHB, 1hHB, 2h
-0.15 -0.1 -0.05 0 0.05x-Axis
-1200
-1000
-800
-600
-400
-200
0
Pre
ssure
, P
a
TransientHB, 1hHB, 2h
Transient HB, 1h err, % HB, 2h err, % MRF err, %
Efficiency 89.72 88.80 1.0 89.76 0.0 89.65 0.07t = T
3Head 81.48 81.80 0.4 80.45 1.3 84.12 3.14
Torque 0.0297 0.0302 1.7 0.0294 0.9 0.0308 3.57
Efficiency 89.92 88.78 1.3 89.81 0.1t = 2T
3Head 81.48 81.85 0.4 80.6 1.1
Torque 0.0296 0.0302 2.0 0.0295 0.4
Efficiency 89.83 88.85 1.1 89.71 0.1t = T Head 81.49 81.79 0.4 80.39 1.3
Torque 0.0297 0.0302 1.6 0.0294 1.0
CFD Modelling with OpenFOAM – p. 15
Block Matrix in OpenFOAM
Background
• OpenFOAM uses equation mimicking to perform field algebra and discretisation:perfect for simple PDE-s or segregated solution algorithms
• . . . but sometimes we use equation segregation inappropriately
• There exists a family of problems that cannot be solved efficiently withoutinter-equation coupling; some simulations “that work” can be performed 10-s or100-s of times faster than with equivalent segregated algorithms
Objective
• Implement flexible and efficient block-coupled solution infrastructure
• Re-use all operator-based discretisation schemes, parallelisation and boundarycondition tools already available in OpenFOAM
• Optimise top-level code for efficient execution and ease of assembly
Examples
• Incompressible steady pressure-velocity system (with turbulence)
• Compressible multi-phase free surface simulations: under-water explosions
CFD Modelling with OpenFOAM – p. 16
Block Matrix in OpenFOAM
Block-Coupled Solution Algorithms
• For cases of strong coupling between the components of a vector, the componentscan be solved as a block variable: (ux, uy , uz) will appear as variables in thesame linear system
• In spite of the fact that the system is much larger, the coupling pattern still exists:components of u in cell P may be coupled to other components in the same pointor to vector components in the neighbouring cell
• With this in mind, we can still keep the sparse addressing defined by the mesh: if avariable is a vector, a tensorial diagonal coefficients couples the vectorcomponents in the same cell. A tensorial off-diagonal coefficient couples thecomponents of uP to all components of uN , which covers all possibilities
• For multi-variable block solution like the compressible Navier-Stokes systemabove, the same trick is used: the cell variable consists of (ρ, ρu, ρE) and thecoupling can be coupled by a 5× 5 matrix coefficient
• Important disadvantages of a block coupled system are
◦ Large linear system: several variables are handled together
◦ Different kinds of physics can be present, e.g. the transport-dominatedmomentum equation and elliptic pressure equation. At matrix level, it isimpossible to separate them, which makes the system more difficult to solve
CFD Modelling with OpenFOAM – p. 17
Block Matrix in OpenFOAM
Matrix Connectivity and Mesh Structure
• Irrespective of the level of coupling, the FVM dictates that a cell value will dependonly on values in surrounding cells
PW E
N
S
• We still have freedom to organise the matrix by ordering entries for variouscomponents of the solution variable x
• Global sparseness pattern related to mesh connectivity: easier coefficientassembly
CFD Modelling with OpenFOAM – p. 18
Block Matrix in OpenFOAM
Coupling Coefficient
• Matrix implemented with block coefficients
• Consider general linear dependence between two vectors m and n
m = Ab
• Component-wise coupling describes the case where mx depends only on nx, my
on ny and mz on nz
1. Scalar component-wise coupling
2. Vector component-wise coupling
3. Full (block) coupling
• Explicit methods do not feature here because it is not necessary to express themin terms of matrix coefficients
• For reference, the linear equation for each cell featuring in the matrix reads
APmP +∑
N
ANmN = R
CFD Modelling with OpenFOAM – p. 19
Coupled p-U Solver
Turbulent Steady Incompressible Flows: SIMPLE or Coupled System
• Equation set contains linear p-U and non-linear U-U coupling
∂u
∂t+∇•(uu)−∇• (ν∇u) = −∇p
∇•u = 0
• Traditionally, this equation set is solved using the segregated SIMPLE algorithm
◦ Low memory peak: solution + single scalar matrix in peak storage
◦ p-U coupling is handled explicitly: loss of convergence (under-relaxation)
◦ Number of iterations is substantial; not only due to non-linearity
◦ Convergence dependent on mesh size: SIMPLE slows down on large meshes
• Block-implicit p-U coupled solution
◦ Coupled solution significantly increases matrix size: 4 blocks instead of 1
◦ . . . but the linear p-U coupling is fully implicit!
◦ Iteration sequence only needed to handle the non-linearity in the U-equation
◦ Net result: significant convergence improvement (steady or transient) at acost of increase in memory usage: reasonable performance compromise!
CFD Modelling with OpenFOAM – p. 20
Coupled p-U Solver
SIMPLE-Based Segregated p-U Solver
// Momentum equation assembly and solution
fvVectorMatrix UEqn
(
fvm::div(phi, U)
+ turbulence->divDevReff()
);
UEqn.relax();
solve(UEqn == -fvc::grad(p));
// Pressure equation assembly and solution
U = UEqn().H()/UEqn.A();
phi = fvc::interpolate(U) & mesh.Sf();
fvScalarMatrix pEqn
(
fvm::laplacian(1/UEqn.A(), p) == fvc::div(phi)
);
pEqn.solve();
phi -= pEqn.flux();
p.relax();
CFD Modelling with OpenFOAM – p. 21
Coupled p-U Solver
Block-Coupled u− p System Matrix Structure
u1 u2p1 p2
(
aP (uu) aP (u p)
aP (pu) aP (p p)
) (
aN (uu) aN (u p)
aN (pu) aN (p p)
)
· · ·
.
(
aP (uu) aP (u p)
aP (pu) aP (p p)
)
· · ·
......
. . .
u1
p1
u2
p2...
=
bu 1
bp 1
bu 2
bp 2
...
CFD Modelling with OpenFOAM – p. 22
Coupled p-U Solver
Coupled Implicit p-U Solver: Source Code
fvVectorMatrix UEqn
(
fvm::div(phi, U)
+ turbulence->divDevReff()
);
fvScalarMatrix pEqn
(
- fvm::laplacian(rUAf, p) == -fvc::div(fvc::grad(p))
);
BlockLduSystem<vector, vector> pInU(fvm::grad(p));
BlockLduSystem<vector, scalar> UInp(fvm::UDiv(U));
BlockLduMatrix<vector4> A(mesh);
blockMatrixTools::insertEquation(0, UEqn);
blockMatrixTools::insertEquation(3, pEqn);
blockMatrixTools::insertBlockCoupling(3, 0, UInp, false);
blockMatrixTools::insertBlockCoupling(0, 3, pInU, true);
UpEqn.solve();
UpEqn.retrieveSolution(0, U.internalField());
UpEqn.retrieveSolution(3, p.internalField());
CFD Modelling with OpenFOAM – p. 23
Performance of the Coupled p-U Solver
Performance of the Coupled p-U Solver: Speed and Robustness
CFD Modelling with OpenFOAM – p. 24
Performance of the Coupled p-U Solver
Performance of the Coupled p-U Solver: Speed and Robustness
CFD Modelling with OpenFOAM – p. 25
Performance of the Coupled p-U Solver
Performance of the Coupled p-U Solver: External Aerodynamics
1e-08
1e-07
1e-06
1e-05
1e-04
1e-03
1e-02
1e-01
1e+00
0 500 1000 1500 2000 2500
Res
idual
Iteration
Ux (simpleFoam)
Uy (simpleFoam)
Uz (simpleFoam)
p (simpleFoam)
BiCGStab Ux
BiCGStab Uy
BiCGStab Uz
BiCGStab p
SAMG Ux
SAMG Uy
SAMG Uz
SAMG p
CFD Modelling with OpenFOAM – p. 26
Performance of the Coupled p-U Solver
Performance Improvements and Extensions in the Coupled p-U Solver
• Improvements in performance for the coupled solver: consistency, numerics
• Extension to compressible flows, MRF and porous media (implicit!)
• Major performance jump: block-coupled AMG with additive correction(Hutchinson 1988)
• Block-coupled k − ǫ and k − ω SST turbulence models
◦ Turbulence equations solved in a single block-coupled system
◦ Analysis of source terms to establish favourable cross-equation coupling
◦ Implemented in Diploma Thesis assignment: Robert Keser, Uni Zagreb
• Example: steady (MRF) and transient centrifugal pump
CFD Modelling with OpenFOAM – p. 27
Performance of the Coupled p-U Solver
Performance Improvements and Extensions in the Coupled p-U Solver
0 100 200 300 400 500Iterations
0.0001
0.01
1
Init
ial
Res
idu
als
Ux_0 (coupled)
Uy_0 (coupled)
Uz_0 (coupled)
p_0 (coupled)
Ux_0 (segregated)
Uy_0 (segregated)
Uz_0 (segregated)
p_0 (segregated)
Comparison between MRFcoupled and MRFsegregated
initial residuals
0 100 200 300 400 500Iterations
1e-05
0.0001
0.001
0.01
0.1
1
Init
ial
Res
idu
als
k_0 (segregated)
omega_0 (segregated)
k_0 (coupled)
omega_0 (coupled)
Comparison between MRFcoupled and MRFsegregated turbulence
initial residuals
0 100 200 300 400 500Iterations
-100
-80
-60
-40
-20
0
20
40
60
80
100
Hea
d [
m]
Head (segregated)
Head (coupled)
Comparison between MRFcoupled and MRFsegregatedOtaBm1 pump head
0 100 200 300 400 500Iterations
0
10000
20000
30000
40000
50000
60000
70000
80000
90000
1e+05
Po
wer
[W
]
Power (segregated)
Power (coupled)
Comparison between MRFcoupled and MRFsegregatedOtaBm1 pump power
0 100 200 300 400 500Iterations
0
20
40
60
80
100
120
140
Eff
icie
ncy
[%
]
Efficiency (segregated)
Efficiency (coupled)
Comparison between MRFcoupled and MRFsegregatedOtaBm1 pump efficiency
CFD Modelling with OpenFOAM – p. 28
Overset Mesh
Open Source Native Overset Mesh Capability in foam-extend
• We have been involved in several previous Overset projects
• . . . but none reached the maturity or provided flexibility required for ageneral-purpose CFD platform
• Project Objective: Complete rewrite without limitations on release
◦ Native implementation of Overset Mesh technology in FOAM-Extend withoutreference to external libraries
◦ Operator-based support for Finite Volume (FV) discretisation: implicit andexplicit operators (fvc and fvm)
◦ Setup via overset patch type, with minimal impact on existing codecomponents: full re-use of existing physical models
◦ Consistent support for strict mass conservation and second order accuratediscretisation in space and time
• Further requirements
◦ Overset mesh functionality must operate on unstructured polyhedral meshhierarchies
◦ Basic layout for massive parallelisation in domain decomposition mode
◦ Support for dynamic overset mesh simulations
CFD Modelling with OpenFOAM – p. 29
Overset Mesh
Overset Mesh Capability in foam-extend: An Update
• Revisited native implementation of Overset Mesh: public release
• Rewrite of basic Overset functionality to improve performance and parallel scaling
Overview of New Overset Capability
• Explicit and Implicit fringe treatments: chosen per-field, depending on application
• Rewrite of parallelised Overset assembly
• New research item: Implicit conservative Overset
• Additional fringe reduction algorithms
• Rewrite of parallel communications to improve scaling on HPC machines
Publicly available Overset Solver forfoam-extend-4.0
CFD Modelling with OpenFOAM – p. 30
Overset Mesh: Example
Overset Mesh Validation: DTC Propeller Case
• Transient propeller simulation for a model scale propeller for DTC hull (DuisburgTest Case, a container ship)
• Case details◦ Model scale propeller for DTC hull, Diameter, D = 0.15 m
◦ n = 890 RPM (14.8343 RPS)
◦ Advance coefficient (J = v/(n*D) = 0.75)
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65Time; t, s
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Thru
st &
torq
ue
coef
fici
ents
; K
T &
10K
Q
KT, CFD (overset,transient)
KT, EXP
KQ
, CFD (overset, transient)
KQ
, EXP
CFD Modelling with OpenFOAM – p. 31
Overset Mesh: Parallel Scaling
Overset Mesh: Parallel Scaling Test
CFD Modelling with OpenFOAM – p. 32
Overset Mesh: Parallel Scaling
Parallel Efficiency of the Overset Mesh
• Implementation of Overset interpolation performed similar to GGI
◦ Interpolation performed in out-of-core multiplication with parallel comms◦ Parallelised using mapDistribute tool
• Parallel scaling test case
◦ Scaling test performed on 20M cells submarine mesh
◦ Approximately 40K donor/acceptor cells (0.4% of total cell count)
◦ Performed 20 iterations with explicit and implicit Overset fringe
• Parallel speed-up on 64 cores: 41 (implicit) and 46 (explicit)
• Parallel efficiency on 64 cores: 64% (implicit) and 74% (explicit)
1 2 4 8 16 32 64Number of cores, n
1
2
4
8
16
32
64
Par
alle
l sp
eed-u
p, S
Explicit fringe
Implicit fringe
Linear scaling
1 2 4 8 16 32 64Number of cores, n
0
20
40
60
80
100
Par
alle
l ef
fici
ency
, E
, %
Explicit fringe
Implicit fringe
CFD Modelling with OpenFOAM – p. 33
NUMAP-FOAM Summer School
NUMAP-FOAM Summer School 2017
• 12th Edition of NUMAP-FOAM Summer School: 21/Aug-1/Sep/2017https://www.fsb.unizg.hr/numap
The idea of the Summer School is to expand the physical modellingknowledge, numerics and programming skills of attendees usingOpenFOAM in their research through direct supervision and one-to-onework.
This is NOT an introductory OpenFOAM course: significantunderstanding of the project and software is a pre-requisite forapplication.
• The School accepts 10-15 attendees bringing their own projects to the School overa period of 10 working days
• Work is embedded in the research group with 4–6 tutors providing daily one-to-oneattention
• School is open to “young researchers” (typically PhD students) but also toindustrial users, government labs and professors
• Strong follow-up collaboration and extensive publication lists
• Approx 150 attendees to NUMAP-FOAM, from the start in 2008
CFD Modelling with OpenFOAM – p. 34
Summary
Summary
• Ongoing research activity at Uni Zagreb on naval hydrodynamics, basic numericsand turbomachinery CFD
• Actively looking for collaboration partners
Current Work Topics
• Naval hydrodynamics: added resistance in regular and irregular waves, full-scaleship simulation, self-propulsion and manoeuvring, green water and freak waveimpact, modelling of irregular sea states
• Numerics: strongly coupled solution algorithms, Discontinuous Galerkindiscretisation, Overset Mesh and Immersed Boundary
• Turbomachinery: quasi-periodic methods (harmonic balance), LES and instabilitymodelling, implicit pressure- and density-based solvers, turbulence and transition
• Solid mechanics and FSI: coupled non-linear FSI problems
• Optimisation: Gradient-free and adjoint-based methods; uncertainty propagationand robust design
CFD Modelling with OpenFOAM – p. 35
About Me
Hrvoje Jasak
• First degree: mechanical engineering, University of Zagreb, Croatia 1992
• PhD, Imperial College London 1993-1996
• Senior development engineer, CD-adapco (Siemens), 1996-2000
• Technical director, Nabla Ltd. 2000-2004
• Consultant on CFD software, numerics and modelling, ANSYS Fluent 2000-2008
Current Work
• Director, Wikki Ltd: UK-based consultancy company 2004-
• Professor, University of Zagreb, Croatia 2007-
• Mercator Fellow, TU Darmstadt, 2016-
• Various software development and commercial support projects based onOpenFOAM with consultants and large industrial partners
• Coordinating open source OpenFOAM development to allow contributions from thepublic domain developers
• OpenFOAM workshops, lectures and seminars, visiting professorships (TU Delft,Chalmers University and others)
CFD Modelling with OpenFOAM – p. 36