Post on 17-Jun-2020
www.lbmethod.org/palabos
Tutorial: the open-source library Palabos in your daily work
Jonas LattEPFL, Lausanne, Switzerland
Rome, DSFD 2010
www.lbmethod.org/palabos
Palabos: a library for developing LB models, and for running LB simulations
Two development interface
I C++: The native interface; powerful and general, but not always easiest to use.I Palabos: A scripting interface which combines the power of Palabos with a
look-and-feel similar to Matlab.
Distinguishing features
I Combines high-performance and interactivity.I Offers a convenient platform to develop new models.
www.lbmethod.org/palabos
Example: flow through a porous media
Let’s develop a simple application which contains
I Creation of the geometry of a porous media.I Execution of the simulation.I Visualization of the result.I Validation of the numerical model.
www.lbmethod.org/palabos
Comment
The Palabos application presented here is written in the Python language and wastyped interactively during the DSFD 2010 tutorial. To run it yourself, you need to
download the Palabos library at www.lbmethod.org/palabos, and access thefile examples/porous.py.
www.lbmethod.org/palabos
The code
#!/usr/bin/pythonfrom palabos import *
nx, ny, nz = 100, 50, 50tau = 0.6lattice = Block3D(nx,ny,nz, D3Q19, BGK(omega=1/tau))pout.display( lattice[10,10,10].getPopulations() )lattice[10,10,10].initializeAtEquilibrium(1.1, [0.,0.,0.])
lattice.collideAndStream()for i in range(1,100): lattice.collideAndStream()imShow(lattice[:,:,nz/2].velocityNorm())
lattice[10:30,10:30,10:30].defineDynamics(BounceBack(1.))dir(lattice)help(lattice.defineDynamics)for i in range(1,100): lattice.collideAndStream()imShow(lattice[:,:,nz/2].velocityNorm())
lattice.defineDynamics(BGK(omega=1./tau))x,y,z = lattice.meshGrid()media = (x−10)**2 + (y−10)**2 + (z−10)**2 < 10**2pout.isoSurface(media, [1.])nobst = 30r = 8for cx,cy,cz in zip(rand_int(nobst,nx), rand_int(nobst,ny), rand_int(nobst,nz)): pout.display(’Sphere at position’, cx, cy, cz) media += (x−cx)**2 + (y−cy)**2 + (z−cz)**2 < r**2pout.isoSurface(media>0, [1.])
boundary.regularized().definePressureBC(lattice[0,:,:])lattice[0,:,:].setBoundaryDensity(1.02)boundary.regularized().definePressureBC(lattice[nx−1,:,:])lattice[nx−1,:,:].setBoundaryDensity(1.)lattice[media>0].defineDynamics(BounceBack(1.))
for i in range(1,200): lattice.collideAndStream()
imShow(lattice[:,:,20].velocityNorm())
pout.display( lattice[media==0][nx/2,:,:].velocityComponent(0).average() )pout.imagesc( [ lattice.velocity().strainRate().symmetricTensorNorm()[:,:,20], lattice.strainRateFromStress().symmetricTensorNorm()[:,:,20] ] )
www.lbmethod.org/palabos
Step-by-step: Create the lattice
Input
nx, ny, nz = 100, 50, 50tau = 0.6lattice = Block3D(nx,ny,nz, D3Q19, BGK(omega=1/tau))
www.lbmethod.org/palabos
Step-by-step: Access a single cell
Input
pout.display( lattice[10,10,10].getPopulations() )lattice[10,10,10].initializeAtEquilibrium(1.1, [0.,0.,0.])
Output
[ 0.33333333 0.05555556 0.05555556 0.05555556 0.027777780.02777778 0.02777778 0.02777778 0.02777778 0.027777780.05555556 0.05555556 0.05555556 0.02777778 0.027777780.02777778 0.02777778 0.02777778 0.02777778 ]
www.lbmethod.org/palabos
Theory: Collision rules
Each lattice node can have different collision ruleI Additionally to providing the algorithm for collision, this operation specifies
information for computing macroscopic variables, rescaling the populations,etc. . .
I This way of treating algorithms like data is typical for object-orientedprogramming.
A lattice cell in Palabos looks as follows:
www.lbmethod.org/palabos
Step-by-step: Run and visualize the simulation
Input
for i in range(1,100):lattice.collideAndStream()
imShow(lattice[:,:,nz/2].velocityNorm())
Output
www.lbmethod.org/palabos
Step-by-step: Instantiate bounce-back nodes
Input
lattice[10:30,10:30,10:30].defineDynamics(BounceBack(1.))for i in range(1,100):
lattice.collideAndStream()imShow(lattice[:,:,nz/2].velocityNorm())
Output
www.lbmethod.org/palabos
Step-by-step: Create a sherical obstacle
Input
lattice.defineDynamics(BGK(omega=1./tau))x,y,z = lattice.meshGrid()media = (x-10)**2 + (y-10)**2 + (z-10)**2 < 10**2pout.isoSurface(media, [1.])
Output
www.lbmethod.org/palabos
Step-by-step: Create 30 sherical obstacles
Input
nobst = 30r = 8for cx,cy,cz in zip(rand_int(nobst,nx), rand_int(nobst,ny),
rand_int(nobst,nz)):pout.display(’Sphere at position’, cx, cy, cz)media += (x-cx)**2 + (y-cy)**2 + (z-cz)**2 < r**2
pout.isoSurface(media>0, [1.])
Output
Sphere at position 4 1 21Sphere at position 86 24 0Sphere at position 60 40 35Sphere at position 38 32 16Sphere at position 28 26 15Sphere at position 67 42 3Sphere at position 45 7 22...
www.lbmethod.org/palabos
Step-by-step: Create 30 sherical obstacles
Input
nobst = 30r = 8for cx,cy,cz in zip(rand_int(nobst,nx), rand_int(nobst,ny),
rand_int(nobst,nz)):pout.display(’Sphere at position’, cx, cy, cz)media += (x-cx)**2 + (y-cy)**2 + (z-cz)**2 < r**2
pout.isoSurface(media>0, [1.])
Output
www.lbmethod.org/palabos
Theory: Limited-range interactions
They are used whenever an operation. . .
I . . . is non-local, orI . . . involves communication between different blocks.
Examples
I Non-local boundary conditions.I Coupling between components in a multi-component fluid.I Evaluation of finite-difference schemes, for example for computing velocity
gradients.
They are executed either
I automatically if they are part of the physical scheme, orI manually if they are part of a data evaluation process.
www.lbmethod.org/palabos
Step-by-step: Instantiate boundary conditions for a pressure-driven flow
Input
boundary.regularized().definePressureBC(lattice[0,:,:])lattice[0,:,:].setBoundaryDensity(1.02)boundary.regularized().definePressureBC(lattice[nx-1,:,:])lattice[nx-1,:,:].setBoundaryDensity(1.)lattice[media>0].defineDynamics(BounceBack(1.))
www.lbmethod.org/palabos
Step-by-step: Run the simulation
Input
for i in range(1,200):lattice.collideAndStream()
imShow(lattice[:,:,20].velocityNorm())
Output
www.lbmethod.org/palabos
Theory: The Palabos collision-streaming cyclelattice.collideAndStream()
The algorithm
1. Iterate once over all lattice cells1.1 Execute an in-place collision on the cell.1.2 Execute a streaming towards all neighbor nodes which are in post-collision state (no
temporary memory).2. Execute all automatic limited-range interaction operations
I Each operation has its own domain of activity (e.g. the boundary condition appliesonly to boundaries.)
www.lbmethod.org/palabos
Step-by-step: Compute the flow through a surface
Input
pout.display( lattice[media==0][nx/2,:,:].velocityComponent(0).average() )
Output
0.00511083245972
www.lbmethod.org/palabos
Theory: Two ways of computing the strain rate tensor in iso-thermal fluids
Local approach: get the stress tensor from particle populations
Π =
q−1Xi=0
cicifi
S = −12ω
ρc2sΠ
In Palabos, this is defined through the local collision rule
Non-local approach: compute velocity gradients
S =12
“∇u + (∇u)T
”In Palabos, this is defined as a limited-range operation
www.lbmethod.org/palabos
Step-by-step: Compare stress and strain-rate
Input
pout.display( lattice[media==0][nx/2,:,:].velocityComponent(0).average() )
pout.imagesc( [ lattice.velocity().strainRate().symmetricTensorNorm()[:,:,20],
lattice.strainRateFromStress().symmetricTensorNorm()[:,:,20] ] )
Output
www.lbmethod.org/palabos
The programmers interface for Palabos users
I Developing with Palabos means to implement either collision rules orlimited-range interactions.
I You do not need to be aware of parallelism while developing these ingredients.
www.lbmethod.org/palabos
Performance on a small parallel machine: Intel 8×quad-core with MPI
400 × 400 × 400 domain, D3Q19
1 2 4 8 16 32
9
17.2
31.5
60.1
114
195
Number of cores
Mill
ion
site
upd
ates
per
sec
ond
www.lbmethod.org/palabos
Performance on a large parallel machine: Blue Gene/P 16’000 cores
1000 × 1000 × 1000 domain, D3Q19
256 512 1024 2048 4096 8192 16384
228
430
790
1467
2725
4685
8056
Number of cores
Mill
ion
site
upd
ates
per
sec
ond
www.lbmethod.org/palabos
The convenient block interface hides a complicated data structure
The internal implementation offers:
I Parallelism.I Sparse-memory implementations.I Grid refinement.
www.lbmethod.org/palabos
The idea of sparse-memory implementations:
A given domain. . .
. . . is internally subdivided to remove unused regions
www.lbmethod.org/palabos
The idea of grid refinement
Example: let’s use grid refinement with the 2D lid-driven cavity
www.lbmethod.org/palabos
The idea of grid refinement
All levels of grid refinement are represented on separate layers (blocks ofdistinct size)
And all layers are put together in a common data structure