Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of...
Transcript of Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of...
![Page 1: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/1.jpg)
Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA
Feiguo Chen, Wei Ge & Jinghai Li
December, 2009 Stanford, CA
Many-core and Accelerator-based Computing forPhysics and Astronomy Applications
State Key Laboratory of Multi-Phase Complex System,Institute of Process Engineering, Chinese Academy of Sciences
![Page 2: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/2.jpg)
2Institute of Process Engineering, Chinese Academy of Sciences
OutlineBackground: GPU computing & MD
GPU-based discrete algorithm
Applications• lid-driven cavity flow•multiphase flow•vaporization of liquid
Conclusions
![Page 3: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/3.jpg)
3Institute of Process Engineering, Chinese Academy of Sciences
FLOPS and Memory Bandwidth for CPU and GPU
GPU computingGPU: much higher performance
(NVIDIA,2008)
![Page 4: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/4.jpg)
4Institute of Process Engineering, Chinese Academy of Sciences
GPU computing
(pictures from internet)
Intel Nehalem CPU 8 cores
nVIDIA G200 GPU 240 cores !!!
936 Gflops (SP)
vs
GPU computing:
high performance
high efficiency
low power consumption
low cost
easy to expand
difficult to handle
CUDA ...chip die
![Page 5: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/5.jpg)
5Institute of Process Engineering, Chinese Academy of Sciences
CUDA: Compute Unified Device Architecture
GPU computing
(NVIDIA,2007)
GPU programming model
“CPU” painting robot
smile face
“GPU” painting robot
Mona Lisa
(videos from YouTube)
![Page 6: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/6.jpg)
6Institute of Process Engineering, Chinese Academy of Sciences
Discrete methods
microscopic scaleMD, DSMC, PPM ...
mesoscopic scaleLBM, DPD...
macroscopic scaleSPH, DEM, FPM, MaPPM ...
_ (rij)i
jrij
![Page 7: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/7.jpg)
7Institute of Process Engineering, Chinese Academy of Sciences
Discrete methods
i
jrij
rc
Molecular dynamics
-10
-5
0
5
10
0.5 1.0 1.5 2.0 2.5 3.0-2
-1
0
1
2
Po
ten
tia
l !
Distance r/"
!
re
Fo
rce f
f
f != "#pair-molecular interaction,
short rangetruncated interaction
interacting distancecutoff scheme
saving computing time
![Page 8: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/8.jpg)
8Institute of Process Engineering, Chinese Academy of Sciences
GPU-based discrete algorithmStart
Update molecular position and velocity
Update the map of molecules to cells
Compute the forces between molecules
Is iteration
completed ?
Finish
Update molecular velocity
Yes
No
Copy information from host memory
to device memory
Copy information from device memory to
host memory and output to files
Read parameters and molecular
information from files
Create the map of molecules to cells
Imp
lem
ent o
n G
PU
linked cell
thread block
stream MP
computing scheme:
![Page 9: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/9.jpg)
9Institute of Process Engineering, Chinese Academy of Sciences
GPU-based discrete algorithm
Pi, j
Pi, j-1
Pi, j+1
Pi+1, jPi-1, j
Pi-1, j-1
Pi-1, j+1 Pi+1, j+1
Pi+1, j-1
particle (thread)
region sub-region cell(block)MPI CUDA
CPU GPU
multilevel parallel mode: inter-GPUs & inner-GPU
![Page 10: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/10.jpg)
10Institute of Process Engineering, Chinese Academy of Sciences
GPU-based discrete algorithmStart
Update molecular position and velocity
Update the map of molecules to cells
Compute the forces between molecules
Is iteration completed ?
Finish
Update molecular velocity
Yes
No
Copy information from host memory to device memory
Copy information from device memory to host memory and output to files
Read parameters and molecular information from files
Create the map of molecules to cells
Parallel partition
Transfer of molecules between processes
Start
Update molecular position and velocity
Update the map of molecules to cells
Compute the forces between molecules
Is iteration completed ?
Finish
Update molecular velocity
Yes
No
Copy information from host memory to device memory
Copy information from device memory to host memory and output to files
Read parameters and molecular information from files
Create the map of molecules to cells
GPU CPU
GPU CPU
PCI-E
PCI-E
MPI
shift communication mode
GPU-CPU-CPU-GPUtransferring
![Page 11: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/11.jpg)
11Institute of Process Engineering, Chinese Academy of Sciences
Performance
11.821055.091054.830.1060.074CPU, rc=2063.1114.8316.7916.570.1440.042GPU, rc=20
31.665.490.480.46GPU, force
12.0729.0714.56CPU, force18.137.190.810.115*0.450.2060.037GPU+MPI21.644.300.680.460.1460.042GPU
12.0514.6714.560.0460.036CPU
SpeedupPerform
.Gflops
Totals/timestep
Comm.s/timestep
Forces/timestep
Mappings/timestep
Movements/timestep
Tesla C870 GPU vs one core of Xeon [email protected] CPU7.29×105 molecules/(CPU or GPU) in 100×100×100 region (n=0.8, T=1)
GPU achieves the speedup of 20~60x over CPU !!
* gigabit ethernet
![Page 12: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/12.jpg)
12Institute of Process Engineering, Chinese Academy of Sciences
Application-- lid-driven cavity flowV
0.37140.4318m/s68nm×68nm×34nm861m/s2.08×10-3P96K1350kg/m3dimensionalvalue
0.37140.42200×200×1005.422.280.80.8dimensionlessvalue
MaReUL×H×DcµTρ
![Page 13: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/13.jpg)
13Institute of Process Engineering, Chinese Academy of Sciences
Three-dimensional structure of the primaryeddy, as seen from different angles
![Page 14: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/14.jpg)
14Institute of Process Engineering, Chinese Academy of Sciences
(a) Z/D=0.5 (b) Z/D=0.25 (c) Z/D=0.05
Velocity vector profiles at Z/D=0.5, 0.25, 0.05
![Page 15: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/15.jpg)
15Institute of Process Engineering, Chinese Academy of Sciences
Velocity vector profiles
X/W=0.25, 0.5, 0.75 Y/H=0.25, 0.5, 0.75
MD
CFD
![Page 16: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/16.jpg)
16Institute of Process Engineering, Chinese Academy of Sciences
Application-- micro-scale multiphase flow
Initial cubic bubble (t=0) relaxes to (nearly)spherical shape at equilibrium (t=200)
Multiphase MD simulation: gas-liquid relaxation
![Page 17: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/17.jpg)
17Institute of Process Engineering, Chinese Academy of Sciences
(a) t=0 (b) t=300 (c) t=500 (d) t=900 (e) t=1,250
Snapshots of gas-liuqid-solid MD simulation at different times
Multiphase MD simulation: particle-bubble interaction in liquid flow
![Page 18: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/18.jpg)
18Institute of Process Engineering, Chinese Academy of Sciences
CPUGPU: from Angstroms toMicrons
CPU: ~104 2D moleculesnano-scale
GPUs: ~108 3D molecules(sub)micro-scale
![Page 19: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/19.jpg)
19Institute of Process Engineering, Chinese Academy of Sciences
Application-- vaporization of liquid
vaporization with openboundary conditions
vaporization with reboundingboundary conditions
![Page 20: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/20.jpg)
20Institute of Process Engineering, Chinese Academy of Sciences
Conclusion & Prospect
Molecular dynamics simulation has wideapplications on micro-scale flow.
GPU-based parallel computation mayprovide a feasible way to bring the methodinto practical use.
More powerful and straightforward GPUcomputing is desired.
Fermi & CUDA 3.0 ?
![Page 21: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/21.jpg)
21Institute of Process Engineering, Chinese Academy of Sciences
ProblemsCUDA memory allocation
Sample code: float *d_M1, *d_M2, *d_M3; … cudaMalloc((void**) &d_M1, mem_size)); … cudaMalloc((void**) &d_M2, mem_size)); … cudaMalloc((void**) &d_M3, mem_size));
time cost
t1
t2
t3
>>=
~102 ms
~1 ms
Why? Are there solutions to shorten this time?The same problem exists in the case of cudaMallocHost() function.
![Page 22: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/22.jpg)
22Institute of Process Engineering, Chinese Academy of Sciences
Problems
F(rij)i
jrij
i1
i2
i3
i4
i5i6
i7 i8
i9
j
i10
ji
ij
Fi=Fi + Fij
Fj=Fj - Fij
Fin=Fin + Fin,j
Fj=Fj - Fin,j
Applying Newton’s 3rd Lawserial computing parallel computing
atomic op. ?
w/o 3rd law?
?
![Page 23: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/23.jpg)
23Institute of Process Engineering, Chinese Academy of Sciences
ProblemsGPU memory DMA ?
GPU CPU
GPU CPU
PCI-E
PCI-E
MPI
GPU-CPU-CPU-GPUtransferring
GPU CPU
GPU CPU
PCI-E
PCI-E
MPI
DMAtransferring
![Page 24: Molecular dynamics simulation of multi- phase flow on ... · Molecular dynamics simulation of multi-phase flow on micro-scale using CUDA Feiguo Chen, Wei Ge & Jinghai Li ... -- lid-driven](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0a19117e708231d42a018a/html5/thumbnails/24.jpg)
Thanks for your attention!