Multiscale modelling of brain bloodflow using HemeLB and Python Navier Stokes (PyNS)
1
Results Multiscale Computing with HemeLB Derek Groen 1 , James Hetherington 2 , Rupert W. Nash 1 , Miguel O. Bernabeu 1,3 , Timm Krueger 1 and Peter V. Coveney 1 1 Centre for Computational Science, University College London, UK 2 Research Software Development Team, University College London, UK 3 CoMPLEX, University College London, UK Number 4 Introduction We present a simulation technique for cerebrovascular bloodflow to investigate cerebrovascular anomalies such as brain aneurysms. Here we couple a 3D model of a cerebral arterial network to a 1D model of the remaining vasculature 1 . Additionally, we are working on including a 3D model which supports stents to simulate treated brain aneurysms. Our multiscale approach offers several advantages : ● Apply different models in areas where they provide maximal added scientific value. ● Avoid the headache of programming and debugging large hybrid simulation codes. ● Save on computational cost compared to a conventional simulation, while retaining accuracy in regions of scientific and/or clinical importance. Methods We model the flow properties of a section of the middle cerebral artery using the HemeLB simulation environment. HemeLB is an open-source lattice-Boltzmann code that allows us to construct vascular networks based on medical images, and efficiently simulate flow in sparse networks at high resolution 2 . Simulation domains in HemeLB are open vasculature systems, and as such require predefined inflow and outflow conditions. These boundary conditions are of large importance to the simulation, and in this work we use the pyNS solver 3 to help provide them. PyNS is a discontinuous Galerkin solver which contains a 1D model for the larger human arterial vessels, and resistance elements (0D) for the rest of the vasculature. inflow outflow Performance tests PyNS ● 0/1D model. ● Adaptive walls. ● Heart rate 70. ● Mean pres. 90 mmHG. ● 4k time steps. ● Cardiac output 5.98 L/min. ● Run on a local desktop. ● Exchange every time step. HemeLB ● 3D lattice-based model. ● Static walls. ● 4.2M lattice sites. ● Voxel size: 3.5x10 -5 m. ● 40k time steps. ● Run on a supercomputer. ● 512 or 2048 cores. ● Exchange every 10 steps. Local test HemeLB only acyclic acyclic cyclic cyclic Core count 4 512 512+4 2048+4 512+4 2048+4 PyNS [s] 116 - 116 116 298 157 HemeLB [s] 14365 296 296 128 298 157 Total [s] 14481 296 412 244 298 157 Speedup 1 - 35.15 59.35 48.59 Times listed here assume the use of advance reservation. We estimate the max. speedup w/o reservation to be ~40. Traction vector estimates of the upstream bifurcation during systole. See Bernabeu et al. 4 . Traction vector estimates during diastole 4 . Coupled us i n g MPWi de 5 . Applications We have applied HemeLB and pyNS to investigate the impact of blood rheology on wall sheer stress 4 , concluding that, although the choice of rheology model has an impact on the assessed wall shear stress, it is only in a specific interval where the differences may have clinical significance. Building on our multiscale approach, we are collaborating with clinicians in the UK and Qatar to compare HemeLB predictions with clinical flow measurements, and investigate the potential effects of blood pressure waveforms and flow stresses on the rupture risk of patient-specific cerebral aneurysms. Modelled using pyNS on a local node. Modelled using HemeLB on a supercomputer. Modelled using Palabos on a supercomputer. The next step We are working towards an enhanced multiscale model, which will use the Palabos code 6 to resolve a stented cerebral aneurysm (left), and using HemeLB and pyNS to respectively resolve the Circle of Willis (middle) and the remainder of the vasculature (right). References 1. Groen et al., Interface Focus 3(2), 2013. 2. HemeLB – http://ccs.chem.ucl.ac.uk/hemelb 3. Manini et al., Comp. Meth. in Biomech. and Biomed. Eng., 2012, DOI: 10.1080/10255842.2012.745857. 4. Bernabeu et al., Interface Focus 3(2), 2013. 5. MPWide – http://www.github.com/djgroen/MPWide 6. Palabos – http://www.palabos.org 2020 SCIENCE We thank our colleagues in the MAPPER consortium, the HemeLB development team at UCL, as well as Simone Manini and Luca Antiga from the pyNS development team. This work received funding from the MAPPER EU-FP7 project (RI-261507) and the CRESTA EU-FP7 project (RI-287703). We made use of a computational resources provided by EPCC in Edinburgh, UK (HECToR) and University College London (local desktops). 3 75 mil e s , 11m s r ound-trip t ime 92.24 Max. obtained speedup Here PyNS and HemeLB are run concurrently, reducing the total time to completion.
-
Upload
university-college-london -
Category
Technology
-
view
273 -
download
2
Transcript of Multiscale modelling of brain bloodflow using HemeLB and Python Navier Stokes (PyNS)
Results
Multiscale Computing with HemeLBDerek Groen1, James Hetherington2, Rupert W. Nash1, Miguel O. Bernabeu1,3, Timm Krueger1 and Peter V. Coveney1
1Centre for Computational Science, University College London, UK2Research Software Development Team, University College London, UK
3CoMPLEX, University College London, UK
Number 4
IntroductionWe present a simulation technique for cerebrovascular bloodflow to investigate cerebrovascular anomalies such as brain aneurysms. Here we couple a 3D model of a cerebral arterial network to a 1D model of the remaining vasculature1. Additionally, we are working on including a 3D model which supports stents to simulate treated brain aneurysms.Our multiscale approach offers several advantages :● Apply different models in areas where they provide maximaladded scientific value.● Avoid the headache of programming and debugging large hybrid simulation codes.● Save on computational cost compared to a conventional simulation, while retaining accuracy in regions of scientific and/or clinical importance.
MethodsWe model the flow properties of a section of the middle cerebral artery using the HemeLB simulation environment. HemeLB is an opensource latticeBoltzmann code that allows us to construct vascular networks based on medical images, and efficiently simulate flow in sparse networks at high resolution2.
Simulation domains in HemeLB are open vasculature systems, and as such require predefined inflow and outflow conditions. These boundary conditions are of large importance to the simulation, and in this work we use the pyNS solver3 to help provide them. PyNS is a discontinuous Galerkin solver which contains a 1D model for the larger human arterial vessels, and resistance elements (0D) for the rest of the vasculature.
inflow outflow
Performance testsPyNS● 0/1D model.● Adaptive walls.● Heart rate 70.● Mean pres. 90 mmHG.● 4k time steps.● Cardiac output 5.98 L/min.● Run on a local desktop.● Exchange every time step.
HemeLB● 3D latticebased model.● Static walls.● 4.2M lattice sites.● Voxel size: 3.5x105m.● 40k time steps.● Run on a supercomputer.● 512 or 2048 cores.● Exchange every 10 steps.
Local test
HemeLBonly
acyclic acyclic cyclic cyclic
Core count 4 512 512+4 2048+4 512+4 2048+4PyNS [s] 116 116 116 298 157HemeLB [s] 14365 296 296 128 298 157Total [s] 14481 296 412 244 298 157
Speedup 1 35.15 59.35 48.59
Times listed here assume the use of advance reservation. We estimate the max. speedup w/o reservation to be ~40.
Traction vector estimates of the upstream bifurcation during systole. See Bernabeu et al.4.
Traction vector estimates during diastole4.
Coupled using MPW
ide 5.
ApplicationsWe have applied HemeLB and pyNS to investigate the impact of blood rheology on wall sheer stress4, concluding that, although the choice of rheology model has an impact on the assessed wall shear stress, it is only in a specific interval where the differences may have clinical significance.
Building on our multiscale approach, we are collaborating with clinicians in the UK and Qatar to compare HemeLB predictions with clinical flow measurements, and investigate the potential effects of blood pressure waveforms and flow stresses on the rupture risk of patientspecific cerebral aneurysms.
Modelled using pyNS on a local node.
Modelled using HemeLB on asupercomputer.
Modelled using Palabos on asupercomputer.
The next stepWe are working towards an enhanced multiscale model, which will use the Palabos code6 to resolve a stented cerebral aneurysm (left), and using HemeLB and pyNS to respectively resolve the Circle of Willis (middle) and the remainder of the vasculature (right).
References1. Groen et al., Interface Focus 3(2), 2013.2. HemeLB – http://ccs.chem.ucl.ac.uk/hemelb3. Manini et al., Comp. Meth. in Biomech. and Biomed. Eng., 2012, DOI: 10.1080/10255842.2012.745857.4. Bernabeu et al., Interface Focus 3(2), 2013.5. MPWide – http://www.github.com/djgroen/MPWide6. Palabos – http://www.palabos.org
2020SCIENCE
We thank our colleagues in the MAPPER consortium, the HemeLB development team at UCL, as well as Simone Manini and Luca Antiga from the pyNS development team. This work received funding from the MAPPER EUFP7 project (RI261507) and the CRESTA EUFP7 project (RI287703). We made use of a computational resources provided by EPCC in Edinburgh, UK (HECToR) and University College London (local desktops).
375 miles, 11m
s roundtrip time
92.24Max. obtained speedup
Here PyNS and HemeLB arerun concurrently, reducing thetotal time to completion.
