Virtual Rheoscopic Fluids for Dense, Large-Scale …web.cse.ohio-state.edu › ~machiraju.1 ›...

6
Virtual Rheoscopic Fluids for Dense, Large-Scale Fluid Flow Visualizations Paul A. Navr´ atil * William L. Barth * Texas Advanced Computing Center Hank Childs Lawrence Berkeley National Laboratory ABSTRACT Fluid flow visualizations typically employ a combination of isosur- facing and particle tracing, such as streamlines. These techniques capture surface flow well, but they can obscure deeper features from view. Further, these techniques present visualizations that are far removed from physical methods, which complicates their analy- sis. However, virtual rheoscopic fluids (VRF) provide a physically- based visualization that is similar to, and can be similarly analyzed as, physical rheoscopic fluid. However, we need a scalable VRF algorithm to apply the technique to large-scale data. In this paper, we present a scalable VRF algorithm based on ray casting implemented in a general visualization toolkit. We demon- strate the application of these techniques to large-scale flow prob- lems beyond what would be physically feasible in the laboratory, and we show that our approach scales through 256 cores. Keywords: Virtual rheoscopic fluid, Fluid flow 1 I NTRODUCTION Visualization of fluid flow typically uses non-realistic methods, such as streamlines or vector glyphs. While researchers have be- come used to interpreting such images, they are removed from our everyday experience. Physically-based methods, however, mimic lab methods and can be analyzed in the same way. Simulated dye or rheoscopic fluid reveal both complex flow structures and sub- surface flow more readily than non-realistic methods. Rheoscopic fluid in particular is useful for visualization since the suspended particles do not dissipate through a fluid like a dye. The particles align with fluid flow, becoming more opaque as alignment increases. Because the structures are typically semi-opaque, it is possible to see into the fluid to identify deeper structures. However, physical rheoscopic fluid is only used for lab experiments, as it is impractical to use in larger fluid experiments. Virtual rheoscopic fluid (VRF) can be used to simulate large-scale flows that are im- practical or impossible to study physically. Unfortunately, previ- ous virtual rheoscopic fluid efforts [6, 14] are custom single-node, GPU-based implementations, which were not built for generality or scalability. In this paper, we adapt the two virtual rheoscopic fluid algo- rithms in the literature for use in a general-purpose visualization pipeline. By this transformation, we are able to leverage a par- allel visualization pipeline to study large-scale flows. Our algo- rithms demonstrate up to 185% scaling efficiency through 256 cores, and yield visualizations that more clearly identify structures within flows than non-realistic methods. The contributions of this paper are: a scalable VRF algorithm usable within a ray casting frame- work; and * Email: [ pnav | bbarth ] @tacc.utexas.edu. Email: [email protected]. a demonstration of this algorithm within VisIt [9], a general VTK-based visualization toolkit. The remainder of the paper is as follows: we discuss related work in Section 2; we present our new algorithm in Section 3; we give our methodology in Section 4; we show our results in Section 5; and in Section 6 we offer future work and conclude. 2 RELATED WORK In this section, we place our approach in the context of prior work, describe previous virtual rheoscopic fluid approaches, and survey previous work in parallel ray-casting, which is computationally similar to our virtual rheoscopic fluid generation. 2.1 Virtual Rheoscopic Fluid Recent papers by Barth and Burns [6] and Hecht et al. [14] describe the implementation of two different methods for computer simula- tion of VRFs using volume rendering with ray casting and tracing methods on a GPU. Both methods employ a local orientation direc- tion derived from the flow velocity field rather than the traditional gradient of a scalar field. Barth and Burns choose their orientation vector from the resul- tant sheer force in the velocity direction by removing the compo- nent parallel to the velocity. This orientation is given by n =2Du ˆ u - 2 ˆ u T Du ˆ u ˆ u (1) where ˆ u is the velocity and Du is the symmetric part of the ve- locity gradient tensor. Their method gives a local orientation vector everywhere in the flow field and requires the computation of the velocity gradient tensor. This orientation choice is ad hoc and jus- tified through an appeal to idealized microscopic platelets seeded homogeneously throughout the flow field. Hecht et al. select their orientation based on a physical model for the long-time behavior of axis-symmetric and triaxial particles seeded in the flow field. This model presents a time-dependent ODE representation of the particle major and minor axes based on the gradient of the velocity field. The long-time behavior of these particles is then governed by the eigenvalues and eigenvectors of the velocity gradient tensor. There are three relevant cases to consider in the eigenanalysis: three real, unequal eigenvalues; a positive-real-part complex conju- gate pair; and a negative-real-part complex conjugate pair. In the first two cases, one of the eigenvectors yields the desired orienta- tion, however, the negative-real-part complex conjugate case repre- sents a constantly tumbling particle without a well-defined orienta- tion. This latter case requires special treatment in their implemen- tation, and leads to problems in implementation in software volume rendering pipelines as described below. 2.2 Ray-Casting From a computational perspective, virtual rheoscopic fluid render- ing is highly similar to ray-casted volume rendering [19]. Rays are cast for every pixel on the screen, with samples calculated along each ray. Rheoscopic and volume rendering differ only in the types of data evaluated at the samples and how color and transparency 79 IEEE Symposium on Large Data Analysis and Visualization 2012 October 14 - 15, Seattle, Washington, USA 978-1-4673-4733-4/12/$31.00 ©2012 IEEE

Transcript of Virtual Rheoscopic Fluids for Dense, Large-Scale …web.cse.ohio-state.edu › ~machiraju.1 ›...

Page 1: Virtual Rheoscopic Fluids for Dense, Large-Scale …web.cse.ohio-state.edu › ~machiraju.1 › teaching › CSE5544 › ...Virtual Rheoscopic Fluids for Dense, Large-Scale Fluid Flow

Virtual Rheoscopic Fluids for Dense, Large-Scale Fluid FlowVisualizations

Paul A. Navratil∗ William L. Barth∗

Texas Advanced Computing CenterHank Childs†

Lawrence Berkeley National Laboratory

ABSTRACT

Fluid flow visualizations typically employ a combination of isosur-facing and particle tracing, such as streamlines. These techniquescapture surface flow well, but they can obscure deeper features fromview. Further, these techniques present visualizations that are farremoved from physical methods, which complicates their analy-sis. However, virtual rheoscopic fluids (VRF) provide a physically-based visualization that is similar to, and can be similarly analyzedas, physical rheoscopic fluid. However, we need a scalable VRFalgorithm to apply the technique to large-scale data.

In this paper, we present a scalable VRF algorithm based on raycasting implemented in a general visualization toolkit. We demon-strate the application of these techniques to large-scale flow prob-lems beyond what would be physically feasible in the laboratory,and we show that our approach scales through 256 cores.

Keywords: Virtual rheoscopic fluid, Fluid flow

1 INTRODUCTION

Visualization of fluid flow typically uses non-realistic methods,such as streamlines or vector glyphs. While researchers have be-come used to interpreting such images, they are removed from oureveryday experience. Physically-based methods, however, mimiclab methods and can be analyzed in the same way. Simulated dyeor rheoscopic fluid reveal both complex flow structures and sub-surface flow more readily than non-realistic methods.

Rheoscopic fluid in particular is useful for visualization since thesuspended particles do not dissipate through a fluid like a dye. Theparticles align with fluid flow, becoming more opaque as alignmentincreases. Because the structures are typically semi-opaque, it ispossible to see into the fluid to identify deeper structures. However,physical rheoscopic fluid is only used for lab experiments, as it isimpractical to use in larger fluid experiments. Virtual rheoscopicfluid (VRF) can be used to simulate large-scale flows that are im-practical or impossible to study physically. Unfortunately, previ-ous virtual rheoscopic fluid efforts [6, 14] are custom single-node,GPU-based implementations, which were not built for generality orscalability.

In this paper, we adapt the two virtual rheoscopic fluid algo-rithms in the literature for use in a general-purpose visualizationpipeline. By this transformation, we are able to leverage a par-allel visualization pipeline to study large-scale flows. Our algo-rithms demonstrate up to 185% scaling efficiency through 256cores, and yield visualizations that more clearly identify structureswithin flows than non-realistic methods.

The contributions of this paper are:

• a scalable VRF algorithm usable within a ray casting frame-work; and

∗Email: [ pnav | bbarth ] @tacc.utexas.edu.†Email: [email protected].

• a demonstration of this algorithm within VisIt [9], a generalVTK-based visualization toolkit.

The remainder of the paper is as follows: we discuss related workin Section 2; we present our new algorithm in Section 3; we giveour methodology in Section 4; we show our results in Section 5;and in Section 6 we offer future work and conclude.

2 RELATED WORK

In this section, we place our approach in the context of prior work,describe previous virtual rheoscopic fluid approaches, and surveyprevious work in parallel ray-casting, which is computationallysimilar to our virtual rheoscopic fluid generation.

2.1 Virtual Rheoscopic FluidRecent papers by Barth and Burns [6] and Hecht et al. [14] describethe implementation of two different methods for computer simula-tion of VRFs using volume rendering with ray casting and tracingmethods on a GPU. Both methods employ a local orientation direc-tion derived from the flow velocity field rather than the traditionalgradient of a scalar field.

Barth and Burns choose their orientation vector from the resul-tant sheer force in the velocity direction by removing the compo-nent parallel to the velocity. This orientation is given by

n = 2Duu− 2(uTDuu

)u (1)

where u is the velocity and Du is the symmetric part of the ve-locity gradient tensor. Their method gives a local orientation vectoreverywhere in the flow field and requires the computation of thevelocity gradient tensor. This orientation choice is ad hoc and jus-tified through an appeal to idealized microscopic platelets seededhomogeneously throughout the flow field.

Hecht et al. select their orientation based on a physical modelfor the long-time behavior of axis-symmetric and triaxial particlesseeded in the flow field. This model presents a time-dependentODE representation of the particle major and minor axes based onthe gradient of the velocity field. The long-time behavior of theseparticles is then governed by the eigenvalues and eigenvectors ofthe velocity gradient tensor.

There are three relevant cases to consider in the eigenanalysis:three real, unequal eigenvalues; a positive-real-part complex conju-gate pair; and a negative-real-part complex conjugate pair. In thefirst two cases, one of the eigenvectors yields the desired orienta-tion, however, the negative-real-part complex conjugate case repre-sents a constantly tumbling particle without a well-defined orienta-tion. This latter case requires special treatment in their implemen-tation, and leads to problems in implementation in software volumerendering pipelines as described below.

2.2 Ray-CastingFrom a computational perspective, virtual rheoscopic fluid render-ing is highly similar to ray-casted volume rendering [19]. Rays arecast for every pixel on the screen, with samples calculated alongeach ray. Rheoscopic and volume rendering differ only in the typesof data evaluated at the samples and how color and transparency

79

IEEE Symposium on Large Data Analysis and Visualization 2012October 14 - 15, Seattle, Washington, USA978-1-4673-4733-4/12/$31.00 ©2012 IEEE

Page 2: Virtual Rheoscopic Fluids for Dense, Large-Scale …web.cse.ohio-state.edu › ~machiraju.1 › teaching › CSE5544 › ...Virtual Rheoscopic Fluids for Dense, Large-Scale Fluid Flow

is decided for each sample. To tackle efficient parallelization ofthis computational workload, Ma and Crockett introduced a paral-lel sort-last cell-projection algorithm [20]. Their concept was laterextended by Childs, Duchaineau and Ma [10] to guarantee fixedupper bounds on performance regardless of camera position. Thislatter system, implemented in VisIt [9], was adapted for the studypresented in this paper.

Multiple studies have shown that parallel ray-casting techniquescan be successfully scaled up to high levels of concurrency andwork on very large data sets. Peterka et al. [21, 22] demonstratedscaling characteristics up to 32,768 cores. Howison et al. [16, 15]performed a separate study, designed to measure the benefits of hy-brid parallelism, and considering scalability up to 216,000 cores.Finally, the VisIt system, including its volume renderer, was shownto perform well with data sets consisting of trillions of cells per timeslice using tens of thousands of cores [11]. All of these findingsprovide evidence that the computations required for virtual rheo-scopic fluid rendering on large data sets can be addressed throughparallelism.

3 ALGORITHM OVERVIEW

This section describes the specific challenges for integration ofVRF algorithms into general visualization pipelines and our newalgorithms to work within those constraints.

3.1 Visualization Pipelines

We first describe our approach in a serial environment and then de-scribe how it extends to a parallel setting.

Our approach fits well within a data flow network design [23, 24,17, 2], and we visualize virtual rheoscopic fluids by constructing apipeline of filters. One of these filters constructs a gradient field thatis used for surface lighting when we do ray-casting. Arbitrary filterscan be inserted in the pipeline, of course: users can opt to limit thevolume based on spatial clips, thresholding out regions that containuninteresting values, etc. Once the pipeline is constructed, it is ex-ecuted, creating the final volume for virtual rheoscopic rendering.Each rendering step operates on this resulting data set. Ray-castingwork occurs once per render, but the pipeline is executed only onetime overall.

The rendering is done with a sampling-based ray-caster. Hun-dreds to thousands of samples are evaluated along each ray, calcu-lating the value of one or more fields, transforming each sample toa color and opacity, and then compositing along a ray to get a finalpixel color. When doing a standard volume rendering, a scalar fieldis sampled and a transfer function is applied to find color and opac-ity. For virtual rheoscopic fluids, a velocity field is sampled and adifferent transformation is used to determine color and opacity (seesections 3.2 and 3.3 to learn more about this transformation).

Our parallelization strategy depends on data parallelism. Eachprocessing elements loads, processes, and renders a piece of thelarger data set. The processing elements instantiate identical dataflow networks; the only thing differentiating the data flow networksis the piece they operate on. The data parallel approach requiresparallel coordination when ray-casting, however. After each pro-cessing element samples its portion of the larger data set, the sam-ples for any ray are likely distributed over many processing ele-ments, preventing compositing. To account for this, the pixels arepartitioned over the processing elements and the samples are redis-tributed so that each processing element has all of the samples forits pixels. Again, our approach evolved from the volume renderingray-caster described by Childs et al. [10], which means that it candeal with unstructured meshes and makes no assumptions about theordering of pieces or that the pieces can be ordered from front-to-back for a given camera location.

3.2 Tensor-Based VRFTo implement tensor-based VRF in the pipeline described above,we modified Barth and Burns’s algorithm [6] to function withinVisIt’s volume rendering pipeline. We first compute a normal ateach data point using Equation 1; we then compute an opacity ateach sample by taking the dot product of the generated normal andthe view vector. This opacity is used to compute each sample in thedirect volume renderer, described above, and the sample is shadedby any direct lighting. The samples are then composited front-to-back, with the later samples attenuated by the opacity of the onesbefore them.

3.3 Eigen-Based VRFTo implement eigen-based VRF, we followed the same model asfor the tensor-based VRF, but with additional modifications to therendering pipeline to accommodate the three cases Hecht et al. de-scribe [14]. We use the GNU Scientific Library (GSL) v1.15 [13] toperform the eigenanalysis, the result of which can fall into three cat-egories. The first two cases (where the particle is stable and wherethe particle is rotating around the minor axis) generate a normal thatwe use to compute the opacity in a way similar to the tensor-basedmethod. The third case (where the rheoscopic particle is spinningalong the major axis), however, has no constant normal and insteadgenerates an intensity. This return value type conflict causes twocomplications when being implemented in a general visualizationpipeline:

• Normals and intensities are generated at different parts of thepipeline, so the intermediate value from the eigenanalysis (themajor axis from the third case, computed at normal genera-tion) must be stored until the sampling phase, where intensityis calculated.

• Normal generation occurs before sampling, so the normal ata particular sample point is interpolated from the surroundinggenerated normals; since the third case does not return a nor-mal, the values for those cases are represented as zeros andtheir interpolation causes a visual artifact between data in thefirst two cases and data in the third case (see Figure 1).

Hecht et al. pre-processed their data and customized their visu-alization pipeline to avoid these artifacts, but doing so in an estab-lished, general visualization pipeline would put special cases insidethe basic ray-casting mechanism and affecting all visualizations thatuse the ray casting code.

In the ray casting sampler, we calculate the opacity using the dotproduct of the light direction and the generated normal, if there isone. Otherwise, we use the stored major axis to generate an inten-sity using the uniform rotation equation in [14]:

Irot = A · sinα2π

∫ 2π

0

| sin θ | dθ = A · 2 sinαπ

(2)

We use this intensity as the opacity for the sample.

4 METHODOLOGY

This section describes our experimental methodology, including thehardware platform, the datasets, and the rendering methods that weused to evaluate our scheduling strategy.

4.1 System ConfigurationAll experiments are run on Longhorn, a 2048 core, 256 node clus-ter hosted at the Texas Advanced Computing Center. Each nodecontains two four-core Intel Xeon E5540 “Gainestown” proces-sors and 48 GB of local RAM. All nodes are connected by a Mel-lanox QDR InfiniBand switch, and we use the MVAPICH2 v1.4

80

Page 3: Virtual Rheoscopic Fluids for Dense, Large-Scale …web.cse.ohio-state.edu › ~machiraju.1 › teaching › CSE5544 › ...Virtual Rheoscopic Fluids for Dense, Large-Scale Fluid Flow

Figure 1: The tensor- (left) and eigen-based (right) VRF methods ona toy dataset designed to emphasize the discontinuities in the eigenmethod.

Figure 2: The wavy Taylor vortex flow created by Barth’s and Burns’stexture-based method (left) and our direct volume rendering method(right).

MPI implementation. The Breaking Waves 1B scaling runs wererun on Lonestar large memory nodes, each with four six-core In-tel Xeon E7540 “Westmere” processors and 1 TB of local RAM.These nodes are connected by a Mellanox QDR InfiniBand switch,and we use MVAPICH2 v1.6 MPI implementation. Our algorithmsare implemented within VisIt [9], a visualization toolkit designed tooperate in parallel on large-scale data. We use the VisIt infrastruc-ture to load data and to render the final images; we implemented allcode related to the generation of VRF normals and final opacities.

4.2 DatasetsWe use four datasets in our experiments: Taylor-Couette flow, in-ternal splash, thermal hydraulics and breaking waves. A chart withthe details for each dataset can be found in Table 1.

Table 1: Dataset Details

Dataset timesteps nodes / step size / step total sizeTaylor

Couette 3224 494 K 167 MB 524 GBInternalSplash 1 432 K 14 MB 14 MB

ThermalHydraulics 100 32 M 673 MB 67 GB

BreakingWaves 600 16 M 129 MB 76 GB

BreakingWaves 1B 10 1 B 10.7 GB 102 GB

4.2.1 Taylor-Couette FlowFlow between concentric rotating cylinders is a classic problemin fluid mechanics (see, e.g. [4, 18] for review of the early liter-ature). Much experimental work has been devoted to discoveringand classifying the flow states and the instabilities that drive them(see, e.g. [4] where Kalliroscope was used to visualize flow fea-tures and, thereby, classify the flow regimes). The simulations ofTaylor-Couette flow used in this work are generated with MGF—a3D, parallel, finite element simulation code developed for the studyof fluid flow and heat transfer problems and now extended to manyother classes of partial differential equation systems [7, 5, 8].

4.2.2 Internal SplashThe second dataset we utilize is the same “internal splash” phe-nomena originally described by Abaid et al. [1] and used by Hechtet al. [14] in their virtual rheoscopic fluid work. One timestep oftheir dataset was provided directly to us for comparison purposes.This dataset models the flow of a heavier-than-fluid ball that passesthrough the interface between a light and heavy fluid.

4.2.3 Thermal HydraulicsIn the thermal hydraulics data set, twin inlets pump air into a box,with a temperature difference between the inlets. The air mixes inthe box and exits through an outlet. Mixing of “hot” and “cold” air,residence time in the box, and identification of both stagnant andhighly turbulent regions are areas of active study. The simulationwas performed using the NEK5000 code [12] with an unstructuredgrid comprised of twenty-three million hexahedral elements.

4.2.4 Breaking WavesThe breaking waves dataset simulates the waves generated by aship’s hull passing through water. As the hull accelerates, the waves“break” and spray sheets from off the hull. The complex interac-tions among the hull, water and air create turbulence that can inter-fere with a ship’s smooth travel through water. These simulationsallow careful study of these phenomena, since such studies are dif-ficult to perform in situ. This particular dataset shows a simulatedwater flow around a NACA 0024 airfoil, previously used by Adamsand Dommermuth [3]. We also use a one billion point supersam-pled version of this dataset.

5 RESULTS

This section evaluates our virtual rheoscopic fluid algorithm on thedatasets described in Section 4. First, we discuss the visual dif-ferences between the tensor and eigen methods. We then focus onthe tensor method’s utility on larger datasets, where use of physicalrheoscopic fluid is impractical. Finally, we demonstrate the strongscaling of our technique on these larger sets.

5.1 Rendering ComparisonsIn Section 3, we discussed the differences between the tensor- andeigen-based approaches. Figure 1 demonstrates this difference vi-sually with a toy dataset designed to do so. The tensor version iscontinuous, whereas the eigen version has discontinuities betweenthe cases that return a normal and the case that returns an intensity(see Section 3.3). These artifacts cannot be repaired unless a con-ditional is put deep within the rendering pipeline, which defeats thepurpose of implementing these techniques in a general visualiza-tion package. For the remainder of the results, we concentrate ontensor-based VRF.

5.1.1 Verification DatasetsWe verify our tensor VRF implementation against the Taylor-Couette images presented by Barth and Burns [6]. Figure 2 com-pares our direct volume VRF visualization of wavy Taylor vortex

81

Page 4: Virtual Rheoscopic Fluids for Dense, Large-Scale …web.cse.ohio-state.edu › ~machiraju.1 › teaching › CSE5544 › ...Virtual Rheoscopic Fluids for Dense, Large-Scale Fluid Flow

Figure 3: Four versions of the internal splash dataset. The imagesproduced by Hecht et al. are on the left, our images are on the right.Eigen-based images are on top, tensor-based images are on bottom.

flow to Barth’s and Burns’s texture-based visualization. These im-ages are quite similar despite the differences in implementation.

We also verify our tensor and eigen VRF implementationsagainst the internal splash dataset used by Hecht et al. [14]. The im-ages in Figure 3 compare both our methods against the versions pre-sented by Hecht et al. The tensor-based visualizations look similar,though the eigen-based images are visually different. The featuresare similar, though the shading is clearly different. The differenceis likely caused by different light positions (as the eigen techniqueis chiefly sensitive to the light location) and different values for thealbedo of the VRF particles.

5.1.2 Large Physical Datasets

To demonstrate our ability to visualize simulations of large physicalphenomena, we use a simulation of air flow in a room (thermalhydrodynamics) and a simulation of a ship hull passing throughwater (breaking waves). Both of these datasets model situations thatwould be difficult to recreate with traditional rheoscopic fluid. Wecompare our visualizations against traditional vector visualizationtechniques.

Figure 4 shows our visualization of the thermal hydrodynamicsdataset compared to a streamline representation. The VRF showsfine-grained details of the flow, and the inherent transparency re-veals internal structures not visible in the streamline image. In addi-tion, the animation of the VRF representation more clearly capturesthe flow evolution than streamlines.

We show the results for the breaking waves dataset in Figure 5.While the isosurface captures the waves well, which was the origi-nal purpose of Adams’s and Domermuth’s study [3], the VRF ver-sion better represents the complex surface and sub-surface turbu-lence throughout the dataset. If enough streamlines were includedin the image, it would obscure inner features, as in the thermal hy-draulics dataset.

5.2 ScalingWe performed strong scaling runs for both the thermal hydraulicsdataset and the up-scaled, 1 billion node breaking waves dataset.We present the scaling efficiency and the raw times of each set in

Figure 4: The thermal hydraulics dataset with tensor-based VRF(top) and streamlines (bottom). The VRF image shows finer detailsand internal flow not visible in the streamline image.

1 2 4 8 16 32 64 128 256 512Thermal

Hydraulics 340 s 130 49.3 31.9 20.7 5.78 3.11 1.79 1.16 0.979Breaking

Waves 1B 166 s 94.3 29.0 20.2 14.0 4.06 2.19 1.24 0.817 0.892

Figure 6: Strong scaling efficiency for the thermal hydraulics andup-scaled breaking waves data and the corresponding timings. Scal-ing for these datasets tails at 256 cores and 128 cores, respectively,when available parallelism has been exhausted.

82

Page 5: Virtual Rheoscopic Fluids for Dense, Large-Scale …web.cse.ohio-state.edu › ~machiraju.1 › teaching › CSE5544 › ...Virtual Rheoscopic Fluids for Dense, Large-Scale Fluid Flow

Figure 5: The breaking waves dataset with tensor-based VRF (left) and an isosurface with streamlines (right). The VRF image shows complexflows and turbulence throughout the data that neither the isosurface nor streamlines capture.

Figure 6. The hydraulics dataset shows super-linear scaling through256 cores, where the parallelism available in the dataset is ex-hausted. Whereas rendering a frame on one core takes an averageof 341 seconds, rendering on 256 cores takes one second per frame.In contrast, rendering the breaking waves dataset on one core takes28 seconds on average. At 32 cores, rendering takes 4.9 seconds onaverage, and much of the available parallelism has been claimed.Increasing the core count eight-fold to 256 only yields a 3× speed-up in runtime.

6 DISCUSSION

In this paper, we have presented a scalable algorithm for virtualrheoscopic fluids that leverages the capabilities of a general visu-alization toolkit. Our technique captures more details than tradi-tional vector flow visualization methods, and because we have im-plemented it within a scalable framework, VRFs can be used forsimulations of large physical phenomena where traditional rheo-scopic fluids cannot be used. We demonstrate that our algorithmscales to over 256 cores and is able to exploit the available paral-lelism in our test datasets.

We hope to further improve the performance of our algorithmwith improvements to the underlying volume renderer. We also planto package the VRF code to be included in VTK-based visualizationtoolkits.

ACKNOWLEDGEMENTS

We thank Nipun Kwatra, Chris Wojtan and Mark Carlson for theirwork on the internal splash dataset. We also thank Karla Vega forthe breaking waves comparison image. This work was funded inpart by U.S. National Science Foundation grant OCI-0906379; bythe Director, Office of Advanced Scientific Computing Research,Office of Science, of the U.S. Department of Energy under ContractDE-AC02-05CH11231; and by the Scientific Discovery throughAdvanced Computing (SciDAC) Institute of Scalable Data Man-agement, Analysis and Visualization.

REFERENCES

[1] N. Abaid, D. Adalsteinsson, A. Agyapong, and R. M. McLaughlin.An internal splash: Levitation of falling spheres in stratified fluids.Physics of Fluids, 16(5):1567–1580, 2004.

[2] G. Abram and L. A. Treinish. An extended data-flow architecture fordata analysis and visualization. Research report RC 20001 (88338),IBM T. J. Watson Research Center, Yorktown Heights, NY, USA, Feb.1995.

[3] P. Adams and D. Domermuth. Visualization of Steep Breaking Wavesand This Spray Sheets around a Ship. IEEE Transactions on Visual-ization and Computer Graphics, pages 555–559, 2003.

[4] C. D. Andereck, S. S. Liu, and H. L. Swinney. Flow regimes in acircular couette system with independently rotating cylinder. J. FluidMech., 4:155–183, 1986.

[5] W. L. Barth. Simulation of Non-Newtonian Fluids on WorkstationClusters. PhD thesis, The University of Texas at Austin, May 2004.

[6] W. L. Barth and C. A. Burns. Virtual rheoscopic fluids for flow visual-ization. IEEE Transactions on Visualization and Computer Graphics,13(6):1751–1758, November/December 2007.

[7] W. L. Barth and G. F. Carey. On a natural convection benchmarkproblem in non-newtonian fluids. Numerical Heat Transfer, Part B,50:193–216, 2006.

[8] G. F. Carey, R. McLay, W. Barth, S. Swift, and B. Kirk. Distributedparallel simulation of surface tension driven viscous flow and trans-port processes. In Computational Fluid Dynamics: Proceedings of theFourth UNAM Supercomputing Conference, pages 143–155. UNAM,June 2000.

[9] H. Childs, E. Brugger, B. Whitlock, J. Meredith, S. Ahern, K. Bon-nell, M. Miller, G. H. Weber, C. Harrison, D. Pugmire, T. Fo-gal, C. Garth, A. Sanderson, E. W. Bethel, M. Durant, D. Camp,J. M. Favre, O. Rubel, P. Navratil, M. Wheeler, P. Selby, andF. Vivodtzev. VisIt: An End-User Tool For Visualizing and Analyz-ing Very Large Data. In Proceedings of SciDAC 2011, July 2011.http://press.mcs.anl.gov/scidac2011.

[10] H. Childs, M. A. Duchaineau, and K.-L. Ma. A Scalable, HybridScheme for Volume Rendering Massive Data Sets. In EurographicsSymposium on Parallel Graphics and Visualization, pages 153–162,2006.

[11] H. Childs, D. Pugmire, S. Ahern, B. Whitlock, M. Howison, Prabhat,G. Weber, and E. W. Bethel. Extreme Scaling of Production Visual-ization Software on Diverse Architectures. IEEE Computer Graphicsand Applications, pages 22–31, 2010.

[12] P. Fischer, J. Lottes, D. Pointer, and A. Siegel. Petascale Algorithmsfor Reactor Hydrodynamics. Journal of Physics: Conference Series,125:1–5, 2008.

[13] M. Galassi, J. Davies, J. Theiler, B. Gough, G. Jungman, P. Alken,M. Booth, and F. Rossi. GNU Scientific Library Reference Manual —

83

Page 6: Virtual Rheoscopic Fluids for Dense, Large-Scale …web.cse.ohio-state.edu › ~machiraju.1 › teaching › CSE5544 › ...Virtual Rheoscopic Fluids for Dense, Large-Scale Fluid Flow

Third Edition. Network Theory Ltd., 2009.[14] F. Hecht, P. J. Mucha, and G. Turk. Virtual rheoscopic fluids. IEEE

Transactions on Visualization and Computer Graphics, 16(1), January2010.

[15] M. Howison, E. W. Bethel, and H. Childs. Hybrid Parallelism forVolume Rendering on Large-, Multi-, and Many-Core Systems. IEEETransactions on Visualization and Computer Graphics, 18(1):17–29,2012.

[16] M. Howison, E.Wes Bethel, and H. Childs. MPI-Hybrid Parallelismfor Volume Rendering on Large, Multi-Core Systems. In Eurograph-ics Symposium on Parallel Graphics and Visualization, 2010.

[17] C. Johnson, S. Parker, and D. Weinstein. Large-scale computationalscience applications using the SCIRun problem solving environment.In Proceedings of the 2000 ACM/IEEE conference on Supercomput-ing, 2000.

[18] E. L. Koschmieder. Benard Cells and Taylor Vortices. CambridgeUniversity Press, New York, NY, 1993.

[19] M. Levoy. Display of Surfaces from Volume Data. IEEE ComputerGraphics and Applications, 8(3):29–37, 1988.

[20] K.-L. Ma and T. Crockett. A scalable parallel cell-projection volumerendering algorithm for 3d unstructured data. In Proc. 1997 Sympo-sium on Parallel Rendering, pages 95–104, 1997.

[21] T. Peterka, H. Yu, R. Ross, and K.-L. Ma. Parallel volume render-ing on the ibm blue gene/p. In Proceedings of Eurographics ParallelGraphics and Visualization Symposium 2008, Crete, Greece, 2008.

[22] T. Peterka, H. Yu, R. Ross, K.-L. Ma, and R. Latham. End-to-EndStudy of Parallel Volume Rendering on the IBM Blue Gene/P. In Pro-ceedings of International Conference on Parallel Processing, pages566–573, 2009.

[23] W. J. Schroeder, K. M. Martin, and W. E. Lorensen. The design andimplementation of an object-oriented toolkit for 3d graphics and visu-alization. In VIS ’96: Proceedings of the 7th conference on Visualiza-tion ’96, pages 93–ff. IEEE Computer Society Press, 1996.

[24] C. Upson, T. F. Jr., D. Kamins, D. H. Laidlaw, D. Schlegel, J. Vroom,R. Gurwitz, and A. van Dam. The application visualization system:A computational environment for scientific visualization. ComputerGraphics and Applications, 9(4):30–42, July 1989.

84