Lecture 6a

Volume Visualization Part II

-GPU Based Volume Rendering

Outline

Graphic Hardware

GPU based Marching Cubes Algorithm

GPU based Ray-casting Algorithm

Texture-Based Volume Rendering

2D-Textured Object-aligned slicing

3D-Textured View-aligned slicing

Common Volume Rendering Software

Graphic Hardware

Accelerate rendering of a virtual scene modeled

by planar polygons

Convert polygons to raster image as a fixed

sequence of processing stages

Ordering of operations is described as a graphics

pipeline

Graphics Processing Unit (GPU)

A dedicated device to manipulate and display computer

graphics

Graphic Pipeline

Scene

Description

Geometric

Processing

Rasterization Fragment

Operation

Raster

ImageProgrammable Pipeline

Vertex

Shader

Fragment

Shader

Vertex Shader

Per-vertex calculations performed here

Without knowledge about other vertices

(parallelism)

Responsibilities

Vertex and normal transformation

Blur and (Per-Vertex) Lighting

Color material application and color clamping

Texture coordinate generation

Set up data for fragment shaders

Fragment Shader

Perform per-pixel calculations

Without knowledge about other fragments

(parallelism)

Can think of a fragment as a potential pixel

Responsibilities and opportunities

Operations on interpolated values

Texture access and application

Fog and color lookup

Assign each fragment a color at the end

GPU Based Marching Cube Background

Marching cubes (MC) and its variants MC: Lorenson and Cline 1987

MT: Doi and Koide 1991

MC 33: Chernyaev 1995

MC *: Nielson 2003

MD: Anderson at el. 2005

GPU-based isosurface rendering MT: Reck at el. 2004

Ray casting: Hadwiger at el. 2005

GPU-based isosurface extraction and rendering MT: Kipfer and Westermann 2005

Klein at el. 2004

GPU Based Marching Cube Background

Traditional pipeline

GPU

System Memory

CPU

CellSelection

Vertex labeling

Cell Indexing

VertexInterpolation

NormalComputation

Rendering

Dataset

Acc. Data Structure

GPU-Friendly Marching Cubes

Parallel Viusalization of Multiple Translucent

Isosurfaces---By Y. M. Xie, G. Y. Wang, T. T. Wong and P. A. Heng

Department of Computer Science & Engineering

The Chinese University of Hong Kong


GPU accelerated pipeline

System Memory

Acc. Data Structure

GPUCPU

Dataset

CellSelection

Vertex labeling

Cell Indexing

VertexInterpolation

NormalComputation

Rendering


Cell Selection


Vertex Labeling


Interpolation

Start


Rendering Translucent Isosurfaces


Extract Triangles in Order

We extend the technique of object-aligned slices (OAS) to achieve the correct order of active cells.

Solve the order of triangles in three levels: Inter-slab order - the order of slabs within the volume to be visualized

Intra-slab order - the order of cells within each slab

Intra-cell order - the order of triangles within each active cell

We rearrange the traditional triangle table to solve the intra-cell order.


Object-Aligned Slices


Inter- and Intra-Slab Order


Implementation and Results

Rendering depend on the Viewing angle

Rendering


Rendering


Rendering

GPU-Friendly Marching Cubes-Some Examples


Time Statistics

Rendering speed , flame per second


Summary

The proposed GPU-friendly MC algorithm can extract and render isosurfaces from high-resolution 3D volume data in real time.

With this framework, we can correctly visualize multiple translucent isosurfaces, without sorting.

The framework can be trivially modified to implement a wide range of MC variants.

GPU-Based Ray Casting

REAL-TIME VOLUME GRAPHICS

Markus Hadwiger

VRVis Research Center, Vienna

Why Ray-Casting on GPUs?

Most GPU rendering is object-order (rasterization)

Image-order is more CPU-like Recent fragment shader advances

Simpler to implement

Very flexible (e.g., adaptive sampling)

Correct perspective projection

Can be implemented in single pass!

Native 32-bit compositing


Markus Hadwiger


Support for Different Grid Types

Uniform grid (voxels)

Reconstruction with

trilinear interpolation

Unstructured grid

Decomposed into

tetrahedra

Reconstruction with

linear interpolation


Daniel Weiskopf

Institute of Visualization and Interactive Systems, University of Stuttgrat, Germany

GPU-Based Ray Casting

Possible optimizations

Early ray termination

Empty space leaping

Adaptive sampling

Hardly applicable to slice-based volume rendering

Combine dynamic sampling and GPU acceleration

Support for unstructured grids

Ray casting in graphics hardware


Daniel Weiskopf



Daniel Weiskopf


GPU-Based Ray Casting-Ray Casting in Uniform Grids

Multi-pass approach

Multiple rendering passes

Constant number of steps per pass

2D textures for accumulating color and opacity

Access volume data from 3D texture map

Ray termination in separate pass

Fragment

program

Ping-pong

scheme


Daniel Weiskopf


Structure of GPU Ray Casting

Ray Traversal

Integration

Check Ray

Termination

Ray Setup

Not

term

ina

ted

Terminated

Initial passes

Precompute ray directions and lengths

Additional passes

Perform raymarching in parallel for each pixel

Split up full raymarch to check for early termination

Optimizations

Early Ray Termination

Compare accumulated opacity against threshold

Empty Space Skipping

Additional data structure encoding empty space in volume

Oct-tree

Encode measure of empty space within 3D texture read from

fragment shader

Ray-marching fragment shader can modulate sampling

distance based on empty space value

Travis Gorkin

GPU Programming and Architecture, June 2009

Travis Gorkin


Performance and Limitations More physically-based than

slice-based volume rendering

Guarantees equal sampling

distances

Does not incorporate

volumetric shadows

Reduced number of fragment

operations

Fragment programs made more

complex

Optimizations work best for

non-opaque data sets

Early ray termination and empty

space skipping can be applied

Size base volume rendering

Applications

Virtual Endoscopy


Markus Hadwiger


Applications

Virtual Colonoscopy


Markus Hadwiger


Texture-Based Volume rendering steps:

Split-volume management for texture

rendering

Extraction of texture slices

Oblique plane cutting of the volume

Stereoscopic perspective rendering

Final composition of rendering image

Interactive selection of volume of interest

How does a 2D texture work?Texture

R G B A

For each fragment:

interpolate the

texture coordinates

(barycentric)

Texture-Lookup:

interpolate the

texture color

(bilinear)


Christof Rezk Salama

Computer Graphics and Multimedia Group, University of Siegen, Germany

Color channel


Decompostition into axis-aligned slices

Draw the volume as a stack of 2D textures

Bilinear Interpolation in Hardware

3 copies of the data set in memory





Texture stack swapped based on closest to

viewpoint

Actual locations of sampling points change

abruptly when the stacks are switched

2D Textured Object-aligned slicing

2D Textures

Sampling rate is inconsistent

dd d

emission/absorption slightly incorrect

Super-sampling on-the-fly impossible




Sampling rate is fix, but can change the sampling angle to change the sampling rate

Issues with 2D-Textured Object-

Aligned Slices

Fast and simple

3 times memory consumption

Data replicated along 3 principle directions

Change in viewpoint results in stack swap

Image popping artifacts

Lag while downloading new textures

Sampling distance changes with viewpoint

Intensity variations as camera moves

How does a 3D Textures work?

R G B A

R

GB

For each fragment:

interpolate the

texture coordinates

(barycentric)

Texture-Lookup:

interpolate the

texture color

(trilinear)




3 coordination system to have 3D


Intersect slicing planes with bounding box

Sort resulting vertices in (counter)clockwise order

Construct polygon primitive from centroid as

triangle fan

Viewing angle

Resampling via 3D Textures

Sampling rate is constant

d d

Supersampling by increasing the

number of slices




Issues with 3D-Textured View-aligned

slicing

Slower, but more memory efficient

Trilinear interpolation at hardware level

Consistent sampling distance

inconsistent sampling rate for perspective

projection

Break

Visible Human Visualization

WebGL & Kinect-based Visible Human Visualzation

3D Print Interface

Some Popular Visualization Tools

Visualization Toolkit (VTK)

What is VTK?Open source, freely available

software for 3D computer graphics, image processing, and visualization

Managed by Kitware Inc.

Strictly object-oriented design (C++)

High-level of abstraction

Use C++, Tcl/Tk, Python, Java

Website: http://www.vtk.org/

Features of VTK

Parallel support (message passing,

multithreading)

Stereo support

Cross-platform and runs on Linux,

Windows, Mac and Unix platforms.

Event handling

3D widgets

What can VTK do?

Visualization techniques for visualizing

Scalar fields

Vector fields

Tensor fields

Texture

Volumetric methods

Implicit modeling and polygon reduction

Mesh smoothing, cutting, contouring, and delaunay triangulation

Image processing

Your own algorithms

Vector fields

OsiriX Imaging Software

MacOS X, Apple Computers

Image processing software dedicated to DICOM images

Also, image processing software for:

Radiology and nuclear imaging

Functional imaging

Molecular imaging.

Navigation and visualization of multimodality and multidimensional images

Website: www.osirix-viewer.com/

OsiriX Imaging Software

Ultrafast performance

Intuitive interactive user interface

Exclusive innovative technique for 3D/4D/5D navigation

Distributed under Open Source licensing - GPL

Open platform for development of processing tools

Features of OrisiX

DICOM File Support

DICOM Network Support

Non-DICOM Files Support

2D Viewer

3D Post-Processing

Optimization

Expansion & Scientific Research

Based on Open-Source components

Mac_OS Version iPhone Version

Analyze Software

Developed at Mayo Clinic's Biomedical

Imaging Resource (BIR)

Software package for 3D biomedical image

visualization and analysis

Website: www.mayo.edu/

Feature of Analyze Software

Available on 5 Unix platforms, 6 windows

platforms, and PC-based Linux operating

systemes

Imaging Modalities: CT, MRI, PET, SPECT,

ECT, Ultrasound Imaging, Biomagnetic Imaging,

Radiographic Imaging

Volume Rendering in CUDA

NVIDIA CUDA SDK Code Samples

Example: Basic Volume Rendering using 3D Textures

http://developer.download.nvidia.com/compute/cuda/sdk/websi

te/samples.html#volumeRender

Travis Gorkin



3D Slicer www.slicer.org

Open source software for visualization and image analysis

Funded by NIH, medical imaging, MRI data

Currently integrating CUDA volume rendering into Slicer 3.2

Travis Gorkin



Volume Raycasting with CUDA Jusub Kim, Ph.D. Dissertation,

Univeristy of Maryland, College Park, 2008

http://creator75.googlepages.com/cuda

Stream model for raycasting implemented in CUDA

Efficiently balance warps of threads and block sizes Single instruction execution

within warp of threads

Avoid memory conflicts with warps of threads

Travis Gorkin


References

Chapter 39. Volume Rendering Techniques, GPU Gems Volume 1, Ikits, Kniss, Lefohn, Hansen, 2003

http://http.developer.nvidia.com/GPUGems/gpugems_ch39.html

Interactive Visualization of Volumetric Data on Consumer PC

Hardware IEEE Visualization 2003 Tutorial

http://www.vis.uni-stuttgart.de/vis03_tutorial/

Acceleration Techniques for GPU-Based Volume Rendering

J. Krugger and R. Westermann, IEEE Visualization 2003

http://wwwcg.in.tum.de/Research/data/Publications/vis03-rc.pdf

3D Slicer: Volume Rendering with CUDA

http://www.slicer.org/slicerWiki/index.php/Slicer3:Volume_Rendering_Wi

th_Cuda

References

Volume Raycasting with Cuda, Jusub Kim, 2008http://creator75.googlepages.com/projects

http://creator75.googlepages.com/cudachapter.pdf

Production Volume Rendering, Jerry Tessendorf, Slides

presented at University of Pennsylvania, 2008

Real-Time Volume Graphics, SIGGRAPH 2004

http://old.vrvis.at/via/resources/course-volgraphics-2004/

Lecture 6a

Documents

Transcript of Lecture 6a