Welcome to the UPPMAX Visualization Workshop 21,22/9

2009 Day one

UPPMAX Introduction to Visualization and Python Read your own data using Python Visualize the data using VTK

Day two Advanced visualizations 3D Stereoscopic renderings (3DIS4U)

Uppsala Multidisciplinary Center for Advanced Computational Science (UPPMAX)

Is Uppsala University's resource of high-performance computers and know-how of high-performance computing (HPC).

The center was founded in 2003.

System Experts

System administration Hardware software

Application Experts

The role of the application experts is to transfer HPC competence and computational know-how within the respective fields to the users. Computational Quantum Mechanics Algorithm and Code Development Scientific Visualization Molecular Dynamics Bioinformatics Theoretical Material Physics

HPC Resources

ISIS (2007-06-01)Technical Summary

200 Compute Nodes with 800 CPU cores

1600 Gigabyte total RAM

32 Terabyte total disk

4 Tflops peak performance

switched Gigabit Ethernet for all nodes

400 CPU's in 200 dual CPU, dual core, nodes

IBM x3455 server

2 AMD Opteron 2220 Processors

140 nodes with 8 GB Memory

40 nodes with 4 GB memory

20 nodes with 16 GB memory

160 GB Hard disk

22.4 Gflops peak performance per node

HPC Resources

Grad (2008-09-15)

Technical Summary

64 Compute Nodes with 512 CPU cores

3 access Nodes with 8 CPU cores

16 Gigabyte RAM per node

18 Terabyte disk in 2 NFS servers for home directories and grid sessions

140 Terabyte disk in 8 nodes of dCache pools

switched Gigabit Ethernet for all nodes

HPC Resources

Set (2007-01-19) Os (2007)

Technical Summary

10 Compute Nodes with 10 CPUs, 20 CPU cores, and 40 threads (Power5 supports SMT)

88 Gigabyte total RAM

switched Gigabit Ethernet for all nodes

10 dualcore CPU's in 10 nodes

IBM p520 server

IBM Power5 Processors

8 GB Memory

150 GB Hard disk

XX Gflops/s peak performance per node

Hardware:

 

IBM X3455 dual Opteron 2220SE, 2.8GHz nodes. 4*10=40 cores

8GB RAM,

Gigabit interconnect

HPC Resources

OctoPS (2009) PowerCell (2009)

IBM PowerXCell 8i cluster containing of two blades with each having two PowerXCell 8i processor on them. Each Blade is equipped with 8GB of RAM, Gigabit Ethernet interconnect and runs Linux

The OctoPS cluster ("Eight Playstations") consists of eight Playstations connected with the built in Gigabit Ethernet card with a x86-based workstation as head node. The Playstation3 gaming console is equipped with the powerfull Cell/B.E. processor that can be used for High Performace Computing. Each node have access to 6 of 8 existing SPUs which gives in total 48 SPUs for the whole cluster. This generation of the Cell/B.E. processor is best suited for single precision floating point numbers and each node have around 200MB of free RAM when booted.

3DIS4U

The acronym 3DIS4U stands for "Three-dimensional image studio for Uppsala"

It is the Visualization Lab located at the Centre for Image Analysis

3D Stereoschopic display

Visualization

'The process of converting data in to a format that is visible, in order to understand the data better'

X Y Z E_x-magn -phase E_y-magn -phase E_z-magn -phase H_x-magn -phase H_y-magn -phase H_z-magn -

phase-0.3123 -0.3123 0.25 2.4407 5.90 4.0327 -154.32 2.1383 -161.19 0.0088004 27.44 0 0.00 0.010007 15.07-0.1874 -0.3123 0.25 2.3308 176.81 4.3846 26.28 3.7298 9.05 0.013289 -160.30 0 0.00 0.008179 -172.12-0.0625 -0.3123 0.25 0.7533 -112.98 4.7824 111.77 5.277 88.60 0.017554 -80.33 0 0.00 0.0024508 -104.34 0.0625 -0.3123 0.25 1.7186 56.50 5.4494 87.55 5.3578 66.87 0.01858 -102.21 0 0.00 0.0058727 68.36 0.1874 -0.3123 0.25 2.7145 -66.31 4.9781 -41.19 3.3362 -52.88 0.013082 137.33 0 0.00 0.010477 -56.00 0.3123 -0.3123 0.25 2.2417 91.01 4.0514 110.04 1.7231 104.21 0.0078549 -65.75 0 0.00 0.010079 100.69-0.3123 -0.1874 0.25 2.9017 -158.62 7.4352 36.42 2.4873 34.20 0.014298 -138.48 0 0.00 0.016719 -150.96-0.1874 -0.1874 0.25 3.7172 43.30 10.758 -114.53 5.717 -126.78 0.027764 63.79 0 0.00 0.017842 54.42-0.0625 -0.1874 0.25 1.0574 141.51 12.89 -14.42 9.0267 -31.09 0.04057 160.18 0 0.00 0.0044505 146.23 0.0625 -0.1874 0.25 2.9917 -74.37 14.398 -53.86 8.0694 -68.67 0.041081 123.01 0 0.00 0.014364 -60.93 0.1874 -0.1874 0.25 3.6098 131.50 11.682 152.36 4.388 139.34 0.026448 -26.95 0 0.00 0.021116 144.13 0.3123 -0.1874 0.25 2.5494 -99.14 7.9445 -81.63 2.0254 -89.10 0.014153 103.33 0 0.00 0.017659 -87.82-0.3123 -0.0625 0.25 1.8733 -34.53 10.567 158.36 1.5743 157.65 0.018745 -15.06 0 0.00 0.021954 -27.42-0.1874 -0.0625 0.25 3.0993 -164.09 18.427 29.92 4.4567 24.21 0.042011 -149.17 0 0.00 0.028006 -155.89-0.0625 -0.0625 0.25 1.4223 -5.70 18.087 143.49 7.0617 150.88 0.051566 -38.98 0 0.00 0.0052572 23.68 0.0625 -0.0625 0.25 1.4044 83.36 20.646 50.67 1.3372 109.55 0.051121 -129.59 0 0.00 0.020975 69.09 0.1874 -0.0625 0.25 1.1212 -128.14 20.006 -113.90 1.5299 -135.01 0.042727 67.61 0 0.00 0.031615 -117.26 0.3123 -0.0625 0.25 0.82567 -10.21 11.539 2.41 0.71926 -3.02 0.019884 -171.41 0 0.00 0.023766 -2.17-0.3123 0.0625 0.25 0.82554 169.70 11.536 -177.68 0.71907 -3.10 0.019879 8.50 0 0.00 0.023763 -2.26-0.1874 0.0625 0.25 1.1213 51.78 20.003 66.03 1.5299 -135.07 0.042719 -112.46 0 0.00 0.031615 -117.33-0.0625 0.0625 0.25 1.404 -96.68 20.649 -129.38 1.3353 109.46 0.051127 50.36 0 0.00 0.020978 69.04 0.0625 0.0625 0.25 1.4212 174.37 18.083 -36.51 7.0609 150.91 0.051556 141.03 0 0.00 0.0052518 23.84 0.1874 0.0625 0.25 3.0995 15.98 18.429 -150.01 4.4581 24.28 0.04202 30.89 0 0.00 0.028005 -155.82 0.3123 0.0625 0.25 1.8737 145.55 10.569 -21.55 1.5748 157.74 0.01875 165.02 0 0.00 0.021957 -27.34-0.3123 0.1874 0.25 2.5491 80.77 7.9437 98.29 2.025 -89.18 0.014151 -76.75 0 0.00 0.017658 -87.90-0.1874 0.1874 0.25 3.6095 -48.57 11.68 -27.72 4.387 139.27 0.026443 152.98 0 0.00 0.021116 144.06-0.0625 0.1874 0.25 2.9924 105.59 14.397 126.11 8.0678 -68.71 0.041075 -57.02 0 0.00 0.014368 -60.97 0.0625 0.1874 0.25 1.0557 -38.48 12.89 165.59 9.0265 -31.08 0.04057 -19.81 0 0.00 0.0044431 146.23 0.1874 0.1874 0.25 3.717 -136.65 10.759 65.52 5.7182 -126.73 0.027769 -116.16 0 0.00 0.017841 54.48 0.3123 0.1874 0.25 2.9022 21.46 7.4365 -143.50 2.488 34.27 0.014302 41.60 0 0.00 0.016721 -150.88-0.3123 0.3123 0.25 2.2416 -89.07 4.0513 -70.04 1.7228 104.13 0.007854 114.17 0 0.00 0.010079 100.61-0.1874 0.3123 0.25 2.7144 113.63 4.9777 138.75 3.3354 -52.94 0.013079 -42.73 0 0.00 0.010477 -56.06-0.0625 0.3123 0.25 1.7192 -123.53 5.4489 -92.48 5.3569 66.84 0.018577 77.77 0 0.00 0.0058749 68.33 0.0625 0.3123 0.25 0.75238 67.02 4.7821 -68.22 5.2768 88.61 0.017553 99.68 0 0.00 0.0024477 -104.34 0.1874 0.3123 0.25 2.3304 -3.14 4.3845 -153.67 3.7302 9.10 0.013289 19.75 0 0.00 0.0081772 -172.07 0.3123 0.3123 0.25 2.4408 -174.03 4.033 25.76 2.1388 -161.12 0.0088018 -152.49 0 0.00 0.010007 15.14-0.3123 -0.3123 0.28 2.5117 -41.76 4.5759 156.86 2.4619 149.18 0.010435 -22.46 0 0.00 0.010605 -32.93-0.1874 -0.3123 0.28 2.3243 122.36 5.0836 -31.54 4.1507 -47.28 0.015428 142.76 0 0.00 0.0085161 132.69-0.0625 -0.3123 0.28 0.70946 -169.22 5.6112 48.32 5.6974 28.54 0.01998 -141.14 0 0.00 0.0024375 -161.55 0.0625 -0.3123 0.28 1.6824 -1.79 6.2464 24.72 5.6915 6.81 0.020825 -162.98 0 0.00 0.0060572 9.44 0.1874 -0.3123 0.28 2.7172 -120.42 5.7108 -97.74 3.704 -109.44 0.015197 80.40 0 0.00 0.010971 -110.52 0.3123 -0.3123 0.28 2.3461 43.94 4.6523 61.91 2.0152 54.82 0.0094932 -115.43 0 0.00 0.010915 53.30-0.3123 -0.1874 0.28 2.8836 147.84 8.1454 -17.77 2.7671 -21.45 0.016508 165.72 0 0.00 0.017206 155.36-0.1874 -0.1874 0.28 3.3664 -19.83 11.326 179.27 5.8134 167.99 0.029727 -2.49 0 0.00 0.016949 -9.87-0.0625 -0.1874 0.28 0.839 75.55 13.125 -89.75 8.4147 -102.71 0.040578 86.41 0 0.00 0.0037022 76.11

Making the invisible visible

One of the goals of visualization is to make it easier to explore data and information

The viewer will gain new insights, sometimes profound and sometimes quite unexpected.

OBS! The same data as in the previous slide!

Visualization techniques

Still the same data...

Visualization

An exploration tool for data visualization has been developed in cooperation with UPPMAX and the Institute for Space Physics

The following 2 slides will show some visualizations from the program The previous one was also produced by this program

Scientific Visualization Scientific visualization is the process of

exploring, transforming, and viewing data as images

The dimensionality of the data is generally larger than or equal to 3

Visualization is often interactive We are not trying to create realistic images,

but to visualize the data in an informative way Dependent on the task given

VTK – The Visualization ToolKit

Open source, freely available software for 3D computer graphics image processing visualization

Managed by Kitware, Inc. Object-oriented design (C++) High-level of abstraction Use C++, Tcl/Tk, Python, Java

True visualization system

Visualization techniques for visualizing scalar fields vector fields tensor fields

Polygon reduction Mesh smoothing Image processing Your own algorithms

Additional features Parallel support

message passing multi-threading

Stereo support Integrates with Motif, Qt, Tcl/Tk,

Python/Tk, X11, Windows, ... Event handling 3D widgets

3D graphics Surface rendering Volume rendering

Ray casting Texture mapping (2D) Volume pro support

Lights and cameras Textures Save render window to .png, .jpg, ...

(useful for movie creation)

VTK Dataset Types

vtkStructuredPoints, vtkImageData vtkRectilinearGrid vtkStructuredGrid vtkPolyData vtkUnstructuredGrid Methods for reading and writing

• Organizing structure plus attributes

– Structured Points

– Rectilinear Grid

– Structured Grid

Datasets

Unstructured Grid A collection of vertices, edges, faces,

and cells whose connectivity information must be explicitly stored

Basic VTK objects to render a scene:

1. vtkRenderWindow2. vtkRenderer3. vtkLight4. vtkCamera5. vtkActor6. vtkProperty7. vtkMapper

SourceSources

Mappers

Actors

Renderers

Filters

Window

First the sources must be defined. Use data from files or predefined objects.

Interactor

Filter the data if necessary.

Add a mapper so that the data is converted to an object.

The object is converted to an actor so that the visual properties can be adjusted

Add the actors to a renderer

Connect the renderer to a window where the objects shall be rendered

Connect the window to an interactor so we can interact with it.

import vtk

object = vtk.vtkOBJReader()object.SetFileName('teapot.obj')

outlineFilter = vtk.vtkOutlineFilter()outlineFilter.SetInputConnection(object.GetOutputPort())

outlineMapper = vtk.vtkPolyDataMapper()outlineMapper.SetInputConnection(outlineFilter.GetOutputPort())

objectMapper = vtk.vtkPolyDataMapper()objectMapper.SetInputConnection(object.GetOutputPort())

outlineActor = vtk.vtkActor()outlineActor.SetMapper(outlineMapper)outlineActor.GetProperty().SetColor(0.5, 0.5, 0.5) objectActor=vtk.vtkActor()objectActor.SetMapper(objectMapper)objectActor.GetProperty().SetColor(0.2,0.6,0.6)

ren = vtk.vtkRenderer()ren.AddActor(objectActor)ren.AddActor(outlineActor)ren.SetBackground(0.6, 0.6, 0.7)

renWin = vtk.vtkRenderWindow()renWin.AddRenderer(ren)renWin.SetWindowName("Teapot")renWin.SetSize(500,500)

iren = vtk.vtkRenderWindowInteractor()iren.SetRenderWindow(renWin)iren.Initialize()iren.Start()

Result

Summary + Free and open source Create graphics/visualization applications

fairly fast Object oriented - easy to derive new classes Build applications using "interpretive" languages

Tcl, Python, and Java Many (state-of-the-art) algorithms Heavily tested in real-world applications Large user base provides decent support Commercial support and consulting available

Software

Python Read your own data files

Simple to learn An interpreted language

VTK (The Visualization Toolkit) Create customary visualizations

Powerful Takes some time to learn...

What is your research problem?

What kind of data do you produce? What do you want to see in this data? Will VTK be useful for you?

Visualization Techniques

Volume rendering Data Capture, often Medical

data: MRI, PET, CAT etc

Methods Marching cubes Contouring Isosurfaces

Volume Rendering Ray casting is commonly used A ray is cast through every pixel in the

view plane. The pixel is given a colour according to the interaction of the ray with the voxels in the volume. This is combined with the light from light sources

Multi-modal Rendering

Register two volumes to each other Rays are cast into the first volume For points on cut planes, values from

both volumes are blended to produce a colour that enhances both modalities

MRI + PET

Surface Reconstruction from Contours

Fluids

Can be rendered as particle systems

But how can we better see the speed and direction of the fluid?

And other parameters as temperature and pressure?

http://people.scs.fsu.edu/~beason/fluid/

Isosurfaces

• Wikipedia: An isosurface is a three-dimensional analog of an isocontour. It is a surface that represents points of a constant value (e.g. pressure, temperature, velocity, density) within a volume of space; in other words, it is a level set of a continuous function whose domain is 3D-space.

Examples

• http://www.vis.uni-stuttgart.de/eng/research/fields/current/spvolren/

• http://ccs.chem.ucl.ac.uk/research/porousmedia.shtml

Visualization of attributes

Vector Oriented Line

Oriented Glyph

Streamline

Streamlines

Glyphs

• What properties can you “read” from the glyphs?

Glyphs

• We can add other properties…• Can you imagine more ways to show

properties?

Glyphs

Are useful to express multidimensional data

Can have different Shapes Sizes Colors Directions Positions

And many more…

Your data might be visualized in a standard way (already in VTK) Fast implementation!

Or you might need some application specific method But probably you can use VTK

anyway!

Where can you get help with your problem?

From me and my contacts!

VTK resources

• www.vtk.org– Download (source and binaries)– Documentation– Mailing lists– Links– FAQ, Search

• www.kitware.com– VTK Textbook– VTK User’s guide– Mastering CMake