TNM093 – PRACTICAL DATA VISUALIZATION AND...
Transcript of TNM093 – PRACTICAL DATA VISUALIZATION AND...
9/12/2013
1
Scientific Visualization GroupLinköping University
TNM093 – PRACTICAL DATA VISUALIZATION
AND VIRTUAL REALITY
Timo Ropinski
CONTENT
This part of the course deals with„An interdisciplinary branch of science, primarily concerned with the visualization of three dimensional phenomena, such as architectural, meteorological, medical, biological systems. The emphasis is on [realistic] rendering of volumes, surfaces, [illumination sources], and so forth, perhaps with a dynamic (time) component.“
TNM093 - Practical Data VisualizationIntroduction
1
9/12/2013
2
SCIENTIFIC VS. INFORMATION VISUALIZATION
Direct Volume Rendering
Streamlines
Line Integral Convolution
GlyphsIsosurfaces
SciVis
Scatter Plots
Parallel Coordinates
Node-link Diagrams
InfoVis[Verma et al.,Vis 2000]
[Hauser et al.,Vis 2000]
[Cabral & Leedom,SIGGRAPH 1993]
[Fua et al., Vis 1999]
[http://www.axon.com/gn_Acuity.html]
[Lamping et al., CHI 1995]
[Tamara Munzner]
TNM093 - Practical Data VisualizationBasics
2
COURSE OVERVIEW
1. Introduction and Basics2. The Data to be Visualized3. Scalar Visualization Techniques4. Vector Visualization Techniques5. Color Mapping6. Volume Rendering Theory7. Volume Rendering Praxis8. Perception and Cognition9. Volumetric Haptics10.Medical Visualization Challenges11.(Reserved)
TNM093 - Practical Data VisualizationIntroduction
3
9/12/2013
3
LITERATURE
Schroeder et al.: The Visualization Toolkit, An Object‐Oriented Approach To 3D Graphics, Prentice Hall.
Hansen, Johnson: The Visualization Handbook, Elsevier Academic Press.
Additional books will berecommended duringthe course
TNM093 - Practical Data VisualizationIntroduction
4
THE SCIVIS GROUP AT LIUStaff
Mark Eric Dieckmann
Björn Gudmundsson
Karljohan Lundin Palmerius
Claes Lundström
Timo Ropinski
Anders Ynnerman
PhDs & Research Engineers
Alexander Bock
Daniel Jönsson
Sathish Kottravel
Stefan Lindholm
Tan Khoa Nguyen
Erik Sundén
TNM093 - Practical Data VisualizationIntroduction
5
9/12/2013
4
SCIVIS RESEARCH AREAS
Volume Rendering
Medical Visualization
Haptics
Visualization in other Domains
TNM093 - Practical Data VisualizationIntroduction
6
SCIVIS WEBSITE
TNM093 - Practical Data VisualizationIntroduction
7
http://scivis.itn.liu.se
9/12/2013
5
WORKING WITH THE GROUP
Student co‐assistant opportunities
Master Thesis projects in interesting environments
PhD positions
TNM093 - Practical Data VisualizationIntroduction
8
CURRENT RESEARCH
TNM093 - Practical Data VisualizationBasics
9
9/12/2013
6
APPLICATION AREAS
TNM093 - Practical Data VisualizationBasics
10
GOOGLE EARTH
TNM093 - Practical Data VisualizationBasics
11
9/12/2013
7
VISIBLE HUMAN PROJECT
[National Library of Medicine]
TNM093 - Practical Data VisualizationBasics
12
HUMPBACK WHALE BEHAVIOR
TNM093 - Practical Data VisualizationBasics
13
[Ware et al., IEEE CG&A 2006]
9/12/2013
8
DISCIPLINES IN SCIENTIFIC VISUALIZATION
Data Domain‐based visualization
• Medical data ⇒ Volume Visualization
• Flow data ⇒ Flow Visualization
• GIS data ⇒ Geovisualization
Other fields• Microscopic data (molecular physics),
• Macroscopic data (astronomy)
• Extremely large data sets
• etc. …
TNM093 - Practical Data VisualizationBasics
14
MODERN APPLICATION AREAS
Medical visualization
Geovisualization• 3D GIS
• Oil and gas industry
Industrial design
Biological visualization
…
[Baumann 2000]
TNM093 - Practical Data VisualizationBasics
15
9/12/2013
9
MEDICAL VISUALIZATION
Image‐based diagnosis
Pre‐operative planning
Illustrative techniques forpresentation andcommunication
[Voxelman Project] [Preim 2001]
[Amide]
TNM093 - Practical Data VisualizationBasics
16
GEOVISUALIZATION – CITY VISUALIZATION
Different application areas• Tourism
• Location‐based services
• Planning & presentation
• …
TNM093 - Practical Data VisualizationBasics
17
9/12/2013
10
GEOVISUALIZATION - SEISMIC VISUALIZATION
Problem• Find optimal location for well drilling(injection and extraction)
Approach• Leverage human intuition
• Couple simulationand visualization
TNM093 - Practical Data VisualizationBasics
18
VISUALIZATION TOOLKITS
TNM093 - Practical Data VisualizationBasics
19
9/12/2013
11
VTK – THE VISUALIZATION TOOLKIT 1/2 Open‐source, freely available software system for 3D computer graphics, image processing, and visualization
VTK consists of a C++ class library and several interpreted interface layers including Tcl/Tk, Java, and Python• scalar, vector, tensor, texture, and volumetric methods
• polygon reduction, mesh smoothing, cutting, contouring, and Delaunay triangulation
• information visualization
• 3D interaction widgets[www.vtk.org]
TNM093 - Practical Data VisualizationBasics
20
VTK – THE VISUALIZATION TOOLKIT 2/2
[www.vtk.org]
TNM093 - Practical Data VisualizationBasics
21
9/12/2013
12
AMIRA
Commercial toolkit for visualization in life sciences
[www.amiravis.com]
TNM093 - Practical Data VisualizationBasics
22
MEVIS LAB
[www.mevislab.de]
TNM093 - Practical Data VisualizationBasics
23
9/12/2013
13
VOREEN
[http://www.voreen.org]
TNM093 - Practical Data VisualizationBasics
24
DATA STRUCTURES
TNM093 - Practical Data VisualizationBasics
25
9/12/2013
14
DATA SOURCES
Two main sources for data
• Real world (Measurement)collected through measurements or observations
• Theoretical world (Simulation)computed on the basis of mathematical models
Size of data depends on• Number of samples
• Number of parameters per sample
• Number of possible values per parameter
TNM093 - Practical Data VisualizationData
26
MEASUREMENT & SIMULATION
Measurement
Simulation
TNM093 - Practical Data VisualizationData
27
9/12/2013
15
DATA SPACE
Data space is the space in which the data is measured or simulated
The data is acquired for different observation points
• e.g., sensors in a sensornetwork
TNM093 - Practical Data VisualizationData
28
DATA SPACE - DIMENSIONALITY
Dimensionality (1D, 2D, 3D,…) determines the number of independent variables• Discrete vs. continuous(4D: running clock vs. epoche‐based)
Scientific visualization: Inherent spatial domain• 2D/3D data space given• Examples: medical data, flow simulation data, GIS‐data, ...
Information visualization: No inherent spatial reference• Abstract data, spatializing through visualization• Examples: data bases, stocks, …
TNM093 - Practical Data VisualizationData
29
9/12/2013
16
DATA SPACE - INFLUENCE
Three different areas of influence exist forobservation points• Punctual influence:just at the location of the observation point
• Local influence:in its vicinity
• Global influence:for the entire data space
Punctual and local influence can be visualized spatially
Local influence requires interpolation
TNM093 - Practical Data VisualizationData
30
DATA SPACE - CONNECTIVITY
Scattered data vs. grid structures
TNM093 - Practical Data VisualizationData
31
9/12/2013
17
DATA CHARACTERISTICS
Data type• Characteristic of a data value(precision & semantic)
Data dimensionality
• Scalar, multivariate / multimodal
Data domain
• Range of possible values
TNM093 - Practical Data VisualizationData
32
CHARACTERISTICS - DATA DIMENSIONALITY
Specifies the number of components (parameters/attributes)• Scalar
• Each data element has a numeric expression
• Example: topography of terrain, expressed as 2D field containing elevations
• Vector• Each data element is considered a straight directed line with a certain length (magnitude) in nD space
• Example: direction of particle flow in a channel
• Tensor• Each data element is expressed as a (transformation) matrix
• Example: higher‐order tensor fields
TNM093 - Practical Data VisualizationData
33
9/12/2013
18
CHARACTERISTICS - DATA DIMENSIONALITY
The number of dependent variables at each observation point
Data DimensionalityData space
1D 2D 3D
1D y=f(x)
2D Flow simulation v(x)
3D Medical scans (CT, MRI, …)i(x)
Multimodal scans (PET/CT)i1(x), i2(x)
DTI scansFlow simulation v(x)
TNM093 - Practical Data VisualizationData
34
CHARACTERISTICS – DATA DIMENSIONALITYData Space /Data Dimensionality
Description Examples
value series bar chart
function graph
function over 2D height map, contour lines, false color map
2D vector field hedgehog plot, LIC
3D intensities volume rendering, iso‐surface rendering
tuple set parallel coordinates, glyphs
TNM093 - Practical Data VisualizationData
35
9/12/2013
19
WHY GRIDS?
Natural phenomena are often continuous (infinite resolution)
Computers are obviously restricted to work with a finite amount of data
Sampling is necessary
TNM093 - Practical Data VisualizationData
36
CHOOSING THE RIGHT DATA STRUCTURE
Which data organization is optimal?
• Is there a neighborhood relationship?
• How is the neighborhood information stored?
• How is navigation within the data possible?
• What calculations are possible with the data?
• Are the locations of the observation points structured?
TNM093 - Practical Data VisualizationData
37
9/12/2013
20
2D GRIDS
Uniform rectilinear
• Cells: rectangles
• Layout: uniform
Curvilinear• Cells: different quads
• Layout: non‐uniform, structured
Unstructured• Cells: polygons (mostly triangles)
• Layout: unstructured
TNM093 - Practical Data VisualizationData
38
3D GRIDS
Uniform rectilinear
• Cells: regular hexahedra
• Layout: uniform
Curvilinear• Cells: different hexahedra
• Layout: non‐uniform, structured
Unstructured• Cells: different convex polyhedra
• Layout: unstructured
TNM093 - Practical Data VisualizationData
39
9/12/2013
21
LOCAL INTERPOLATION
Interpolation is an approximate reconstruction of the original signal
• Input: data values given for cell corners
• Output: data value for an arbitrary point inside a cell
?
TNM093 - Practical Data VisualizationData
40
1D - LINEAR INTERPOLATION
Compute a new data value based on two known values
TNM093 - Practical Data VisualizationData
41
9/12/2013
22
2D – BILINEAR INTERPOLATION
TNM093 - Practical Data VisualizationData
42
3D –TRILINEAR INTERPOLATION4 linear interpolations
2 bilinear interpolations
1 trilinear interpolation
TNM093 - Practical Data VisualizationData
43
9/12/2013
23
ALGORITHM TYPES
Algorithm type based on underlying data
• Scalar algorithms
• Vector algorithms
• Tensor algorithms
TNM093 - Practical Data VisualizationChapter 3: Scalar Visualization
44
VOLUME RENDERING
TNM093 - Practical Data VisualizationVolume Rendering
45
9/12/2013
24
OUTLINE
Data Set 3D Rendering
in real‐time on commodity graphics hardware
Classification
TNM093 - Practical Data VisualizationVolume Rendering
46
VOLUME RENDERING
Image Plane
Image order approach:
For each pixel {calculate color of the pixel
}
Data Set
Eye
TNM093 - Practical Data VisualizationVolume Rendering
47
9/12/2013
25
GPU-BASED RAYCASTING
Has several benefits over slicing• Easy to implement
• Flexible (e.g., adaptive sampling, ERT)
• Correct perspective projection
• Native 32‐bit compositing
TNM093 - Practical Data VisualizationVolume Rendering
48
BASIC RAY SETUP / TERMINATION
TNM093 - Practical Data VisualizationVolume Rendering
49
Two main approaches:• Procedural ray/box intersection[Röttger et al., 2003], [Green, 2004]
• Rasterize bounding box[Krüger and Westermann, 2003]
Some possibilities• Ray start position and exit check
• Ray start position and exit position
• Ray start position and direction vector
9/12/2013
26
PROCEDURAL RAY SETUP/TERMINATION
Everything handled in the fragment shader
Procedural ray / bounding box intersection
Ray is given by camera positionand volume entry position
Exit criterion needed
Pro: simple and self‐contained
Con: full load on the fragment shader
TNM093 - Practical Data VisualizationVolume Rendering
50
- =
"IMAGE-BASED" RAY SETUP/TERMINATION
Rasterize bounding box front faces and back faces[Krüger and Westermann, 2003]
Ray start position: front faces
Direc on vector= (back faces−front faces)
Independent of projection(orthogonal/perspective)
TNM093 - Practical Data VisualizationVolume Rendering
51
9/12/2013
27
FRAGMENT SHADER
Rasterize front facesof volume bounding box
Texcoords are volumeposition in [0,1]
Subtract camera position
Repeatedly check forexit of bounding box
TNM093 - Practical Data VisualizationVolume Rendering
52
COMPOSITING
Emission/Absorption Maximum Intensity Projection
TNM093 - Practical Data VisualizationVolume Rendering
53
9/12/2013
28
CLASSIFICATIONMapping intensities to optical properties through transfer functions
TNM093 - Practical Data VisualizationVolume Rendering
54
COLOR MAPPING
Maps a scalar to a color
Implemented using lookup table indexed through the scalar value
)
Values stored in lookup table• RGB or HSV color components
• Alpha/opacity value
TNM093 - Practical Data VisualizationScalar Visualization
55
9/12/2013
29
MAPPING TO COLOR
To label (indicate set membership)
To depict a measurement
To represent reality
To decorate (consider data/ink ratio)
representinformation
TNM093 - Practical Data VisualizationScalar Visualization
56
MAPPING TO COLOR
Color perception influenced by fruit picking
Bad choice: color ramps (rainbow pitfall)
TNM093 - Practical Data VisualizationScalar Visualization
57
9/12/2013
30
LOOKUP TABLES IN VTK
Example
TNM093 - Practical Data VisualizationScalar Visualization
58
VtkLookupTable* lut = vtkLookupTable::New();lut->SetHueRange(0.6667, 0.0);lut->SetSaturationRange(1.0, 1.0);lut->SetValueRange(1.0, 1.0)lut->SetNumberOfColors(256)lut->Build();
vtkPolyDataMapper* mapper=vtkPolyDataMapper::New();mapper->SetLookupTable(lut);mapper->SetScalarRange(min,max)
CLASSIFICATION
Classification is performed through the transfer function
Intensities are mapped to optical properties
• Which color to be assigned to which intensities?
• Which intensities are transparent?
TNM093 - Practical Data VisualizationVolume Rendering
59
9/12/2013
31
TRANSFER FUNCTIONS IN 2D
TNM093 - Practical Data VisualizationVolume Rendering
60
TRANSFER FUNCTIONS 3D
TNM093 - Practical Data VisualizationVolume Rendering
61
9/12/2013
32
VOLUME ILLUMINATION
TNM093 - Practical Data VisualizationVolume Rendering
62
WHAT IS VOLUMETRIC SHADING? Differences to surface shading
• Light/volume interaction insteadof light/surface interaction
• Translucent materials are frequent
• Volumetric shadows occur
Key problems• How does light interact with thevolume?
• How can materials be represented?
• How can scattering and shadowingbe supported?
TNM093 - Practical Data VisualizationVolume Rendering
63
9/12/2013
33
WHY VOLUMETRIC SHADING?
Langer, Bülthoff: Depth Discrimination from Shading under Diffuse Lighting [Perception 00]:
Gribble, Parker: Enhancing Interactive Particle Visualization with Advanced Shading Models[APGV06]
„… depth discrimination under diffuse lighting is superiorto that predicted by a classical sunny day model, …“
TNM093 - Practical Data VisualizationVolume Rendering
64
EMISSION ABSORPTION MODEL
Assumption:Volume consists of small particles which are
• opaque
• non‐reflecting
• light emitting
• the only light sources in the scene
TNM093 - Practical Data VisualizationVolume Rendering
65
9/12/2013
34
LOCAL VOLUMETRIC SHADING
Illumination limitations so far
• Sample points are only light sources
• No interaction with external light
Practical solution
cr(s') cri(s',Li)i1
k
')'()'()'(),'(),()( dsscscsssTIssTsI re
s
s eBebe
e
b
s'
0L
1L
2L
TNM093 - Practical Data VisualizationVolume Rendering
66
DIFFUSE LIGHTING
Diffuse lighting effects are mostprominent in volume data
Light calculation is based onLambert‘s law
Ratio can be expressed by dot product
LL N N
TNM093 - Practical Data VisualizationVolume Rendering
67
9/12/2013
35
DIFFUSE LIGHTING
no shadingdiffuse lighting
darkdiffuse lighting
bright
TNM093 - Practical Data VisualizationVolume Rendering
68
DIFFUSE LIGHTING
TNM093 - Practical Data VisualizationVolume Rendering
69
9/12/2013
36
SPECULAR LIGHTING
Specular highlights can addrealism to certain tissues
Lighting calculation is view dependent
Can be expressed as follows
LN
RV
H
diffuse
diffuse + specular
TNM093 - Practical Data VisualizationVolume Rendering
70
SPECULAR REFLECTIONS
diffuseonly
=10 =30
=120=60 =90
TNM093 - Practical Data VisualizationVolume Rendering
71
9/12/2013
37
SPECULAR LIGHTING
TNM093 - Practical Data VisualizationVolume Rendering
72
AMBIENT LIGHTING
Add constant light in shadowed regions
Drawback: contrast reduction
TNM093 - Practical Data VisualizationVolume Rendering
73
9/12/2013
38
AMBIENT LIGHTING
ambient bright+diffuse + specular
ambient medium+diffuse + specular
ambient dark +diffuse + specular
TNM093 - Practical Data VisualizationVolume Rendering
74
TNM093 - Practical Data VisualizationVolume Rendering
75
PHONG LIGHTING
Volume Rendering
9/12/2013
39
ADDING ATTENUATION
TNM093 - Practical Data VisualizationVolume Rendering
76
PHONG SHADING + ATTENUATION
no shadingPhong shadingand attenuation
Phong shading
TNM093 - Practical Data VisualizationVolume Rendering
77
9/12/2013
40
LN
RH
GRADIENT CALCULATION
Surface normal is required for diffuse and specular illumination
The gradient is a good approxi‐mation for a surface normal
[Levoy, CG&A 1988]
L N
TNM093 - Practical Data VisualizationVolume Rendering
78
GRADIENT ESTIMATION
The gradient vector is the first‐order derivative of the scalar field
We can estimate the gradient vector using finite differencing schemes• Forward/backward differences• Central differences
partial derivativein x‐direction
partial derivativein y‐direction
partial derivativein z‐direction
TNM093 - Practical Data VisualizationVolume Rendering
79
9/12/2013
41
BACK-/FORWARD DIFFERENCES
Forward differences
Backward differences
TNM093 - Practical Data VisualizationVolume Rendering
80
FORWARD DIFFERENCES
TNM093 - Practical Data VisualizationVolume Rendering
81
9/12/2013
42
CENTRAL DIFFERENCES
TNM093 - Practical Data VisualizationVolume Rendering
82
WRAP UP
TNM093 - Practical Data Visualization83
9/12/2013
43
WRAP UP
Visualization of spatial data
Scalar visualization algorithms
GPU‐based volume rendering
Classification
Volume illumination
TNM093 - Practical Data Visualization84
LITERATURE
Hansen and Johnson: The Visualization Handbook, Elsevier.
Engel, Hadwiger, Kniss, Rezk‐Salama, Weiskopf:Real‐Time Volume Graphics, AK Peters.
TNM093 - Practical Data Visualization85
9/12/2013
44
ACKNOWLEDGEMENTS
These slides contain figures credited to• Christof Rezk‐Salama
• Markus Hadwiger
• Klaus Engel
TNM093 - Practical Data Visualization86