Structured Multidimensional Information · N×M Array of values (attribute), (typically two spatial...
Transcript of Structured Multidimensional Information · N×M Array of values (attribute), (typically two spatial...
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
Representation Primitives, Location and Attribute(s)
Structured Multidimensional Information
1
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
(Medical) Imaging
Energy,Informationor Stimuli
Interaction
Image Formation (Physics)
Sensors Acquisition
Image Production (Technology)
Multidimensional InformationProcessing, Analysis
2
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
Medical ImagingInput stimuli
Light (coherent)EM-field
Radiation:
νx-rayα,β,γe− e+
UltrasoundLabelled probesContrast agentsBiochemical signalsElectric signalsMechanical signalsCognitive activity
Endoscopy, keratopography, thermographyMoiré-pattern imaging, polarographyMicroscopies (confocal, multi-photon, ...Radiography, Fluoroscopy, CT, axial, spiralContrast angiography, dynam. MR-angio.NMRI, nuclear medicine, functional NMRScintigraphy, gamma-camera,...TEM, SEM, synchrotron EM, PET, SPECTFlow, tensor, parametric imagingUS mode A,b, Doppler, Time-of-flightDiffusion / perfusion imagingImpedance tomography, cardiographyMagnetoelectroencephalography map ... .... . .
Multidimensional InformationN-dimensional reconstruction
Processing, AnalysisQuantification, Visualization
Interpretation Diagnosis
DataBase of Atlases and PhantomsMapping & registrationInter-comparisonGround truth & referenceDBs: anatomical, functional, Simulation 3
Unstructured and Structured Information
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
Samples
Signals
Images
Volumes & Videos
Hyper-vols
Set of attribute values (no order, no spatial or temporalstructure; may be point samples in N dimensions)
Ordered set of values (attribute), in one dimension(typically time or one spatial dimension): f (t)
N×M Array of values (attribute), (typically two spatialdimensions): f (x,y) ~ “bidimensional signals ”.
N×M×L Array of values, (typically three spatialdimensions): f(x,y,z) and sequences of frames f(x,y,t)
N×M×L×K array of values; e,g, volume sequence f (x,y,z,t)
4
Structured Information II
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
Boundaries&
Manifolds
N-dimensional local-structured sets in M dimensions andboundaries of other manifolds (topology differences):
• Lifting of 2D images to 3D volumes (or ND to (N+1)D)• Boundaries of objects in M dimensions:
o Endpoints, sets of endpoints, hot points.o Contours (planar and embedded in M ).o Surface boundaries
• Parametric curves: (x(s), y(s)), (x(s), y(s), z(s))• Time evolving sets (shape and topology changes):
o Snakes: (x(s), y(s), z(s))(t)o Lightings (branching snakes)o Balloons and branching balloons
• Networks, branching curves, attributed graphs in M
• Parametric surfaces, interfaces, waves, patches andmembranes: (x(r, s), y(r, s), z(r, s))
• Hypersurfaces, branching manifolds and branes• Mixed manifolds 5
⎧⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎩
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
Location: x, y, z, t,σ,...(spatial) configuration
Associated Attribute(s) & Features :
Representation primitive of discrete samples
Scalar value (intensity, depth, density)Vector value: chanels (color, optical flux)…
Structured Multidimensional Information
Orthogonal discrete grid
6
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
Where is the “Pixel”?
7
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
*Euclidean Representation Primitives
Facets (triangles, voxel-faces, polygons)
Points and nodes, pixels, voxels, hyper-voxels
Edges, lines, arcs, arrows, flux-lines, planes
Tetrahedra, spheres, polytopes, ellipsoids
Polygons, disks
Non Euclidean:Geodesic pixels, spels, surfels and “patches”
8
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
*Other Representation Elements(may have location and attributes)
Contours (iso-…), discrete boundaries, meshes, finite/volume element, solid boundaries
Medial Axis (regional-reps) = skeletons Graphs
∂iV , iV
Skel i V , Γi V
Smoothed ParticlesPDFs (Point Spread Functions, LDFs, EDFs)
Texels (textural primitives), symbol sets, icons, images
Cellular complexes (vertices+edges +facets+voxels)
Polynomials, Splines, Basis Functions, Wavelets
Neighborhoods, cliques, structuring elements, kernels
A 変
9
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
Location: x, y, z, t,σ,... Geometry, (spatial) configuration, domain
Orthogonal discrete gridHoneycomb or hexagonal grid
Set of points, (cartesian) array, grid, lattice or tiling, irregular mesh/graph
Offset-square gridInterlaced tiling
Spheroidal polar coordinatesStatistically regular tri-mesh
10
vertex-and-edge grid(= graph)
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
Scalar image (e.g. intensity, density, opacity, transparency)Binary attribute (boolean value, orientation, “up/down”)Parametric image (e.g. range images = depth) Vector image (n-channels), feature vector (color, optical flux, α-blend,…)Transform domain (Fourier, Cosine, Hermite, EDT, MAT,…)Output result of detectors or operatorsLabels (classification, contours,…)Local Information (fractal dimens, texture, coherence, directionality,…)Field image (distance, tensor = transformation, displacement, warp)An image itself (per point or pixel)Neighborhood relations, regional, hierarchical, connectivity, ...Meta-information (data structures) & pointers to information
Associated Attributes & Features:
11
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
Scalar and Vector attributes can be visualized through:• Gray levels, colors and Look-Up-Tables (indexed color)• Patterns, textures, symbols, icons and widgets• Material properties: texture, transparence, shinniness, etc.• Animation, blinking, stretching, morphing• Lifting from 2D to 3D, projections, shades, depth cues• Image fusion techniques, color-channel (de)composition• Optical techniques: Moïré, polarization, filters, phase-
interference, holography
Attribute Visualization:
(Visualization and Computer Graphics domain)
12
OpenGL Primitives
GL_QUAD_STRIP
GL_POLYGON
GL_TRIANGLE_STRIPGL_TRIANGLE_FAN
GL_POINTS
GL_LINES
GL_LINE_LOOP
GL_LINE_STRIP
GL_TRIANGLES
13(C) Copyright - Jorge Marquez 2010 CCADET UNAM
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
A 2-bit indexed color image. The color of eachpixel is represented by a number; each number(the index) corresponds to a color in the colortable (the palette).
Visualizing 2D information
Scalar attribute grayscale attribute
14
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
In classical halftone images, the primitives are dots or squares of varying size and fixed intensity (Black or Color). An intensity is visualized by using a size proportional to gray-level (or color) intensities.
15
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
In modern halftones, the primitive is a B/W dot, with a
quasi-random position; the attribute is gray-level and
visualization is doneby varying spatial-location
statistics: dots agglomerate in function of intensity
16
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
In “ASCII” art, the primitives are a set of B/W characters,
with fixed location. The gray-level attribute is visualized by
using sorted characters in function of their average gray-level by area unity.
17
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
18
Floyd-Steinberg dithering algorithm, based on error diffusion
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
A vector field may be a displacement field to model an image deformation: each pixel has a displacement
vector associated (Δx(x,y), Δy(x,y))
Δx(x,y)
Δy(x,y)
I(x,y) I(x+Δx,y+Δy)
The displacement Δx, Δy changes at each point
19
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
20
T(I(x,y)) + ΔI(x,y)
T(x,y) (I(x,y)) ⋅M(x,y) + ΔI(x, y)
Transfer function uout = T(uin )
Position-dependent transfer function
Scalar-field “deformations” (intensity-only)
Offset field(background)
Modulation field
Pixel-by-pixel addition (+) and product (⋅)
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
21
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
22
Vector Images (Format .svg)
Filled vectorized contours(polygons)
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
23
L: → 2 Discrete Polyline of N vertices:L = {(xs,ys)}s=0,…, N−1
Edge vectors: vs = (xs,ys)−(xs−1 ,ys−1)
L is a (closed) contour of N−1 verticesif (x0,y0) = (xN−1, yN−1)
Region A (polygon) = solid /geometric interior of ∂A. We call ∂A the boundary of A.
L may be a vectorized curve C. Set of pixels {(x, y)} fitting C = rasterized curve:
C: → 2 parametrical curve, C=(x(s), y(s)), s may be the arc of lenght
A (very) simple manifold
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
24
{ },
3
, 1
2
0, [0,1]s s s s s N
t t tt= … −
∈+ + +a b c d
cubic-spline interpolant
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
Spatial Discretization Spacing / sample geometry
Atributte
x, y, t, etc.
25
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
Atributte(s) Quantization Scale of L values (integers)
Atributte
x, y, t, etc.
26
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
Digitization = Discretization AND Quantization
Atributte
x, y, t, etc.
27
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
Continuous Appearance – Discrete Model
Atributte
(x, y)
28
“2.5D” Intensity Surface= (binary 3D object) 2D Image (gray levels)
29(C) Copyright - Jorge Marquez 2010 CCADET UNAM
3D Surface = (binary 3D object)
2D Image (gray levels)Iso-intensity sets representation
30(C) Copyright - Jorge Marquez2010 CCADET UNAM
Processing, Analisis & Visualization:• On representation primitives
o Format conversions, coding, compression, cryptingo Re-sampling, texturing, meshing, finite element
• On attributes and parameterso Transfer functions, segmentation & labelingo Convolutions (average = special case)o Integer transforms (e.g., FFT)o Functions and arithmetic operationso Logic operations, predicate segmentationo Fusiono Re-sampling, Interpolation
• On localization/coordinateso Geometric transformations – Distortions/warpso Registration, image fusiono Re-sampling, zoom
• On visualization attributes• On attributed manifolds
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
31
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
1. Scalar image (e.g. Intensity)2. Parametric image (e.g. range images) 3. Features vector (color, optical flux,…)4. Transform domain (Fourier, EDT, MAT,…)5. Output of detectors or operators6. Labels (classification, contours,…)7. Local Information (fractal dimension,
texture, coherence, directionality,…)8. Field image (distance, tensor)9. Neighborhood relations, regional,
hierarchical, connectivity, ...10. Meta-information (data structures)11. Pointers to local information12. Etcétera…
Location & Geometry: x,y,z,t,σ,...
Associated Attributes & Features:
Representation:
Facets, meshes
DiscreteBoundaries
Medial Axis& Graphs
∂iV , iV
Skel i V , Γi V
Smooth Particles32
(C) Copyright - Jorge Marquez 2010 CCADET UNAM
33
HERE: manifold examples• Parametric curves, polylines, polygons (vectorized contours)• Discrete oriented boundaries: Facet, voxel, triangle mesh,
hamiltonian paths = (directed closed) graphs• Interfaces, thin-plate splines, bilinear Coons patches• Möebius strips, Klein bottle, Penrose triangle• Branching structures: Medial Axis Transform and SKIZ,
coraline manifold, mixed sets• Topology properties