Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student...
-
date post
20-Jan-2016 -
Category
Documents
-
view
214 -
download
1
Transcript of Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student...
![Page 1: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/1.jpg)
Manifold Learning– from Brain Visualization to Advanced Image Processing
Anders Brun, PhD studentMedical Informatics, Dept. of Biomed. Eng. (IMT)Center for Medical Imaging and Visualization (CMIV)Linköpings universitet, Sweden
IMTIMT
GeodesicsGeodesics
TensorsTensors
ManifoldsManifolds
Diffusion Weighted MRIDiffusion Weighted MRI
VisualizationVisualization
Intrinsic and Extrinsic meansIntrinsic and Extrinsic means
Manifold LearningManifold Learning
Machine LearningMachine Learning
Image ProcessingImage Processing
![Page 2: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/2.jpg)
2
Thesis Advisors Hans Knutsson
Prof.Medical InformaticsDepartment of Biomedical EngineeringLinköpings universitet
Carl-Fredrik WestinAssociate Prof. of Radiology Laboratory of Mathematics in ImagingHarvard Medical School / Brigham and Women´s Hospital
Magnus HerberthsonAssistant Prof.Department of MathematicsLinköpings universitet
![Page 3: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/3.jpg)
3
Abstract
In this talk I discuss the need to standardize image processing on manifolds. Two tools are presented, a framework for storage of geometric tensor arrays and a method for manifold learning, related to the Riemannian exponential and logaritmic map. Finally some future perspectives are presented, in particular the fusion of manifold learning and image processing.
![Page 4: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/4.jpg)
4
Processing Tensor fields on Manifolds
Storage, processing and visualization of functions M ! V V ... V V* V* ... V*
Intrinsic formulations, Charts, Atlases, Coordinate Free Methods ...
Extrinsic methods can be problematic and there are more complicated cases than 2D surfaces embedded in 3D!
RP2
also SO(3), ...
![Page 5: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/5.jpg)
5
A collaboration by:
Anders Brun, Burak Acar, Emma Munos, Leila Cammoun, Andreas Sigfridsson,Dario Sosa, Marcos Martin Fernandez, Björn Svensson, Magnus HerberthsonHans Knutsson
The Similar Tensor Array Core
The Similar Tensor Array Conventions
”Simple conventions for storing tensor array data ...”
![Page 6: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/6.jpg)
6
What problem are we solving?
”The Ideal Customer”A fresh MSc or PhD student starting to work
with tensor data
Wanting to:Apply new algorithms to old (known) test dataApply old algorithms to new datsetsExchange tensor data between algorithms
and/or people
![Page 7: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/7.jpg)
7
Core and Extensions
Mathematics
Processing paradigms
Data storage
Physics
Worlds
STAC
STAE A lot of tensor
processing can be done in the core
Extensions should include conventions for: DTI Elastography Image Processing Geoscience ...
Existing standards NRRD, VTK, Analyze, ...
Applications
![Page 8: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/8.jpg)
8
Scalar data
array_dimensionality = 2 array_size = [4, 3] tensor_order = 0 tensor_index_types = {} data = [0.1 0.3 0.7 0.5 0.7 0.5 ...
0.5 0.6 0.8 0.1 0.5 0.3]
(1,1) (1,2) (1,3)
(2,1) (2,2) (2,3)
(3,1) (3,2) (3,3)
(4,1) (4,2) (4,3)
(4 * 3)
![Page 9: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/9.jpg)
9
Vector data
array_dimensionality = 2 array_size = [4, 3] tensor_order = 1 tensor_index_types = {contravariant} data = [1.2 1.4 1.2 0.4 0.5 0.4]
... but in what basis???
(1,1) (1,2) (1,3)
(2,1) (2,2) (2,3)
(3,1) (3,2) (3,3)
(4,1) (4,2) (4,3)
(4 * 3 * 2)
![Page 10: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/10.jpg)
10
The basis describing tensors
Indices are coordinates:(c1, c2) or generally ci
Basis vectors given by:
or generally
(1,1) (1,2) (1,3)
(2,1) (2,2) (2,3)
(3,1) (3,2) (3,3)
(4,1) (4,2) (4,3)
![Page 11: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/11.jpg)
11
Dual vectors
array_dimensionality = 2 array_size = [4, 3] tensor_order = 1 tensor_index_types = {covariant} data = [0.2 2.4 3.1 0.3 1.2 0.7]
(1,1) (1,2) (1,3)
(2,1) (2,2) (2,3)
(3,1) (3,2) (3,3)
(4,1) (4,2) (4,3)
(4 * 3 * 2)
![Page 12: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/12.jpg)
12
A metric instead of voxelsize
array_metric_tensor = gij
(1,1) (1,2) (1,3)
(2,1) (2,2) (2,3)
(3,1) (3,2) (3,3)
(4,1) (4,2) (4,3)
(1,1) (1,2) (1,3)
(2,1) (2,2) (2,3)
(3,1) (3,2) (3,3)
(4,1) (4,2) (4,3)
![Page 13: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/13.jpg)
13
A metric instead of voxelsize
array_metric_tensor = gij
(1,1) (1,2) (1,3)
(2,1) (2,2) (2,3)
(3,1) (3,2) (3,3)
(4,1) (4,2) (4,3)
(1,1) (1,2) (1,3)
(2,1) (2,2) (2,3)
(3,1) (3,2) (3,3)
(4,1) (4,2) (4,3)
![Page 14: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/14.jpg)
14
Additional core fields
[tensor_name] = D[tensor_description] = ”Diffusion”[tensor_index_names], e.g. {a, b}[array_index_names], e.g. {i, j, k}
![Page 15: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/15.jpg)
15
Image Processing is often invariant ...
... to how the image is displayed!
![Page 16: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/16.jpg)
16
Tensor Array Processing should be invariant too.
Remember, in some applications there is no extrinsic space!
![Page 17: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/17.jpg)
17
Even without an extrinsic space you can do ...
Geodesic tracking (given a metric field gij)) Distances (given a metric field gij) Balls (given a metric field gij) Fiber tracking (given a metric tensor) Image processing on curved manifolds (given a metric field gij) Tilted volumes with non-isotropic voxelsizes (given a metric tensor) Correct visualizations, up to actions of the orthogonal group O(n)
(with a metric) or general linear group GL(n) (without a metric) Solve PDEs Interpolate the field
![Page 18: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/18.jpg)
18
Manifolds in Image Processing
– It´s a Klein bottle!
![Page 19: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/19.jpg)
19
Manifolds in Biology
Adapted from Visual Cortex: Looking into a Klein Bottle by Nicholas Swindale,Current Biology, Vol. 6 No. 7:776–779, 1996
Optical recording from the visual cortex of a macaque monkey.
“[...] reasoning based on topological arguments may be increasingly helpful in attempting to understand how the world, Klein bottles and all, is unfolded onto the surface of the cortex.” – N. Swindale
![Page 20: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/20.jpg)
20
Dimension reduction / Manifold Learning
Dimension Reduction Principal Components Analysis, PCA (Pearson, 1901; Hotelling, 1933) Multidimensional Scaling, MDS (Torgerson, 1952; cf. Young and Householder, 1938) Projection Pursuit (Friedman and Tukey, 1974) Independent Components Analysis, ICA (Herault and Jutten, 1986)
Non-linear Dimension Reduction Self Organizing Maps, SOM (Kohonen, 1982) Principal Curves ([Hastie and Stuetzle, 1989) Kernel PCA (Schölkopf et al, 1998) Isomap (Tenenbaum et al, 2000) Locally Linear Embedding, LLE (Roweis and Saul, 2000) Laplacian Eigenmaps, LE (Belkin & Niyogi, 2002) Hessian Eigenmaps, H-LLE (Donoho & Grimes, 2003) Diffusion Maps (Coifman, S. Lafon, 2004) Local Tangent Space Alignment, LTSA (Zhang and Zha, 2004) Sample Logmaps (Brun et al 2005)
![Page 21: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/21.jpg)
21
Isomap
Isomap (Tenenbaum et al, Science, 2000)
![Page 22: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/22.jpg)
22
Geodesics, Exponential maps and Logarithmic maps
Lagrangian style (Exponential maps)
Eulerian ...
![Page 23: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/23.jpg)
23
Why are the Exp and Log functions Important in Signal Processing?
Analogies of operations in vector spaces and in manifolds, reproduced from (Pennec et al.,2005).
![Page 24: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/24.jpg)
24
An Eulerian approach to Logp(x) ...
... easy if we can calculate distances on M!
![Page 25: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/25.jpg)
25
Example: The Swiss Roll
No distortion for a “flat” metric!
LOGMAP
![Page 26: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/26.jpg)
26
LOGMAP – manifold hunting
From LOGMAP: A Method for estimating Riemannian normal coordinates for a set of datapoints sampled from a manifold byA. Brun, C.-F. Westin, M. Herberthson and H. Knutsson, submitted to ICIP 2005.
Logmap
Data(high.dim.)
– It´s a Klein bottle!
![Page 27: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/27.jpg)
27
![Page 28: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/28.jpg)
28
Example: 2D shape (S-Logmap)
![Page 29: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/29.jpg)
29
Example: 2D shape (PCA)
![Page 30: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/30.jpg)
30
Manifolds of Images and Image Processing
x y
time
M
L2(Rn)
– What manifold is this?Find your favourite application for R3 £ P2 ...
![Page 31: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/31.jpg)
31
Problems in Manifold Learning
Calculation of distances!What is the topologyWhat is a suitable metric?Large samples sizesSmall sample sizes
![Page 32: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/32.jpg)
32
Problems in Tensor Processing
How to move tensors on a manifold to perform filtering in a point?
The structure tensors in Hyper Image Manifolds?
Choosing and managing charts in an Atlas
![Page 33: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/33.jpg)
33
![Page 34: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/34.jpg)
34
![Page 35: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/35.jpg)
35
![Page 36: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/36.jpg)
36
![Page 37: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/37.jpg)
37
![Page 38: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/38.jpg)
38
![Page 39: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/39.jpg)
39
![Page 40: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/40.jpg)
40
![Page 41: Manifold Learning – from Brain Visualization to Advanced Image Processing Anders Brun, PhD student Medical Informatics, Dept. of Biomed. Eng. (IMT) Center.](https://reader035.fdocuments.in/reader035/viewer/2022062806/56649d385503460f94a120bf/html5/thumbnails/41.jpg)
41
Learning
Encyclopedia Britannica:“Process of acquiring modifications in existing knowledge, skills, habits, or tendencies through experience, practice, or exercise.”
Oxford Advanced Learner’s Dictionary:“gain knowledge or skill by study, experience or being taught.”