Post on 26-Dec-2015
ter Haar Romeny, ISS 2014
Prof. Bart M. ter Haar Romeny, PhD
Biomedical Image Analysis & Interpretation
TU/e, Department of Biomedical Engineering
B.M.terHaarRomeny@tue.nl
Multi-scale segmentation
Edge focusing, watershed, winding numbers
ter Haar Romeny, ISS 2014
MR slice hartcoronair
scale
• toppoints
• graphtheory
Edge focusing
ter Haar Romeny, ISS 2014
Structures exist at their own scale:
Original = e0 px = e1 px = e2 px = e3 px
Noise edges
ter Haar Romeny, ISS 2014
100 200 300 400 500
4
2
2
4
6
8
10
The graph of the sign-change of thefirst derivative of a signal as a function of scale is denoted the scale-space signature of the signal.
Zero-crossingsof the secondorder derivative= max of firstorder derivative,
as a functionof scale
ter Haar Romeny, ISS 2014
The notion of longevity can be viewed of a measure of importance for singularities [Witkin83]. The semantical notions of prominence and conspicuity now get a clear meaning in scale-space theory.
In a scale-space we see the emergence of the hierarchy of structures. Positive and negative edges come together and annihilate in singularity points.
ter Haar Romeny, ISS 2014
Example:
Lysosomesegmentationin noisy 2-photonmicroscopy3D images ofmacrophages.
ter Haar Romeny, ISS 2014
Marching-cubes isophote surface ofthe macrophage.
slice 24 slice 21 slice 25 slice 18 slice 22 slice 21
slice 24 slice 23 slice 24 slice 20 slice 18 slice 24
Preprocessing:- Blur with = 3 px- Detect N strongest maxima
ter Haar Romeny, ISS 2014
We interpolatewith cubic splinesinterpolation35 radial tracksin 35 3Dorientations
ter Haar Romeny, ISS 2014
The profiles are extremely noisy:
Observation: visually we can reasonably point the steepest edgepoints.
ter Haar Romeny, ISS 2014
Edge focusingover all profiles.
Choose a startlevel based onthe task, i.e. finda single edge.
ter Haar Romeny, ISS 2014
Detected 3D points per maximum.
We need a 3D shape fit function.
ter Haar Romeny, ISS 2014
1
2
,1
23
2 sin, 1
23
cos,
1
23
2 sin, 1
42 15
2 sin2,
1
215
2 cossin, 1
45
3 cos2 1,
1
215
2 cossin, 1
42 15
2 sin2
The 3D points are least square fit with 3D spherical harmonics:
ter Haar Romeny, ISS 2014
Resulting detection:
ter Haar Romeny, ISS 2014
An efficient way to detect maxima and saddlepoints is found inthe theory of vector field analysis (Stoke’s theorem)
ter Haar Romeny, ISS 2014
Topological winding numbers
p Li1dLi2. . .dLin i1i2...inL jpL jpn2p L1dL2 L2dL1
L12 L22
N-D
2-D
is the wedge product (outer product for functionals)
ter Haar Romeny, ISS 2014
In 2D: the surrounding of the point P is a closed path around P.
The winding number of a point P is defined as the number of times the image gradient vector rotates over 2 when we walk over a closed path around P.
maximum: = 1minumum: = 1regular point: = 0saddle point: = -1monkey saddle: = -2
ter Haar Romeny, ISS 2014
The notion of scale appears in the size of the path.
Winding number = +1 extremum
Winding number = -1 saddle
ter Haar Romeny, ISS 2014
Generalised saddle point (5th order): (x+i y)5
The winding numbers add within a closed contour, e.g.A saddle point (-1) and an extremum (+1) cancel, i.e. annihilate.
Catastrophe theory
Winding number = - 4 monkey saddle
ter Haar Romeny, ISS 2014
ter Haar Romeny, ISS 2014
2 4 6 8 1010200
300
500
700
1000
Slopefor MR image: 1.66555
2 4 6 8 1010
1000
1500
2000
Slopefor white noise: 1.91549
The number of extrema and saddlepoints decrease as e-N over scale
Decrease of structure over scale scales with the dimensionality.
ter Haar Romeny, ISS 2014
Fertility Prospects
In most developed countries a postponement of childbearing is seen.
E.g. in the Netherlands: Average age of bearing first child is 30 years.
Computer-Assisted Human Follicle Analysis for Fertility Prospects with
3D Ultrasound
ter Haar Romeny et al., IPMI 1999
Application:
ter Haar Romeny, ISS 2014
The number of follicles decreases during lifetime
ter Haar Romeny, ISS 2014
Resting 0.03 mm initiation of growth> 120 days?
Early growing 0.03 - 0.1 mm
Preantral 0.1 - 0.2 mm basal growth~ 65 days
Antral 0.2 - 2 mm
Selectable 2 - 5 mm rescued by FSH window~ 5 days
Selected 5 - 10 mm
Dominance 10 - 20 mm maturation~ 15 days
Ovulation
A follicle’s life
ter Haar Romeny, ISS 2014
3D Ultrasound is a safe, cheap and versatile appropriate modality
Kretz Medicor 530D
ter Haar Romeny, ISS 2014
Two 3D acquisition strategies:
1. Position tracker on regular probe
2. Sweep of 2D array in transducer
Trans-vaginal probe
Regular sampling from irregularly space slices
ter Haar Romeny, ISS 2014
ter Haar Romeny, ISS 2014
Manual counting is very cumbersome Automated follicle assessment
• 2-5 mm hypodense structures• structured noise• vessels may look like follicles• ovary boundary sometimes missing
ter Haar Romeny, ISS 2014
Detection of a singularity (i.e. a minimum)
From theory of vector fields several important theorems (Stokes, Gauss) exist that relate something happening in a volume with just its surface.We can detect singularities by measurements around the singularity.
P
1-D: difference of signs of the gradient i zero crossing or extremum
yxi
,
i i
The surrounding of the point P are just 2 pointsleft and right of P 1D sphere.
ter Haar Romeny, ISS 2014
. iji jWd
1 2 2 1
2 21 1
.d d
d
. iji jd d In subscript notation:
where ij is the antisymmetric tensor.
01
10ij
ter Haar Romeny, ISS 2014
Example of a result:
1 cm
Dataset 2563, radius Stokes’ sphere 1 pixel, blurring scale 3 pixels
ter Haar Romeny, ISS 2014
Detection of follicle boundaries:
• generation of 200 - 500 rays in a homogeneous orientation distribution• determine most pronounced edge along ray by winding number focusing• fit spherical harmonics to get an analytical description of the shape• calculate volume and statistics on shape
Distance along rayDistance along ray
Sca
le
US inte
nsi
ty
Sca
le
Distance along ray
ter Haar Romeny, ISS 2014
3D scatterplot ofdetected endpoints
3D visualisation of fittedspherical harmonics function
ter Haar Romeny, ISS 2014
Validation with 2 bovine ovaria
• anatomincal coupes• high resolution MR• 3D ultrasound
Follicle#
xcenter
ycenter
zcenter
distance toneighbor(pixels)
volume fromspherical harmonics
(mm3)
volumefrom MRI
volumefrom
anatomyv00 99 51 35 25.4 259.7 250.0 262.1
93 28 44 45.0 28.4 27.0 29.8113 55 74 41.6 56.2 54.9 59.3
v01 33 41 66 25.3 242.347 22 75 44.5 34.064 49 44 38.9 54.7
v44 72 49 84 25.4 239.769 28 70 45.2 28.432 51 82 40.1 59.3
ter Haar Romeny, ISS 2014
Conclusions:
• 3D ultrasound is a feasible modality for follicle-based fertilitiy state estimation• automated CAD is feasible, more clinical validation needed• winding numbers are robust (scaled) singularity detectors• a robust class of topological properties emerges
ter Haar Romeny, ISS 2014
Multi-scale watershed segmentation
Watershed are the boundaries of merging water basins, when theimage landscape is immersed by punching the minima.
At larger scale the boundaries get blurred, rounded and dislocated.
ter Haar Romeny, ISS 2014
Regions of different scales can be linked by calculating thelargest overlap with the region in the scales just above.
ter Haar Romeny, ISS 2014
The method is often combined with nonlinear diffusion schemes
E. Dam, ITU
ter Haar Romeny, ISS 2014
Nabla Vision is an interactive 3D watershed segmentation tooldeveloped by the University of Copenhagen.
Sculpture the 3D shape by successively clicking precalculatedfiner scale watershed details.