A. Criminisi, T. Sharp and K. Siddiqui. Properties of our algorithm efficient on high-res./nD images...
-
Upload
christopher-dunlap -
Category
Documents
-
view
215 -
download
1
Transcript of A. Criminisi, T. Sharp and K. Siddiqui. Properties of our algorithm efficient on high-res./nD images...
A. Criminisi, T. Sharp and K. Siddiqui
Properties of our algorithmProperties of our algorithm
• efficient on high-res./nD images (~milliseconds)• easy to edit and fix• accurate (e.g. handling of thin structures)• robust to noise• edge sensitive• captures uncertainty (probabilistic output)
State of the art segmentation State of the art segmentation algorithmsalgorithms
• QPBO• Tree-reweighted mess. passing• Random Walker• Min-cut/max-flow• Belief propagation• Level Sets• Geodesic active contours• Region growing• K-means
Most based on complex energy minimization -> slow.
Input image User-entered brush strokes Likelihood of Fg v Bg
This is noisy!In order to obtain segmentation we need to encourage spatial smoothness
Notation
Point position
Image intensities
Output segmentation
Appearance likelihood
Appearance likelihood is computed from histograms of intensities accumulated under the two brush strokes
Legend• Green indicates high probability of Fg• Red indicates high probability of Bg• Grey for uncertain
A very efficient way of propagating image information around
Image Binary mask
unit tangent vector
Geodesic distance (Euclidean for =0)
Coronal CT view of r. kidney User-drawn brush stroke Binary mask of brush stroke Gradient and tangent vectors
Output geodesic distance D(p)
Output geodesic distanceOutput geodesic distance
Forward pass: (top-left to bottom-right)
Backward pass: (bottom-right to top-left)
Properties of algorithm • Contiguous memory access• Parallelizable• GPU-friendly
• no region growing• no FMM• no level set
InputInput
Image (8bpp) after W/L mapping!
GDT raster scan algorithmGDT raster scan algorithm
with
GPU algorithm implemented on NVidia processors using the CUDA language.GPU algorithm implemented on NVidia processors using the CUDA language.
Typical CT image resolution = 512 X 512
Timings
Downward pass:
with
…similarly for the other three passes.
The downward pass. The red column shows pixels processed by the current thread. The distance values in the top (green) row Have already been computed. Distances along the arrow directions Are computed from texture reads as in the equation on the left panel.
In 2D four passes are necessary: top-bottom, bottom-top, left-right, right-left
Downward pass:
1 2 3y
x
1 2 3
4 5 6
7 8 9
dire
ction
of ra
ster
sca
n
GPU algorithm implemented on NVidia processors using the CUDA language.GPU algorithm implemented on NVidia processors using the CUDA language.
Pass 1 of 6:In 3D six passes are necessary.
with
Pass 1 of 6:
y
xz…similarly for the other five passes.
How do we impose smoothness ?
Input binary mask M
withSigned distance from boundary
Signed distance Ds from boundary (in green) Signed distance Ds (zoomed)
For ease of explanation we focus on a toy 2D example here.
Geodesic morphology
Signed distance Ds from boundary (green)Input binary mask M
Geodesi
c d
ilati
on
Geodesi
c ero
sion
Eroded mask
Dilated mask
In each of the eroded and dilated masks part of the noisehas been removed. Now we need to combine the two so asto remove all of the noise.
Geodesic morphology, real example
Th
is is all e
xtre
mely
fast to
com
pu
te.
Distance Ds
Geodesic erosion
Geo. dilation , small d Geo. dilation , large d
Eroded maskDilated mask
Final, symmetric signed distance
Symmetric signed distance
The new distance is much smoother than the original one because the effect of noise has been reduced.
Original signed distance
The GSF operator produces the final, noise-free mask Ms as:
Input binary mask M Symmetric signed distance Thresholding at 0 Final noise-free mask
More examples with different noise patterns
Input Distance Filtered image Input Distance Filtered image
Binary case was
Real-valued caseinput GSF
GGDT:GGDT:GeneralizedGeneralizedGeodesic DistanceGeodesic Distance
Ms: output segmentation
GSF( )
Ms: Output of GSF for small
Ms: Output of GSF for large
smoothsmooth
lockedlocked
Larger values of produce smoother segmentations
The actual algorithm
Output of GSF for fixed
Appearance lik.
Soft input mask
Distance term with
Data likelihood
I. Compute pixel-wise likelihoods from user hints
combine
User brushes Appearance likelihood L a Distance term L d
II. Segmentation from pixel-wise likelihoods
Output of GSF for fixed
Geodesic symmetric distance
Segmentation via GSF
GSF( )
Appearance likelihood La
Output segmentation Ms
Distance signal Ld
Aortic aneurism
The aorta, bottom of heart and pelvis have been segmented in 3D by our technique.The whole process (including user interaction) takes only a few seconds.
Three views of the segmented aorta. Bony structures (Bg) are shown faded to provide spatial context. Input is CT.
Carotid arteries
The carotids have been segmented interactively by our technique. Segmenting such long and thin structures is usually a problem for other existing algorithms.
Three views of segmented carotids. Bony structures (Bg) are shown faded to provide spatial context. Input is CT.
Studying the interaction between aorta and spine
The aorta and its thin bifurcations have been segmented by our technique in only a few seconds.
Three views of segmented aorta. The spine and heart (Bg) are shown faded to provide spatial context. Input is CT.
A liver tumor
The tumor has been segmented with a single user click. The whole operationhas taken only a few milliseconds. Now that the tumor has been isolated statistics about itsdensity, shape and texture are easily computed (right panel).
Axial CT slice. The user segments a tumor in only a few milliseconds Automatic measurements of tumor
Segmenting different structures in MR images
Segmentation of noisy images
Very noisy input image. Input is CT. Our segmentation
Encouraging spatial smoothness is especially useful when dealing with very noisy images.
Very noisy input image. Input is MR Our segmentation
Segmentation of noisy images
Encouraging spatial smoothness is especially useful when dealing with very noisy images.
Testing accuracy in low-contrast images
Very low-contrast input image Our segmentation
The geodesic term in the definition of the geodesic distance enables segmentation in low contrast images.
Capturing uncertainty in low-contrast regions
In low gradient regions the algorithm correctly returns low confidence segmentation
In low gradient regions the algorithm correctly returns low confidence segmentation
Sharp segmentation (high confidence) becauseof strong gradients
Sharp segmentation (high confidence) becauseof strong gradients
Images and ground-truth from the standard GrabCut test dataset Not much of a difference in terms of accuracy.
[ Rother, C. Kolmogorov, V and Blake, A. GrabCut: Interactive foreground extraction using iterated graph cut. SIGGRAPH 2004. ][Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., Rother, C.: A comparative study of energy minimization methods for markov random fields. In: ECCV. (2006)]
• up to 60X speed up compared to min-cut.• with similar accuracy
[ Rother, C. Kolmogorov, V and Blake, A. GrabCut: Interactive foreground extraction using iterated graph cut. SIGGRAPH 2004. ]
[ Bai, X. and Sapiro, G. A geodesic framework for fast interactive image and video segmentation and matting. ICCV 2007, Rio, Brasil. ]
• up to 30X speed up compared to Bai et al., while avoiding topology issues.• also, Bai et al. do not impose spatial smoothness