1
Variational Approaches and Image Segmentation
Lecture #5Lecture #5Hossam Abdelmunim1 & Aly A. Farag2
1Computer & Systems Engineering Department, Ain Shams University, Cairo, Egypt
2Electerical and Computer Engineering Department, University of Louisville, Louisville, KY, USA
ECE 643 – Fall 2010
Variational Formulation in Computer Vision and Image Analysis
Variational Formulation in Computer Vision and Image Analysis
Image/Curve/Surface is required to evolve to minimize a Image/Curve/Surface is required to evolve to minimize a certain energy function.certain energy function.
The change of the Image/Curve/SurfaceThe change of the Image/Curve/Surface is described by a is described by a partial differential equation (PDE).partial differential equation (PDE).
In image segmentation, the evolution continuous until the In image segmentation, the evolution continuous until the curve/surface hits the object (region) boundaries.curve/surface hits the object (region) boundaries.
Advantages of Introducing the Variational Formulation to the Computer Vision FieldAdvantages of Introducing the Variational Formulation to the Computer Vision Field
Modeling images in continuous time domain which simplifies the formalism and makes it grid independent.
Research on numerical analysis makes it possible to achieve high speed, accurate, and stable solutions of the derived PDE’s.
Viscosity solutions provide a framework for dealing with non-smooth signals used with these approaches.
Formal analysis in this area give the possibility of providing successful algorithms and useful theoretical results like existence and uniqueness of the solution.
Euler Lagrange & Gradient Descent (Recall)
Euler Lagrange & Gradient Descent (Recall)
RRm :Given a real function:
dLE ),(
Define the following energy functional:
)(
Ldiv
LE
The Euler Lagrange equation will be:
0
E
E
t
The Gradient Descent will be:
Classical Deformable ModelsClassical Deformable Models
Parametric Planar Curve RepresentationParametric Planar Curve Representation
C(0)
C(0.5)
C(1)
,)](),([)( TpypxpC ]1,0[p
Parametric Form:
Arc Length is:
p
p dpCps0
||)(
Snake/Contour EvolutionSnake/Contour Evolution
ExternalInternal EEE
The contour is required to evolve minimizing the following energy function:
Contour characteristics Image data (force)
Internal EnergyInternal Energy
1
0
22
21 )||)(||)((5.0)( dpCpwCpwCE pppInternal
w1 and w2 are non negative.
The term weighted by w1 reduces curve length and removes ripples.
The second term controls the bending and smoothness of the contour.
External EnergyExternal Energy
1
0
))(*()( dpCIGCEExternal
G is a Gaussian smoothing filter with width .
λ is the magnitude of the external force .
This term derives the contour towards the object boundaries.
Vector Valued PDEVector Valued PDE
Using calculus of variation, the contour evolves according to the following equation:
))*(()()( 22
2
1 IGCwp
Cwp
C Cpppt
))*(())](())(([5.0 2222
222
1 IGyxx
wp
yxx
wp
x xpppppp
ppp
t
))*(())](())(([5.0 2222
222
1 IGyxy
wp
yxy
wp
y ypppppp
ppp
t
uv
Parametric Form:
,)],(),,(),,([),( TvuzvuyvuxvuS
]1,0[]1,0[),( vu
Energy:
Parametric Surfaces Representation and Evolution
Parametric Surfaces Representation and Evolution
1
0
1
02
022
20
211
201
210
]||),(||),(
||),(2||),(||),([)(
dudvSvuwSvuw
SvuwSvuwSvuwSE
vvuu
uvvuInternal
1
0
1
0
))(*()( dudvSIGSEExternal
Complete the derivation.
ExampleExample
Aorta segmentation using active contours
Aorta segmentation using active contours
Problems of Classical Deformable ModelsProblems of Classical Deformable Models
Dependency on the parameterization.
Troubles with topology changes (merging/splitting).
Initialization needs to be near the final solution (a big problem in 3D).
Tuning of the weighting coefficients in the PDE (w’s, λ).
Level SetsLevel Sets
Curve Representation using Level SetsCurve Representation using Level Sets
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
x
y
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
x
y
)1,,( tyxZ
)2,,( tyxZ
Zero Level Set or Z=0 plane
Resulting Curve
(Evolving Front)
Invented by Osher and Sethian 1985, to overcome disadvantages of classical deformable models.
Add extra dimension to the problem
Topology ChangeTopology Change
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
x
y
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
x
y
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
x
y
)0,,( tyxZ )1,,( tyxZ )2,,( tyxZ
Topological changes in the evolving front are handled naturally. No parameterization is needed.
This representation allows merging and breaking of the front as t advances
Level Sets and The Signed Distance Function
Level Sets and The Signed Distance Function
17
outsideifXD
boundarytheon
insideifXD
X
)(
0
)(
)(
• D is the min Euclidean distance between X and the contour.
• Ф is continuous and differentiable around the zero level.
Example of Signed Distance FunctionsExample of Signed Distance Functions
1 2
1 2 NwN
Example of Image RepresentationExample of Image Representation
Image Representation
Applications of the Distance Transform (DT)Applications of the Distance Transform (DT)The DT is a fundamental operator in shape analysis, having numerous applications,some of which are briefly listed bellow.•Separation of overlapping objects through watershed segmentation.•Computation of morphological operators.•Computation of geometrical representations and measures, such as skeletonization.•Shape matching.•Shape measures related to distance.•Other interesting areas in which the DT has been applied are:
• image registration• subpixel contour tracing• multidimensional data analysis (classification, clustering)• image enhancement• ray-tracing optimization• embedding surface for level set-based methods• efficient belief-propagation• multiphase flow simulations• medical image analysis• analysis of interaction between biological structures
Signed Distance Generation ComplexitySigned Distance Generation Complexity
So, to compute the distance function you need to get the closest point on the boundary/curve for each position in the domain.
This process is very time-consuming especially for volumes in 3D and higher.
Some approximation methods are used to expedite this process. See the following slides.
Distance Transform GenerationFaster Algorithm
Distance Transform GenerationFaster Algorithm
∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞∞ ∞ ∞ 0 0 0 0 0 ∞ ∞ ∞ ∞∞ ∞ ∞ 0 ∞ ∞ ∞ 0 ∞ ∞ ∞ ∞∞ ∞ ∞ 0 ∞ ∞ ∞ 0 ∞ ∞ ∞ ∞∞ ∞ ∞ 0 ∞ ∞ ∞ 0 ∞ ∞ ∞ ∞∞ ∞ ∞ 0 ∞ ∞ ∞ 0 ∞ ∞ ∞ ∞∞ ∞ ∞ 0 0 0 0 0 ∞ ∞ ∞ ∞∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞
1. Set boundary points to zeroes while others to ∞.2. Initialize an auxiliary set of pixels (usually a queue), the so-called contour set, with
the boundary pixels and their zero values.
3. While the contour set is not empty do the following:-a) Remove a pixel from the contour set . Let us call it the central pixel (xp,yp) now.b) For every pixel (xq,yq) of the neighbors of the central pixel, do the following:-
I. Compute the distance to the central point as: dd=|xp-xq|+|yp-yq|;II. Compute a temporary distance as : dtq=min(dist(p)+dd,dist(q));III. If dtq < dist(q), add the pixel q to the contour set;IV. Dist(q)=dtq;
Distance Transform GenerationFaster Algorithm (Cont…)
Distance Transform GenerationFaster Algorithm (Cont…)
• Note that computing the distance as the difference is faster than the square root function.
• Also, this methodology updates the distance values around the boundary points/pixels until there are no points in the queue.
• Try this method with the dynamic implementation using linked lists. It will be much faster than the ordinary methods….
Assignment: Implement using C++. Apply it for 2D as well as 3D
shapes.
Top Related