1/28
A new morphological hierarchical representationof images
Application to text segmentation in document and natural images
Thierry Géraud Lê Duy Huynh Yongchao Xu
EPITA Research and Development Laboratory (LRDE), France
GT GeoDis 2016, Massilia
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
http://www.lrde.epita.fr/wiki/User:[email protected]://www.epita.frhttp://www.lrde.epita.fr
2/28
Objective
labeling
inclusiontree
( (12
34
5
6
input application
......1
2
3
4
5
6
Objective:
to compute a (useful / easy to use) hierarchical representation,
which structures the image contents by inclusion
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
3/28
Illustration
...
input object inclusion
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
4/28
Illustration
...A(
......
input ↝ label image + incl. tree
⇒
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
5/28
Application #1: Text detection in natural images
Qualitative evaluation (from Robust Reading at ICDAR):
input labeling (+ incl. tree) result
Quantitative evaluation:
in A Morphological Hierarchical Representation with Application to Text Segmentation in NaturalImages (submitted to ICPR 2016)
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
6/28
Application #2: Self-dual binarization of doc images
Used in the competition SmartDoc at ICDAR 2015:
↝
..."box"...
"background"
⇒input image tree (+label image) output
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
7/28
So What?
The point is:
it is not trivial to get such inclusion structures
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
8/28
Outline of the Proposed Solution
Laplacian
well-composedinterpolation labeling
inclusiontree
( (12
34
5
6
input application
...
based on a Laplace operator and a well-composed interpolation
(so there’s some topological considerations...)
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
9/28
About the Laplace Operator(s)
Zero-crossings of the Laplace operator ∆u = uxx + uyy :
Original image LoG 17 × 17
Morphological version →
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
10/28
Morphological Laplace Operator
Morphological Laplace Operator: ∆B = (δB − id) − (id − εB)
few sensitive to
contrast changes
uneven illumination
the structuring element B
zeros stick to data :-) but...
we have “spurious” zeros (Pb #1)
there is no “trivial” inclusion (Pb #2) :-(
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
11/28
Solving Pb #1
Laplacian
gradient
labeling
inclusiontree
( (1
234
5
6
auxiliary data
input application
...
main data...
we compute ∇B = δB − εB to ignore spurious ∆B = 0(not detailed here)
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
12/28
Solving Pb #2
Rationale:
we are interested in zero-crossings of the Laplace operator
we want to get a partition of the image (the label image)
we want the regions to arrange into an inclusion tree
we do not want to favor a given contrast
last we want to be fast...
↝ we want the 0-level lines given by the “tree of shapes” ofsign(∆Bu)
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
13/28
Level Sets and Shapes
Given an image u ∶ D ⊂ R2 → Rlower level sets: ∀λ, [u < λ ] = { x ∈ D ∣ u(x) < λ}upper level sets: ∀λ, [u ≥ λ ] = { x ∈ D ∣ u(x) ≥ λ}
A U B FD
EB
AC
F
O
D
E
A U O U C U F
a lower level set u a upper level set
with the cavity-fill-in operator Sat:
lower shapes: S
14/28
Tree of Shapes and Tree of Lines
We have an inclusion tree of shapes:
S(u) = S 2 2> 2
> 2
> 0
2
1
>4>
>
>
D
E
BA
C
F
O
an image u its tree of shapes S(u) its level lines
The blue pill: so we have an inclusion tree of level lines...
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
15/28
Illustration
surrounding 0-crossing
background
s 0-crossings pixels
e hole 0-crossing
e hole interior
y 0-crossingy pixels
e0-crossing
external
e pixels
∆◻(u) colorized S( sign(∆◻(u)) )
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
16/28
Tree of Shapes and Tree of Lines
The red pill: in the discrete settings we face some difficulties...
1 0 0 0 1 2
1 1 1 1 1 1 1 1
2 2
1 0
1 0
1 0
0 1
1 2
1 1
1 2
2 2
1 1 1 1 1 1 1 1
1
1
1
1
1
0 21
1 1
an image its level lines
what is the inclusion of shapes, i.e., of level lines?
consider the left inner square at 1, is it inside the 0s?
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
17/28
Tree of Shapes and Tree of Lines
A solution in the continuous setting:
1 0 0 0 1 2
1 1 1 1 1 1 1 1
2 2
1 0
1 0
1 0
0 1
1 2
1 1
1 2
2 2
1 1 1 1 1 1 1 1
1
1
1
1
1
0 2
1
1 1
an image its level lines
considering an upper semi-continuous function
level lines are Jordan curves and an inclusion tree exists
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
18/28
Tree of Shapes and Tree of Lines
A solution in the discrete setting:
1 0 0 0 1 2
1 1 1 1 1 1 1 1
2 2
1 0
1 0
1 0
0 1
1 2
1 1
1 2
2 2
1 1 1 1 1 1 1 1
1
1
1
1
1
11
1
11
1
1
1
1
01
0
01
1
1
0
0
01
1
1
1
1
1
1
01
0
11
1
1
011
1
11
1
1
1
1
2
2
11
1
1
2
2
2
1
2
1
1
2
2
2
1
1
2
2
1
1
1
1
1
1
1
11
1
11
1
1
1
1
2
2
2
1
1
2
2
2
2
2
1
1
2
2
2
2
11
1
1
2
2
2
2
1
1
2
2
01
1
01
1
1
1
1
01
1
01
1
1
1
11
12
2
2
2 2 2 12 22 2
2 2 12 22
11
1
11
1
1
1
1
an image its max-interpolated
It is as if we use c8 (resp. c4) for upper (resp. lower) shapes...
(D)
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
19/28
Tree of Shapes and Tree of Lines
A solution in the discrete setting:
1 0 0 0 1 2
1 1 1 1 1 1 1 1
2 2
1 0
1 0
1 0
0 1
1 2
1 1
1 2
2 2
1 1 1 1 1 1 1 1
1
1
1
1
1
11
1
11
1
1
1
1
01
0
01
1
1
0
0
01
1
1
1
1
1
1
01
0
11
1
1
011
1
11
1
1
1
1
2
2
11
1
1
2
2
2
1
2
1
1
2
2
2
1
1
2
2
1
1
1
1
1
1
1
11
1
11
1
1
1
1
2
2
2
1
1
2
2
2
2
2
1
1
2
2
2
2
11
1
1
2
2
2
2
1
1
2
2
01
1
01
1
1
1
1
01
1
01
1
1
1
11
12
2
2
2 2 2 12 22 2
2 2 12 22
11
1
11
1
1
1
1
an image its max-interpolated
we do have Jordan curves and an inclusion tree but we are not self-dual
we do not want to favor a particular contrast ↝ solution discarded!
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
20/28
A Self-Dual Digital Representation
A self-dual solution in the digital setting:
1 0 0 0 1 2
1 1 1 1 1 1 1 1
2 2
1 0
1 0
1 0
0 1
1 2
1 1
1 2
2 2
1 1 1 1 1 1 1 1
1
1
1
1
1
11
1
11
1
1
1
1
01
0
01
1
1
0
0
01
1
1
1
1
1
1
01
0
11
1
1
011
1
11
1
1
1
1111
2
1
1
1
2
2
2
1
1
2
2
1
1
1
1
1
1
1
11
1
11
1
1
1
1
2
2
11
1
1
201
1
01
1
1
1
1
01
1
01
1
1
1
11
1
11
1
11
1
1
1
1
1
1
1
1
1
1
1
1 1 11
1 1 11 111
1
1
1
1
1
1
1
1
1
1
1
1
2
1
2
1
11
1
1
1
2
21
1
1
an image a F-interpolated
interpolation obtained thanks to a front propagation method F
↝ How to make nD functions WC in a self-dual way (ISMM 2015)
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
21/28
About the F-Interpolation
7 1 139 11 15
3 5 37 1
9 11
3 5
13
15
3
98 8 8 8
88 855
57
7 7
1311
an image u the F-interpolation of u
This nD well-composed interpolation is invariant by:
contrast changes,
classical rotations and symmetries,
inversion (so it is self-dual).
...
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
22/28
About the F-Interpolation
7 1 139 11 15
3 5 37 1
9 11
3 5
13
15
3
98 8 8 8
88 855
57
7 7
1311
an image u the F-interpolation of u
Actually:
this interpolation is a digitally well-composed image,
it has a “purely self-dual” tree of shapes.Links btw the morphological ToS and WC gray-level images (ISMM 2015)
its level sets are Jordan curves.
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
23/28
Recap
Original image Morphological Laplacian ↝ WC
Hierarchical repr. (lab.+tree) App. (here node selection and grouping)
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
24/28
Computing the Hierarchical Structure
Laplacian
gradient
well-composedinterpolation labeling
inclusiontree
( (1
234
5
6
auxiliary data
applicationinput
(Label image + tree) computation.
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
25/28
Hierarchical Structure Computation
This is a simple front propagation blob labeling (yellowish) algorithm!with contour elimination and inclusion tree (parent) computation
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
26/28
Hierarchical Structure Computation
Properties:
linear time complexity,
parallelizable,
no ×4 factor: we can emulate the F-interpolation!
easy to code.
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
27/28
Conclusion
A hierarchical representation:
- data simplification,- easy to use.
Emphasis put on inclusion:
- useful for some applications.
From theory to practical results:
- self-duality of cross-sections in digital topology...- some effective results.
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
28/28
Bibliography
T. Géraud, E. Carlinet, S. Crozet, and L. Najman, “A quasi-linear algorithm to compute the tree ofshapes of n-D images.,” in Proc of ISMM, vol. 7883 of LNCS, pp. 98–110, Springer, 2013. [PDF]
S. Crozet and T. Géraud, “A first parallel algorithm to compute the morphological tree of shapesof nD images,” in Proc of ICIP, pp. 2933–2937, 2014. [PDF]
N. Boutry, T. Géraud, and L. Najman, “On making nD images well-composed by a self-dual localinterpolation,” in Proc of DGCI, vol. 8668 of LNCS, pp. 320–331, Springer, 2014. [PDF]
T. Géraud, E. Carlinet, S. Crozet, “Self-duality and digital topology: Links between themorphological tree of shapes and well-composed gray-level images,” in Proc. of ISMM, vol. 9082of LNCS, pp. 573–584, Springer, 2015. [PDF]
N. Boutry, T. Géraud, and L. Najman, “How to make nD functions well-composed in a self-dualway,” in Proc. of ISMM, vol. 9082 of LNCS, pp. 561–572, Springer, 2015. [PDF]
L.D. Huynh, Y. Xu, T. Géraud, “A morphological hierarchical representation with application totext segmentation in natural images,” Submitted to ICPR, 2016. [PDF]
T. Géraud, L.D. Huynh, Y. Xu (LRDE) A new morphological hierarchical repr. of images (GT GeoDis 2016)
http://www.lrde.epita.fr/~theo/papers/geraud.2013.ismm.pdfhttp://www.lrde.epita.fr/~theo/papers/geraud.2014.icip.crozet.pdfhttp://www.lrde.epita.fr/~theo/papers/geraud.2014.dgci.pdfhttp://www.lrde.epita.fr/~theo/papers/geraud.2015.ismm.pdfhttp://www.lrde.epita.fr/~theo/papers/geraud.2015.ismm.boutry.pdfhttp://www.lrde.epita.fr/~theo/papers/geraud.2016.icpr_submitted.pdfTop Related