Data Structures and Image Segmentation

Luc BrunL.E.R.I., Reims University, Franceand Walter KropatschVienna Univ. of Technology, Austria

SegmentationSegmentation: Partition of the image into homogeneous connected componentsS1S2S3S4S5

SegmentationProblemsHuge amount of dataHomogeneity: Resolution/Context dependentNeedsMassive parallelism

Hierarchy

Contents of the talkHierarchical Data Structures

Combinatorial Maps

Combinatorial Pyramids

Regular Pyramids

Matrix-PyramidsStack of images with progressively reduced resolutionLevel 0Level 1Level 22x2/4 PyramidLevel 3

M-PyramidsM-Pyramid NxN/q (Here 2x2/4)NxN : Reduction window. Pixels used to compute fathers value (usually low pass filter)

q : Reduction factor. Ratio between the size of two successive images.

Receptive field: set of children in the base level

M-PyramidsNxN/q=1: Non overlapping pyramid without hole (eg. 2x2/4)

NxN/q1: Overlapping pyramid

Regular PyramidsAdvantages(Bister)makes the processes independent of the resolution.Drawbacks(Bister) : RigidityRegular GridFixed reduction windowFixed decimation ratio

Irregular PyramidsStack of successively reduced graphs

Irregular Pyramids [Mee89,MMR91,JM92]From G=(V,E) to the reduced graph G=(V,E)Selection of a set of surviving vertices VV

Child Parent link Partition of V

Definition of E

Selection of Roots

Stochastic PyramidsV : Maximum Independent Set

maximum of a random variable[Mee89,MMR91]a criteria of interest[JM92]

Stochastic PyramidsChild-Parent link :

maximum of a random variable[Mee89,MMR91]a similarity measure[JM92]

Stochastic PyramidsSelection of surviving edges ETwo father are joint if they have adjacent children

Stochastic PyramidsSelection of Roots:

Restriction of the decimation process by a class membership function [MMR91]

contrast measure with legitimate father exceed a threshold [JM92]

Stochastic Pyramid [MMR91]Restriction of the decimation process : Class membership function

Stochastic PyramidsAdvantagesPurely local Processes [Mee89]Each root corresponds to a connected region[MMR91]Drawback Rough description of the partition

Formal DefinitionsEdge Contraction

Formal DefinitionDual GraphsTwo graphs encoding relationships between regions and segments

Formal DefinitionDual GraphsTwo graphs encoding relationships between regions and segments

Dual Graphs Advantages (Kropatsch)[Kro96]Encode features of both vertices and faces

Drawbacks [BK00]Requires to store and to update two data structuresContraction in G Removal in GRemoval in G Contraction in G

Decimation parameterGiven G=(V,E), a decimation parameter (S,N) is defined by (Kropatsch)[WK94]:a set of surviving vertices SVa set of non surviving edges NEEvery non surviving vertex is connected to a surviving one in a unique way:

Example of Decimation: S:N

Decimation parametersCharacterisation of non relevant edges(1/2) df = 2

Decimation parametersCharacterisation of non relevant edges(2/2) df = 1

Decimation ParameterDual face contractionremove all faces with a degree less than 3

Decimation ParameterEdge contraction: Decimation parameter (S,N) Contractions in GRemovals in GDual face contraction : Dual Decimation parameterContractions in GRemovals in G

Decimation parameterCharacterisation of redundant edges requires the dual graph Dual graph data structure (G,G)

Decimation ParameterAdvantagesBetter description of the partition

DrawbacksLow decimation Ratio

Contraction KernelsGiven G=(V,E), a Contraction kernel (S,N) is defined by:a set of surviving vertices SVa set of non surviving edges NESuch that:(V,N) is a forest of (V,E)Surviving vertices S are the roots of the trees

Contraction kernelsSuccessive decimation parameters form a contraction kernel

Example of Contraction Kernel: S:N,,

Example of Contraction kernelRemoval of redundant edges: Dual contraction kernel

Hierarchical Data Structures / Combinatorial Maps M-PyramidsOverlapping Pyramids Stochastic Pyramids Adaptive Pyramids Decimation parameter Contraction kernel

Combinatorial Maps Definition G=(V,E) G=(D,,)decompose each edge into two half-edges(darts) :D ={-6,,-1,1,,6}

Combinatorial Maps DefinitionG=(D,, ) : vertex encoding*(1)=(1,*(1)=(1,3*(1)=(1,3,2)12-13-34-45-5-26-6

Combinatorial MapsPropertiesComputation of the dual graph :G=(D,,) G=(D, = , )

The order defined on induces an order on

*(-1)=(-1,*(-1)=(-1,3*(-1)=(-1,3,4*(-1)=(-1,3,4,6)15-5-4-3-662-2-143

Combinatorial MapsPropertiesComputation of the dual graph :G=(D,,) G=(D, = , )

*(-1)=(-1,*(-1)=(-1,3*(-1)=(-1,3,4*(-1)=(-1,3,4,6)1-123-34-45-5-26-6

Combinatorial MapsPropertiesSummaryThe darts are ordered around each vertex and faceThe boundary of each face is ordered The set of regions which surround an other one is orderedThe dual graph may be implicitly encodedCombinatorial maps may be extended to higher dimensions (Lienhardt)[Lie89]

Combinatorial Maps/Combinatorial Pyramids Combinatorial MapsComputation of Dual GraphsCombinatorial Maps propertiesDiscrete Maps[Bru99]http://www.univ-st-etienne.fr/iupvis/color/Ecole-Ete/Brun.ppt

Removal operation

G=(D,, )dD such that d is not a bridgeG=G\ *(d)=(D, , )

d-d

Removal OperationExample-13-34-4-26-612 d=5

Contraction operations

G=(D,, )dD such that d is not a self-loopG=G/*(d)=(D, , )

d-d

Contraction operations

Preservation of the orientation1234dcba1234dcba

Basic operationsImportant PropertyThe dual graph is implicitly updated-13-34-45-5-26-612 d=5d=5removalcontraction1

Contraction KernelGiven G=(D,, ), KD is a contraction kernel iff:K is a forest of GSymmetric set of darts ((K)=K)Each connected component is a tree

Some surviving darts must remainSD=D-K

Contraction Kernel Example123456789101112131415161718192021222324K=

Contraction Kernel Example

K=

Contraction KernelHow to compute the contracted combinatorial map ?What is the value of (-2) ?

Contraction KernelHow to compute the contracted combinatorial map ?What is the value of (-2) ?124131415-2-1-13177

Contraction KernelConnecting WalkGiven G=(D,, ) , KD and SD=D-KIf d SD, CW(d) is the minimal sequence of non surviving darts between d and a surviving one.

The connecting walks connect the surviving darts.

Contraction KernelConnecting Walk123456789101112131415161718192021222324-2-1CW(-2)=-2.-1.13.17.21.10

Contraction KernelCW(-2)=-2,-1,13,17,21,10

123456789101112131415161718192021222324-2-12356891112151619202324-218142274-11-11

Contraction KernelConstruction of the contracted combinatorial map.For each d in SDcompute d: last dart of CW(-d)(-d)=(d)(d)=(-d) = (d)

Extensions(1/2)Dual contraction kernelReplace by

Successive Contraction kernels with a same typeConcatenation of connecting walks

Extensions(2/2)Successive contraction kernels with different typesFrom connecting walks to Connecting Dart Sequences Label Pyramids: for each dart encodeIts maximum level in the pyramid (life time)How its disappear (contracted or removed)

ConclusionGraphs encode efficiently topological features. Combinatorial maps:Encode the orientation Provide an implicit encoding of the dualMay be generalised to higher dimensionIrregular Pyramids overcome the limitations of their regular ancestors

Combinatorial Pyramids