Efficient Image Region and Shape Detection by Perceptual Region

Contour Grouping

Huiqiong Chen, Qigang Gao

Faculty of Computer ScienceDalhousie University

OutlineOutline

• Introduction

• Proposed Framework

• Experimental Results

• Conclusion

IntroductionIntroduction

• Region Detection– Aim to find coherent regions from image.

• Motivated by a variety of applications– Industrial inspecting systems – Vision based automation tasks – Region-based Image/Video retrieval

• Challenges– Region associated with perceptual meaning– Detection accuracy– Reasonable time complexity

IntroductionIntroduction

• Main solutions of Region Detection– Region based methods:

• Identify the region interior by calculating pixel similarities.

Difficult to extract accurate boundary and shape from pixel collections.

Sensitive to noise; Over-segmentation

– Boundary based methods: • Employ edge information to find region

boundaries. Pixel-level based (local information)

Time consuming

IntroductionIntroduction

• Goal of our research– Extract meaningful regions from image

and estimate their shapes without intensive computation.

• Proposed method:– Taking advantage of inherent structure

information carried by each Generic Edge Token (GET) feature, the perceptual structure of region shape can be obtained easily as well as region interior attributes.

OutlineOutline

• Introduction

• Proposed Framework

• Experimental Results

• Conclusion

Introduction of Generic Edge TokenIntroduction of Generic Edge Token

• GETs: Include Generic Segments (GS) and curve partition points (CPP).

• GSs: Perceptually distinguishable segments classified by descriptive features.

Categories of GSs The perceptual definitions of GSs

Introduction of Generic Edge TokenIntroduction of Generic Edge Token

• CPPs: Perceptually significant points where adjacent GSs meet and curve turning takes place.

Basic Categories of CPPs

CPP1 CPP2

CPP3 CPP4

CPPGS

Curve partition example

Key IdeasKey Ideas

• Hypothesis– The inside of a homogeneous region does not

have consistent boundaries.

• Region representation– Meaningful regions can be represented perceptual

GET-based closures in Perceptual Region/Closure hierarchy.

• Perceptual closures extraction – An image is transformed into GET space on the fly

represented by GET graph, which presents perceptual organization of GET associations.

– Region detection can be achieved by perceptually grouping closures in GET graph.

Region RepresentationRegion Representation

Perceptual region/closure concept hierarchy

e1

e2

e3

e4

e5

e6

e7

e1

e4

e3

e5

e6 e7

e8

e2

Object outline Basic regions Basic regionsObject outline

Object contour closures Basic contour closures

( A)(B) Object level

Meaningfulregion level

Perceptualclosure level

GET levelGS categories CPP catergories

e8

e9

Region Representation (Cont)Region Representation (Cont)

• Regions can be represented by two types of perceptual closures (GET-based):– Basic contour Closure

GET closure which can not be divided further into other closures.

– Object Contour Closure GET closure which describes the outline of an object/individual component of object.

Perceptual Closures ExtractionRegions Detection

System ArchitectureSystem Architecture

GET feature Extraction

Perceptual Closure Detection

GET graph Construction& Reduction

GET feature Maps

GET Graph

GET features can be extracted by the edge tracker.

Closure Definition

GET graph can be derived from GET feature map to code the perceptual structure of GET associations.

Perceptual closures are detected though perceptual cycle search in GET graph.

Image

GET closures Regions can be constructed based on closures.

Image regions GET-based closures

GET Feature ExtractionGET Feature Extraction

Original image Extracted GET features

GET Graph Construction and ReductionGET Graph Construction and Reduction

• Transform image into GET graph

– Convert GSs and CPPs to graph edges and vertices.– Remove noise edges not belonging to any cycle

GET-based closures extraction graph cycle searchBasic Contour Closure Basic cycleObject Contour Closure Outline cycle

(a) GET features of an object in image (b) GET graph

V2

V3

V4

V5V6

V7

V8V9

V10

V1

V0

Closure Detection Flow ChatClosure Detection Flow Chat

Starting edge selection

Result classification

Closure detection by cycle search

Starting edges

Cycles/Closures

track approximate adjacent edges along the GET graph and select proper edges to group the contour closures. (Cycle search strategy )

Classify result closures by its perceptual type.

Select search initial edge for closure detection (get all closures with minimal efforts).

GET Graph

Classified closures

Determine Closure’s Initial edgeDetermine Closure’s Initial edge

• Starting edge selection– Closure searches only start from selected edges in

stead of from all graph edges.– It reduces detection computation burden.– Edges not belonging to spanning trees are selected.– The correctness of selection is proven by Lemma 1.

• Lemma 1: all perceptual closures can be obtained by tracking in graph starting from edges not belonging to spanning trees. That is, edges not belonging to spanning trees are considered as search starting edges.

Perceptual Cycle SearchPerceptual Cycle Search• Observation

Basic cycle is a cycle without sub-cycle inside

Outline cycle is cycle without any cycle outside.

The outmost edge in one direction is the innermost edge in the other direction

e6

e1

e2

e3e4

e9

e5

e8

e7

e10

e11e12

v9

v7v2

v4

m2

v3

v8

v11v10

m1

Starting edge

Cycle search forming perceptual closures

Basic cycle edge is Innermost edge in each step of tracking

Outline cycle edge is the outmost edge in each step of tracking

Perceptual Cycle Search (Cont)Perceptual Cycle Search (Cont)

• Cycle Search strategy– Starting from closure edge ei with endpoints vm, vn

select proper edges in each step of tracking so that form a path p = (vm)<ei, ..., ek, ei >(vn) in GET graph .

– In each step of path selection, select the innermost edge in given direction (clockwise or anticlockwise) as next cycle edge among all adjacent edges.

• Strategy validity proof– Lemma 2: A closure extracted using perceptual

cycle search algorithm must be a perceptual closure.

Perceptual Closure DetectionPerceptual Closure Detection

• Detect all perceptual closures in GET graph via Cycle Search algorithm Starting from each edge not included in any spanning

tree, perform both anticlockwise and clockwise cycle searches by applying the cycle search strategy.

• Detection method validity– Lemma 4: By using this method, (1) all extracted

closures are perceptual closures; and (2) all perceptual closures are extracted from graph.

Closure ClassificationClosure Classification

Classify closures into two perceptual types• Classification criterion: Total, summation of Included

angles of the polygon simulating closure in cycle search direction.

For a n-edge closure • if Total = 180*(n-2), the closure is basic contour closure; • if Total =180*(n+2), the closure is object contour closure.

e6

e1

e2

e3e4

e9

e5

e8

e7

e10

e11e12

v9

v7v2

v4

m2

v3

v8

v11v10

m1e1

e2

e3e4

e5

e7

e10v9 m1

v2v3

v4

m2v7

e6

e1

e2

e3e4

e5

e8

e11e12

v12v10

m1

v8

v7m2

v4

v3 v2

Simulate closure by polygon: (b) clockwise search from v7 of e7. (c) anticlockwise search from v10 of e11

ExampleExample

Object contour closure Basic contour closures

Original imageGET graph

OutlineOutline

• Introduction

• Proposed Framework

• Experimental Results

• Conclusion

Result Samples Result Samples

Original image Extracted GET features

GET-based region contour Filled regions

•A set of test images is selected with different characteristics

Result EvaluationResult Evaluation

Image No

Number of correct detections

Number of Error detections

Number of missing detections

Number of closures in image

Percentage of correct detection

1 22 0 0 22 100%

2 33 2 1 36 91.7%

3 23 3 1 26 88.5%

4 18 0 0 18 100%

5 32 1 7 39 82.1%

6 15 1 1 16 93.8%

7 27 1 5 33 81.8%

Table. Experiment result statistics. Average correctness= 91.13%.

OutlineOutline

• Introduction

• Proposed Framework

• Experimental Results

• Conclusion

ConclusionConclusion

• Region detection is converted to GET graph search so that the computation is on GET feature level in stead of image pixel level.

• It achieves high accuracy of detected regions without intensive computation.

• It is suitable for segmenting regions with arbitrary shapes. The shape structure can be estimated by GET types.

• Both object outline contours and their components can be detect with the association information.

Thank You for Your Attention

Questions?

Region and Boundary integrationRegion and Boundary integration

Integrate boundary and region information to achieve a better detection result.

• Use pre-extracted edge information in region-based processing.– Decision criterion control– Guidance of seed pixel selection

• Fuse results of region and boundary based methods by combining their outputs. – Over-segmentation merge– Boundary refinement

GET Graph ReductionGET Graph Reduction

• Aim to reduce the computation cost of closure extraction in GET graph.

• Remove noise edges not belonging to any cycle.

(a) GET graph with noise edges (b) Reduced GET graph

If two vertices are close within a small region (Normalized Distance dist(vi, vj) < T):

Bridge edge gaps by virtual vertex-Virtual combination of two real vertices

Bridge Edge Gaps in ClosureBridge Edge Gaps in Closure

edge gap

edge gap

e6

e1

e2

e3e4

e9

e5

e8

e7

e10

e11e12

v9

v7

v2

v4

v3

v8

v1

v5v6

v10v11

v0

virtual vertex m1

virtual vertex m2

(a) GETs in image with edge gaps (b) GET graph with virtual vertices

Images used in TestImages used in Test

Result SamplesResult Samples

Comparative ResultsComparative Results

P. Bonnin, etc (1989) Y. Xiaohan, ect (1992)

D. Sinclair (1999) F. Chan, ect (1996)

Future ResearchFuture Research

• Introduce other features such as color distribution and texture in processing to increase the robustness of detection.

• Remove image noises and edge gaps in advance by additional knowledge or perceptual principal.

• Encode region with a complete set of region attribute descriptions.