Lecture 1 4

Lecture 1Lecture 144

Previous Works

2

Problem: Target Image SearchProblem: Target Image Search

A target is assumed to be Scaled, Rotated Version Template With Edges Distorted

Search on Images Database

Template Target

3

MethodologyMethodology: An Overview: An Overview

InspirationJain et al [1] , “Object Matchin

g Using Deformable Templates”

Our Algorithm Finding Hypotheses : MGH

T Peak Clustering :

Watershed Method Contour Matching : Smo

oth Membrane Fitting

Template Target Contours Hypotheses 3 Peaks in 3D Hough Space

Rejected Hypothesis

Accepted Hypothesis

Rejected Hypothesis

4

Finding Candidate Target:Finding Candidate Target:Modified Generalized HoughModified Generalized Hough

2

r

L,

xc,yc

= 0

•[Nimkerdpol and Madarasmi, 2001]•A line at the contour edge is extended in the direction until it meets the other end of the contour •In MGHT, the relation between r,l) is stored as a linked list in R-Table, not as r) like in GHT

r1, 1, 1, l1 r2, 2, 2, l2 r3, 3, 3, l3 r4, 4, 4, l4

0...19 15,180,195,99 9,179,219,101 8,177,216,102 9,176,198,100

20...39 17,160,23,5 14,159,38,7 18,161,175,62 15,162,195,95

30…49 19,165,31,53 20,170,8,52 22,167,15,52 18,159,158,12

… … … … …

340...359

23,105,346,11 24,103,165,11 21,102,346,18 22,104,195,24

5

MGHT: Rotation/ScaleMGHT: Rotation/Scale

2

L,

= 0 2

L,= 0

= 30-200 = -170 = 190 = 300-110 = 190

6

MGHTMGHT

θθβ c

L

L S c

Rotation Factor:

Scaling Factor :

New ref. Point :

xc = x + S r cos (

yc = y + S r sin (

7

Watershed for Peak ClusteringWatershed for Peak Clustering

1. Shed, by labeling, at the first level, calculate peaks of each label2. Increase to higher level, shed again

2.1 Meet an area of previous level, shed to that area2.2 Not meet any area of previous level, make a new area ,

calculate a new peak

8

Deformation : Contour MatchingDeformation : Contour Matching

Parameter : xyor (u,v) range -7, -6, …,0,… 6, 7

9

Coarse and Fine MatchingCoarse and Fine Matching

10

Matching AlgorithmMatching Algorithm

Energy FunctionUpdate (u,v) : Gibbs Sampler with simulated annealing

Template

Target Edge

11

Experiment on Image SearchExperiment on Image Search

Template Target Edge Map ResultHough Space

12


Target Edge Hypotheses 1st Match

2nd Match 3rd Match 4th Match

13


Template Target Edge Map The Best Match Hough Space

14

Threshold Selection: Guitar 1Threshold Selection: Guitar 1

3.986274 0.929011 2.705226Template Target

EdgeHypotheses

Threshold : 1.0 - 2.6

15

Threshold Selection: Guitar 1Threshold Selection: Guitar 1

2.165488 0.965049 1.755835

Threshold : 1.0-1.6

Template Target Edge Hypotheses

16

Threshold Selection: Vase 1Threshold Selection: Vase 1

1.799267 1.114566 5.074061

Threshold : 1.2-1.6

Template Edge Map Hypotheses

17


Template

TargetEdge

Hypotheses 0.868600 0.879799 3.799124

Threshold : 0.9-3.6

18


Template Target Edge Hypotheses

1.293034 1.452130 3.364521 4.4185782

Threshold : 1.5-3.2

19

Energy ThresholdEnergy Threshold

0

0.5

1

1.5

2

2.5

3

3.5

4

accepted rejected

guitar1

guitar 2

vase

bottle

lamp

20

Experiment on Image QueryingExperiment on Image Querying

Database Search for Circle shape

21

Experiment on Image QueryingExperiment on Image Querying

Database

Search for bulb shape

22

ConclusionConclusion

A deformation model Contour Matching A method for image search Future work: large image database, efficient

method for minimizing energy, coarse-and-fine approach to computer vison modules

23

Accurate 3-D surface map using stereo vision

This proposal research addresses 2 issues:

• Find an accurate 3-D surface map using stereo vision

• Combine surface from different views to a single 3-D object for CAD applications.

24

Combine surface from multiple views to a single 3-D object.

To combine multiple view, we need to find the rotation and transformation matrices from each camera combined to the world or reference co-ordination system.

==================Rotation : 0.7122 0.7019 0.0130 Rotation : 0.0386 -0.0207 -0.9990 Rotation : -0.7010 0.7120 -0.0418

Translation : 16.6342 Translation : 32.6633 Translation : 181.5649 ==================

25

Expect Result After Combine Multiple View

26

A Relaxation Method for Shape A Relaxation Method for Shape from Contours from Contours

Input Contour Images:1. Geodesics Contours Only2. Developable Surface (No Folds/Twists)3. Non-Accidental View

1. Place Grid Points in X and Y Direction to have Smooth2. Draw Horizontal and Vertical Lines through Grid to

Form a Regular Square Texture3. Use Shape from Texture to Obtained Surface Normals

27

Experiment 2Experiment 2

Step 1

Step 2 Step 3

Original

28

Shape from ContourShape from Contour

Figure 1. Example of various surfaces with geodesic contours.

(A) LINE DRAWING IMAGE (B) GRAY SCALE IMAGE

FIGURE 2. COMPARING THE PERCEIVED SHAPE OF OBJECTS IN LINE-DRAWING VS. SHADED, GRAY-SCALE IMAGES.

29


FIGURE 3. EXAMPLES OF OBJECTS THAT CONSIST OF PARTS OF

QUADRILATERAL DEVELOPABLE CURVATURE SURFACES.

Figure 5. Examples of images that consist of only line-drawings. Even without shading and texture, a human viewer can determine the shape of the object.

30


(a)Occluding contour

(b)Texture contour

(c)Shadow contour

(d) Geodesic Contour (e) surface fold

contour Figure 4. Types of Contours (a) Occluding Contour (b) Texture Contour (c) Shadow Contour (d)Geodesic Contour (e) surface Fold Contour

31


(A) (B)

(C) (D)

FIGURE 7. (A) ORIGINAL CONTOUR (B) INTERSECTION POINTS DERIVED

FROM RELAXATION METHOD (C) CREATE LINES BY USING SPLINE

TECHNIQUE (D) SHAPE FROM TEXTURE DETERMINES THE NORMAL VECTOR

FOR EACH INTERSECTION POINT.

32


(A) (B)

FIGURE 9. EXAMPLE OF SOME SURFACES CAN BE PERCEIVED FROM

TEXTURE. (A) UNDULATE SINGLE TEXTURE (B) UNDULATE TEXTURE NET

LIKELIHOOD

(a) (b) (c)

FIGURE 10. COMPUTING SHAPE FROM A TEXTURE PATCH. (A) ARBITARY

PATCH WITH INTERSECTING OUTLINES. THE ANGLE SHOWN IN (B) IS

MEASURED IN THE IMAGE PLANE. (C) VECTORS U AND V ARE 3-D VECTORS

ON THE SURFACE, AND N IS THE SURFACE NORMAL AT THEIR INTERSECTION.

V

N

33


2222

1111)H

~H~

()H~

H~

()H~

H~

()H~

H~

(E)c,r()c,r()c,r()c,r()c,r()c,r()c,r()c,r( PPPPPPPPh

2222

1111)V

~V~

()V~

V~

()V~

V~

()V~

V~

(E)c,r()c,r()c,r()c,r()c,r()c,r()c,r()c,r( PPPPPPPPv

vhcontinuity EEE

22 )V ~V ~()H~H~(E)c,r()c,r()c,r()c,r( PPPPsmoothness

smoothnesssmoothnesscontinuitycontinuity EEE

T h e c o n s i d e r e d v e c t o r s w i l l b e d e n o t e d a s s y m b o l s b e l o w :

F I G U R E 8 . S M O O T H N E S S A N D C O N T I N U I T Y D E F I N E D . P ( R , C ) I S T H E P O I N T

U N D E R C O N S I D E R A T I O N W I T H I T S N E I G H B O R S S H O W N .

1 . 1 P ( r

, c ) = P

P ( r - 1 , c )

P ( r , c - 1 ) P ( r , c + 1 )

P ( r + 1 , c )

)c,r()c,r(P PPH~)c,r( 1

)c,r()c,r(P PPH~)c,r( 1

)c,r()c,r(P PPV~

)c,r( 1

)c,r()c,r(P PPV~)c,r( 1

34


a n d

A N D T I L T A N D T H E S L A N T A R E :

a n d N o w i f t h e i n t e r s e c t i n g p h y s i c a l c u r v e s a r e l i n e s o f c u r v a t u r e , t h e y a r e p e r p e n d i c u l a r a t t h e i n t e r s e c t i o n . H e n c e t h e d o t p r o d u c t o f U a n d V i s z e r o . T h e n S o t h e N o r m a l v e c t o r N w i l l r e s t o n l y a u n k n o w n v a r i a b l e

a,,U 01 b,sin,cosV

sin,cosa,sinaN,N,NN zyx

x

y

N

Ntan 1 21222

1

/

zyx

z

NNN

Ncos

acos

b

sin,cos

aa

,sinaN12

35


(A) SURFACE WITH

GEODESIC CONTOUR (B) SURFACE WITH

OPTIMAL

INTERSECTION POINT

SET

(C) SURFACE WITH

TEXTURE FORMED BY

THE INTERSECTION

POINT SET

Figure 16. Preliminary Experimental Results.

(A) (B) (C) (D)

36

Contour SearchContour Search

2

r

L,

xc,yc

= 0

37


a) No Deformation b) Deformed by order

1 (N=M=1) c) Deformed by order 2 (N=M=2)

d) Deformed by order 3 (N=M=3)

Figure 2.3. Various deformation resolutions (copied from [1]).

Figure 2.4. An example of motion correspondence on 2 images

38


39

Contour MatchingContour Matching

E T o ta l = E C o a rs e G rid + 1E F in e G rid + 2E In te r G rid (1 4 )

E G r i d = E C o n t o u r M a t c h i n g + E M a t c h i n g S m o o t h n e s s ( 1 5 )

j)(i,E Grid = j)(i,Ij))v(i,jj),u(i,(iI Tangent TemplateTangentTarget + (1 6 )

j)Nbr(i, nm,

j)u(i,n)u(m, +

j)Nbr(i, nm,

j)v(i,n)v(m,

E I n t e r G r i d = E u _ i n t e r g r i d + E v _ i n t e r g r i d ( 1 7 )

E u _ in te rg rid (i, j)fin e = 0 , |u (i, j) c o a rs e s c a le – u (i, j)fin e | s c a le /2 (1 8 )

1 , o th e rw is e

40

Template

Target Image

Target edge

(f) Iteration 25, Energy=8.5315

(g) Iteration 26, Energy=7.0977

(h) Iteration 29, Energy=2.6243

(i) Result at Iteration 50, Energy=1.3657

41

(a) Template

(b) Transformed template

(c) Target Image

(d) Target edge Map

(g) Iteration 25, Energy = 8.269747

(h) Iteration 27, Energy = 4.643370

(i) Iteration 35, Energy = 1.257369

(j) Final result at iteration 40, Energy = 1.159310

42

Template E=1.081968 E=1.174364 E=1.332886 E=1.190744 (a) (b) (c) (d) (e)

(a) Template

(b) Target Image

(c) Target edge

(d) Hough space

(g) It 26, E=5.9126

(h) It 27, E=3.8052

(i) It 29, E=2.0100

(j) It 35, E=1.284

43

(a) Template

(b) Target Edge

(c) Hough Space

(d) 1st match, Energy=1.799267, rejected

(e)2nd match 1.114566, accepted

(f) 1st vote, Energy=5.074061, rejected

44

(a)

(b) Target edge

(c) Hough space

45

Paper InspectionPaper Inspection

46

Multi-Layer StereoMulti-Layer Stereo

47

Multi-Layer StereoMulti-Layer Stereo

48

Multi-View StereoMulti-View Stereo

Z

Y

X

inateFirstCoorddinateSecondCoor

t

t

t

Z

Y

X

eeeeeeeeeeee

eeeeeeeeeeee

eeeeeeeeeeee

Z

Y

X

23

22

21

2010322031

103223

22

21

203021

2031302123

22

21

20

)(2)(2

)(2)(2

)(2)(2

.123

22

21

20 eeee

49

Multi-View StereoMulti-View Stereo

50

Rubber Sheet InspectionRubber Sheet Inspection

51

Input Image ofBackIlluminatedRubber Sheet

Preprocessing1: NoiseRemoval viaGaussian Blur

Preprocessing2: BoundaryErosion

Partition imageinto rectangularregions

AutomaticThresholding fromHistogram PerRegion

Threshold

BinaryImagePerRegion

AssembledBinaryImage ofDefects

b.

c.

d.

e.

a.

b.

c.

d.

e.

a.

b.

52

Garment Layout using GeneticGarment Layout using Genetic

W

-กำ��หนดจำ��นวน Population และสุ่��มเพื่��อสุ่ร้��ง Population-คั�ดเล�อกำ Solution ที่��ด�ที่��สุ่�ดออกำม�จำ��นวนหน��ง-น�� Solution ด�งกำล��วม�จำ�ดเร้�ยงจำ�กำปร้ะสุ่ ที่ธิ ภ�พื่ด�สุ่�ดไปแย�สุ่�ดRepeat

-สุ่��มคั��แบบ Uniformly Distributed Random ที่��ม�คั�� 0 ถึ�ง 1-กำ��หนดให� r คั�อคั��ที่��สุ่��มได� และ p คั�อคั��คัว�มน��จำะเป'นIf r < p (กำ��หนดให� p = 0.9) Then

- สุ่��มเล�อกำ Solution จำ�กำ Population ด�วยกำ�ร้ใช้� Linearly Biased Random Number Generator ออกำม�จำ��นวนหน��ง- สุ่��มเล�อกำ Strip จำ�กำ Solution ที่�)งหมดที่��เล�อกำม�ด�วยกำ�ร้ใช้� Linearly Biased Random Number Generator- สุ่ร้��ง Solution ม�ใหม�ด�วยกำ�ร้ Crossover โดยน�� Strip แต่�ละ Solution ที่��ได�สุ่��มเล�อกำม�น��ม� Crossover กำ�น และ Mutate ด�วยกำ�ร้น��ช้ )นง�นที่��เหล�อม�ไล�จำ�ดว�งใหม�- ที่��กำ�ร้ Adjust โดยเล�อกำ Strip ที่��ปร้ะสุ่ ที่ธิ ภ�พื่สุ่,งม�จำ�ดเร้�ยง และ น��ช้ )นง�นที่��เหล�อไล�จำ�ดว�งใหม�- น�� Solution ใหม�ที่��ได�ไปเป'น Population

Else (r p)- สุ่��มเล�อกำ Solution จำ�กำปร้ะช้�กำร้ แล�วสุ่��มเล�อกำ Strip โดยกำ�ร้สุ่��มแบบ Uniform Distributed Random เพื่��อน��ม�Crossover และ Mutate ด�วยกำ�ร้น��ม�ไล�จำ�ดว�งใหม�- น�� Solution ใหม�ที่��ได�ไปเป'น Population-จำ�ดเร้�ยง Population ด�วยคั�� Fitness และเกำ-บ Solution ที่��ต่ ดล��ด�บไว�

Until ปร้ะช้�กำร้ไม�เปล��ยน หร้�อ ปร้ะสุ่ ที่ธิ ภ�พื่ที่��ด�ที่��สุ่�ดคังที่�� หร้�อ คัร้บจำ��นวนคัร้�)ง ที่��กำ��หนด

53

Garment Layout using Garment Layout using GeneticGenetic

54

Food Inspection: TextureFood Inspection: Texture

ห� Co-occurrence matrixที่�� d[3,3] และ d[-3,3]

กำร้องเฉล��ย

LoG

Lecture 1 4

Documents

Transcript of Lecture 1 4