EIGEN-CSS SHAPE MATCHING

AND RECOGNIZING FISH IN UNDERWATER. VIDEO

All riglrts rcw~~\rc.tl. 'l 'l~is \vork Ilia!: 11ot I)(,

~q) ro t l l~c :c~t l ill \\-lrol(~ o r i l r p r r t . I)?- ~ ) l ~ o t o c q ) y

o r 01 l ~ c , r I I K Y I I I S . \\.it I I O ~ I I t 1 ~ ' I ) ( ~ ~ I I I ~ S S ~ ~ I I of' I IN' 21111 1101..

Date Approved: April 5,2007

SIMON FRASER U N ~ Y E M ~ Y I i bra ry

DECLARATION OF PARTIAL COPYRIGHT LICENCE

The author, whose copyright is declared on the title page of this work, has granted to Simon Fraser University the right to lend this thesis, project or extended essay to users of the Simon Fraser University Library, and to make partial or single copies only for such users or in response to a request from the library of any other university, or other educational institution, on its own behalf or for one of its users.

The author has further granted permission to Simon Fraser University to keep or make a digital copy for use in its circulating collection (currently available to the public at the "Institutional Repository" link of the SFU Library website <www.lib.sfu.ca> at: ~http:llir.lib.sfu.calhandle118921112>) and, without changing the content, to translate the thesislproject or extended essays, if technically possible, to any medium or format for the purpose of preservation of the digital work.

The author has further agreed that permission for multiple copying of this work for scholarly purposes may be granted by either the author or the Dean of Graduate Studies.

It is understood that copying or publication of this work for financial gain shall not be allowed without the author's written permission.

Permission for public performance, or limited permission for private scholarly use, of any multimedia materials forming part of this work, may have been granted by the author. This information may be found on the separately catalogued multimedia material and in the signed Partial Copyright Licence.

The original Partial Copyright Licence attesting to these terms, and signed by this author, may be found in the original bound copy of this work, retained in the Simon Fraser University Archive.

Simon Fraser University Library Burnaby, BC, Canada

Revised: Spring 2007

Abstract

Acknowledgments

Contents

. . Approval 11

Contents vii

List of Figures xi

List of Algorithms xv

2 Previous Work 8

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. 1 I I I I I ~ I I I 1101s !I

vii

3 Shape Rctricvill with Eigen-CSS Search

.i . 1 1111 rotl11c.1 i011 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2 Syllol)sis o f CSS I\.latc.lling 1)y Col~t~ollr A'laxil~~a . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 . 1 CSS Ilcl)rosc:~ltatio~i

2 2 I\Iilt(.lli~~g 1)y ( :SS Ck)llt . ollr A. I a~ i l l~ i i . . . . . . . . . . . . . . . . . . . .

2 . 3 (:lass hI.~l.(.l~il~g 13~;111latiol1 hlcbtl~otl . . . . . . . . . . . . . . . . . . .

3 N t ~ t c:hi~~g 1)y Eipp I-CSS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3 . 3 . I E:igcl~l~pit(.(.. PCA vi;) SVD . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . 3 . 2 i\Iilrgi1lnl-Sl1111 F'('i\tllr~ VO(~OI .S

3 . 3 l'll;~sc Cbrrol;it.ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3 . 4 AIirror rvfIc~.t iom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 . 5 i\lgol.itl~ll~ St r l~(. t l~r(:

1 likl)c~ril~~c~~lf s ailtl Rc~i111ts . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4 . 1 .li. st l1at.a Svts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 . 2 I l r~p l r :~~m~ta l io11 Jlc.t.ilils . . . . . . . . . . . . . . . . . . . . . . . . . .

1 l+;\;.rll~;\(. io11 I)? Cliws h.Iatc. l~ing . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.4 Itr~slllts

5 h I o t i t i o ~ ~ for I - I - S I f : r s . . . . . . . . . . . . . . . . . .

5 . o i c l ~ s i o ~ ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4 Rccognizir~g Fish iu U ~ ~ l e r w a t c r Video 57

4. 1 11iI 1~)(111(.tiol1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

1 . 1 . I 1'11vio11s \\C)rlc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

1.2 Ap~)voa(.ll . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . (iO

2 . 1 hlotl(~1 p , ( ' l i ( ' r<~t io~~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . (il

4.2.2 T)r.li)rtlialk ~ ( ~ I I I I ) I N I ( ~ ~ ~ ~ i l t ( . t i i ~ l g . . . . . . . . . . . . . . . . . . . . . . (3

I . 7 i x t ~trc~-l>wsc~l c.Iasific at ioli . . . . . . . . . . . . . . . . . . . . . . . . (i5

1 . l<c*slllls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . (i(i

4.3 o l s i i . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

List of Tables

List of Figures

1 Si11111)1(' 01' 1 ) 0 1 1 1 1 ( 1 i l i . ~ ( . I~~ .vc 's . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2 ( ? I ) : Ga~~ssiai l sirloothi~ig 1)roc:c:ss ol' il c:losc!tl c.11rvo sllo\vl\ a t t l ~ : I d t 111ost

liglll'~.. ( 1 ) ) : ' I ' l l ( : ( ~ O l ' l ' c ! ~ ~ ) O l l t l i l l g (~111'\:;1~.111'(: ~(:ilI(' Sl)il('(' i l l l i l g ( ' . . . . . . . . . . . %!I

. t l i~tal) i~sc~ wit l i I :$I Lisli tli\.i(lul i l l 1 o 17 c:lassc~s. I:\vi.y 1 .o~ . rc'l)i,c:sc~il s >I ( ' I ~ I s s

o f MI ( s c ~ [(iO].) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

Alatc-l~i~rg r c w ~ l t s for ;I lisl~ i l l t l i l t i ~ l ) ; ~ s c ! I : 11s i11g ~ I I I . illg0~itl1111. 'L ' I IP to!)-left

(WIW: sl~o\\:s 1 . 1 1 ~ i111)11[, fis11. ' I ' I I c r ( s t o S t . 1 1 ~ lish sl~o\v t lw 1 ) v s t 15 1 1 i i 1 t d 1

1.c .sr11ts . l i ~ ~ l k ( ' t l 1)). tl~c!ir E I I ( . I ~ ( I C ~ I I I ( l i ~ t i l ~ ~ ( : ( : to t 1 1 ~ i ~ i l ) ~ ~ t lisl~ i l l t 11(' ( \ i g ( \ ~ ~ s l ) t l ( v

r . S I I ~ ) S I ) ~ N Y > . 1 I f i l s I I I S I I l l O W I t i . . . . . . . . . .

( i t ) : AI I i111ilg(1 i l l ( l i l I . i l l ) i l s ( ! :I ( i ( : l ~ ~ ) ~ ) l ) ( ' ~ - O I .giS'): (1)): 'L'11(' ~ I i ~ l ~ ( l i ~ ~ . ( l - l ( ' ~ l g t I ~ ( ~ 1 1 -

t o t ~ r . ( ( e l : ' I ' I I ( ; ( . o r 1 . ( : ~ 1 ) o 1 i ( l i 1 1 g CSS i ~ ~ ~ a g c . ( ( I ) : '1'11(! ( , o r ~ . ( > s l ) o ~ ~ ( l i ~ ~ g f v i l t , ~ ~ r t >

\ . (Y;~,oI . : f 1 1 0 ~ ) I I ~ I S ( ~ - ( : O I ~ I ~ ( ~ ~ ~ I ~ ( ~ ( ~ I ~ I ~ I I . ~ ~ I I ~ I ~ - S I I I I I ( ~ ) I I ~ ~ ) o I I ( ~ I I ~ is S ~ I O \ V I I i l l I ) ~ I I v . i111(1

t11c ~ O W - S I I I I I is s l l o w l ~ i l l ~? ; I .WII , ( IRsII(xI . . . . . . . . . . . . . . . . . . . . . .

HOC c . 1 1 1 . v ~ lor t.Ilc! c : l i l s s 0 f i s l l s l ~ o w ~ i i l l I:ig~~rc: ;I.li. . . . . . . . . . . . . . .

I,('ft: P l O t . t i l l g t l l ( ' l l l i l t ( ' l l i l 1 ~ il\!('l.ilg(' 1'01. ( I i l t i l t ) i l S ( ' 1 V S . t l l ( ' 1111111~)(?1. 01' I ) ~ ~ s ( ' s

ll!X'tl t o fOl.111 111(! <!iff(>l l~l) i l ( ' ( ' . I l i g I l t . : rl ' ll( ' 1)1Ot ~ O L . ( I i l t i l I ) i l ~ ( ' 2. . . . . . . . . .

L('1.t: I ' l 0 t . t i l l g t l l c ' l l l i l ~ . ( : l l i l l g ?lV('l.ilZ(? 1.01' ( I i l ~ i l I ) i l ~ ( ! 1 VS. t l l ( ' 1 1 1 1 1 1 1 ~ ~ ( ' 1 . ~ 01' l ) i 1 5 ( 1 ~

~~scvl to Sor.111 t , l l c \ c ? i g c , ~ ~ s l ) ; l c , c . Soli(l l i ~ i c : : raw C'SS 111c:t l10t l . I l o t t c ~ l l i ~ ~ c , :

sii

S I V

List of Algorithms

Chapter

Introduction

1.1 Objcct rccogr~ition

1 .I .2 Challenges

I I o ~ w v ( : ~ ~ . o l ) , j c ( ~ t , r v ( . o g ~ ~ i t i o ~ ~ is :I ( l i f i c ~ ~ l t I)IY)I)I(:III . v s 1 ) ( \ ( . i a I l y \ \ - I I ( ~ I I t l w I ) i ~ ~ i s of' ( - o t ~ i l x ~ r i s o ~ ~

is t,ll(! ~ ~ ~ ! ~ f 0 1 . 1 1 ~ 1 1 1 ( : ~ ~ O f :I. 1 l I I l l l i I l l Ol,sc.1.~(!1.. I \ ~ i l l l y ilS])(>(.tS of 0111. ~ i ~ l l i 1 1 s ~ S t ( ~ l l 1 1 l l i l l \V(' fill<(' hl'

g r i l l l l ( ! t l ill '( ' ( ' l l i l 1 l ( ~ l l g i l l g t 0 l'(>l)li(:il.t( ' 1110( '~ l i l l l i ( ' i l l l \ '1 f o l ' ( \ ~ i l l l l l ) I ( ' , i l ( l , j l l S l i l l g 1 0 ( l ifk!r(: l l( ' ( :s i l l

f I I V c ~ ) l o r or. i r t t , o ~ l s i t y o f ir s c : c ~ l c ' s i l l t ~ r r l i ~ l i r t ~ i o ~ ~ . Also. i l l i ~ t l ( l i t i o l ~ to t hc: ~ ) o \ v c ~ f ~ ~ l ~ ) r o c : c ! s s i l ~ g

c . ; l p a l ) i l i t y o f t . 1 1 ~ I ) r ; l i l ~ i111tl o p t i c : i l l s y s l c 1 1 1 , I IUIII ; I I IS I I ~ I V C i~c:c:c:ss lo ;I \\.(,it11 11 ol' ( . O I I ~ O X ~ I I ~ I ~

i ~ ~ S o ~ m i a t i o ~ i i111(l 1)rior I ( \ ~ I ~ I I ~ I I ~ ivit 11 wI1ic11 to ; l i t1 t l ~ ( > i ~ , r o ( x ) l ? ; ~ ~ i t i o 1 1 ( l ( > ( , i s i o ~ ~ s . F I I I . ~ , ~ I ( Y . l ) w 1 ) l c

11i1\:(' ~ I I I I ~ I I ( > I ) i o l o g i ( . i d : t f f i ~ t i t i v s for ~ - ( ? c o g ~ ~ i z i ~ ~ g ( :c r t : t i11 l ) a t t ( ! r ~ l s . s11c.11 21s S;I(YY [70].

&!( . i l l~~(! o f t l l c l)r(;mIt.l~ (of t h pro1)1~:111~ I I I O S ~ (:cm11)111vr visio~~ oI).ic(.t ~ ~ c o g ~ ~ i t io11 I I I ( ! ~ , ~ I O ~ S

( ' ~ I I ( ' ( ' ~ . I I ~ I ( ' ~ I I S ( ~ ~ \ ' ( ! S \ v i t , l l l i l i l i t ( Y l ~ ( . ( : l l i l l . i o s rat 11(('1. t I I i I l I i l l t ( ' l l l l ) t i l l g lo g(>l l ( ' l ' i l l i%( ' to ( : o ~ l ~ l ) l ( ! l ( ' l , y

l l l l : o l l s i l l \ s l l S I < S ~ O I I S ( > ( ~ I I O I I ~ ~ V ~ t ( Y : l l l l i ( j l l ( : ~ Il l i lJ ' I)(\ ])l.( '(li('ilt( '(l 011 i 1 ~ ~ l l l l l l ) t i O l l ~

i O l 1 I ! ! O f J ~ S 1 1 i 1 1 1 1 ~ fO1. ( !~ i l l l l l ) I ( ! , t l l ( \ ~ i l l l O l l ( ' t ~ e l l l i l ~ ( ~ ~ l i l 1 g il~$!$)rithlll

tl(:sc:ril)c:tl i l l C : l ~ i ~ l ) t o ~ : 1 rc.cl~~irc,s ~ . c : l a t i v c l ~ . c~1ci111, c. losc,rl l ) i ~ ~ i ~ r y c x ) ~ ~ t o111.s. O r . i l l t I I C c.ix-ic, o f t Iio

~ ~ ~ c > t l ~ o t l for r c x c v g ~ ~ i z i ~ ~ g tish slwc:ic>s t l ( w , ~ - i l ) c x l i l l ( I l ~ i ~ p t , o r 4. t l lc . ~ l r ~ ~ r l l ) c > r o f sl)c,c%:s is l i ~ l l i t . e ( l

to 111i1k(' t I J C 1)1.01)1(!111 11101.~ I ~.il(:t.ill)l(!. '1'110 ~~cn!ssit-j~ o f ~ l l i l k i l l g t I I (w ' t ~ l ) ( ' ~ OS ( . o ~ ~ l l ) r o ~ l ~ i s c s

is ;I I ( > S ~ ~ I ~ ~ I ( ~ I I ~ t , ~ 1110 po\v('r o f 111(' I I I I I I I ~ I I I ~ i s l l i l l s\is1('111: 11owowr i l 110 Il1('ill1S ( l i l 1 1 i l l i s 1 1 ( ~ s

t11c r~sc~f~~l~~c:ss of 'o l ) , j ( . c : l r c . c , o g ~ ~ i l i o ~ ~ lc:c.l~iclr~c:s i l l ( : \ ~ , i ' y t l i . \ ~ . i l l ) l ) l i ( , i l t i o ~ l s . A ( , O I I ( . I . ( : ~ ( : ( x ; ~ I I I ~ ) I ( ~

o f ( W I I I ~ I ~ ~ . ( T visiou w o r k i ~ ~ g i l l ~ ; I I I ( ~ ( ! I I I wit . l~ I I I I I I I ~ I I I o I ) s o r ~ i l t . i o r l is ( . O I I I I ) I ~ ~ , ( ' I . - ~ I ~ ( I ( > ( I ( l i a g ~ l o s i s

( ( : A l l ) : wl~ic: l~ rcf'crs to i l ~ ~ i l g ( ! i ~ l ~ i ~ l , y ~ i ~ S?;S~,(:IIIS t l ~ i l t a i l 1 1 10 I , ( Y ~ I I ( , ( > I l l ( > 11111111)(~1. 01' ~ I I I I I O ~ S

l l l i s s ( Y / I)? r i ~ ( ~ i o ~ o g i s t s v i ( w i l 1 g X - r i l y i i l ~ i l g ( % . 1 3 i 1 t h ( \ r tI l i111 h w v i ~ l g ~ 1 1 ~ ~ ~ 1 iI (m~( . i ;~ l t iWI< f1111y

10 i l l1 i l l l t . 0 l l l i l t C Y I ~ J 7 ~ t ( \ l l l ~ t l l ( : ~ . 0 1 1 1 ~ ) 1 1 ~ ( ~ 1 ' i l l l i 1 1 ~ 5 i 5 (:01111)1(!111('11t~S i l (10( ' tO1. '5 ~ ' ( ~ i l ( l i l l g 1 ) ~ f l i l g g i l l g

~ ) o s s i l ) I ( , l o s i o i ~ s , wl~i l ( : l o i l . \ ~ i ~ ~ g 1 1 1 ~ l i ~ ~ a l s ~ ~ l ) j ( x . t i v o t l ( G i o 1 1 t o 111(! l111111ill1 ( ~ s 1 ) o r t WIIO l l1ilY

o1111)1oy i ~ ~ ~ c l ~ ~ a ~ ~ t i l i a l ) l ( : C S ~ ) ( : ~ ~ ( : I ~ ( ~ ( ! - I ) ~ I S ~ : ( I j l i ( l g ( w ~ ( ' ~ ~ t , t o l i ~ ~ i l l l y ( I c ( . i ( l v i f il 11111101. is ( l i u ~ g c r o ~ ~ s .

A l t , l l O l l g l l 1,11(' ( :Olll l , l l t(!r V i s i O l l ~ l ~ g O r i t , l l l l l ~ ( ' i l l l l lOI ( ' l l l ' l . ( ! l l t l , ~ 111;11('11 111(: ~)Ol ' fOl . l l l i l l l ( ' ( ! 0f i l

l l l l l l l i l l l OI)S(~~.\:(:I"~ t l l(! ( : O l l l ~ ~ i l l i ~ t i o l l Of (10(:101' i l l l ( I i l l l t .Olll i l t(!( l ~.(!(X)gll if iOll ~?;5t( '111 IllilJ' 1X!

l)c\tt.or t l ~ i ~ l l t.11(1 ( I o c t o r ; ~ O I I C [ l X ] . h ; 1 1 1 1 1 ) 1 ( ~ i s11(.11 ;IS 1 1 1 ( ~ ! i ~ ~ o t i v i l ( o ( : o l ~ t i i ~ ~ l ( ~ ( l \vork o i l

ol),jcc:t r c x . o g i ~ i l i o l i 1 ) r o l ) 1 ( ' 1 1 1 ~ sii~(':(: f I 1 ~ ~ 7 l ) r o \ - v t , l l o wort11 o!' s r ~ c l l s y s t cv~~s t l c s l ) i l ( ~ t l l o !'i~(.t

t l l t l l , t l l ( > y 1 l l ; Iv h! for(Y!d t o t l ( l ( l r ( 3 ~ ~ i l l x l r ~ , i i l l l ~ - ~ i l l l i t c ! ( l ] ) ~ o ~ ) ~ ( ~ l l l ( ~ O l l l i l i l l .

( - ~ O I I I ~ ) I I ~ ( Y v i s i o 1 1 c x 1 ) l o i I s i~ \vi(I(: i r i ~ r i ( ' t ~ r of' 111vt 11o(Is i l l 1 )11rs1 i i t o S o I ) , j u : t r ( u ) g ~ ~ i t io11 [Mi].

s ( > l ~ l ( > o f w l l i d l iIIX? l)(>t.t.~!r s l l i t ( ! d t o ( : ( ! r ta i l l i ~ l ) ] ) l i ( : i ~ t ~ i o ~ ~ ~ 1 Ilii11 o I I I ( ? ~ s . /!II ~ I I I I > ( ) ~ ~ ~ I I I ~ f ' i l ( ' t0r

i l l t l l ~ SII(XCSS o f i l l ) i l r t i ( * ~ l l i l r 111(\tl10(1 is ils s 1 1 i t i l 1 ) i l i t y t,o t 1 1 ~ i111;1g(1s to w I i i ( ~ 1 1 it. is i l l ) l ) l i ( ~ ( l .

OIIV way o f w w r t a i 1 1 i 1 1 g a p r t i ( : l ~ l i l r vision I I I ( ~ ~ I I O ( I ' S i ~ l ) l > l i ( : i ~ l ) i l i t \ i to i l l)rob1(~111 set tiug is t o

c ! s i l ~ n i l l c ) 111(! i l v i ~ i h b i l i t ~ l o f t l l (? i l l l i lg ( ! ( . l l i ~ r i l ( . t ( : r i ~ I , i ( : ~ 011 \v l l ic . l l t l ~ c , 111(:t l lo(l r( \ l i(!s . E ' o i l l I I r ( : ~

s11(,11 as s l i ; l l ) ( ~ . (:olor. t , ( x t , ~ i i ~ ! i ~ i i ( l N V l i t iIizo(1 f o ( l i f f c r ( ? i ~ t ( l ( ~ g r ( : c s by ( I i f f ( ~ r v 1 1 t r w o g ~ i i t , i o ~ ~

t w l ~ ~ ~ i q ~ w s . so t.110 i ~ v ; ~ i l i l l ) i l i t y o f i l l ) i ~ r t i c ~ i l i t r ~ ' ( ' ~ I ~ I I H , i11 it11 i 1 1 1 a g o o r vi(Ivo st r o i 1 g I . v i ~ ~ f j ~ i ( ! ~ ~ ( : ( l s

1 ]I(! l n o s t ( ? f I ' ~ ( , l ivc r c ~ ~ o g l l i t ~ i o n 111('I 1'01. 111111 ])ill.( i ( . l l l i ~ l . ~( . ( ! l l i l l ' i ( ) .

' I ' l~ is t l i c s i s t l w c : r i l ) o s two ~ ~ ~ c l . l l o ( l s I hilt i lrc ' t w i l o r o t l to 1 1 ~ l ) r o l ) l ( ~ ~ ~ ~ s to w l l i d l t . l l c y arc a ] ) -

])li(:(l. c ; l l i l ] ) t ( , r :) l ) l ' ( W > l l t ~ i l t , ( ~ d l l l i ( ~ l l ( ! for 1 T t r i W i 1 l g 1 1 1 i l t C ~ l i l l g S I l i l l ) P ~ f r o l l l i l I;lrg'(> ( 1 i l t i l I ) i l X '

of c:Iosc~l 1 ) i n ; l r y s i l l i o ~ ~ c t tc c : o n l . o 1 1 r s . 111 ( : O I I ~ r i ~ s l . C I t i l ] ) t ( ' r 4 ( k ! t , i ~ i l s i l 111(!t 110(1 for t ( : x t ~ ~ r t ! -

1);lsc;tl ~ . c u ) g ~ ~ i t i o ~ ~ o f ~ l ( : i l r l ~ r i t l ~ ~ i i t i c . a l l y ~ l l i l l ) ( ' t l fisl~. F i g 1 1 i . c 1.1 ( l i s ~ ) l i l y s ( \ s ~ I ~ ~ I I ) ~ ( ' s of two

s c ~ i l - c ~ c ~ i l t ~ ~ r c c ~ ) i ~ t o ~ i i - s w l i o s c : s l ~ i l l ) c s iiiillcc! tll('111 ( l i s t i i ~ ( ' t i i . ( ' . (1w1)it o t l l ~ i i i l : l g ( s ' ~ . ( \ l i ~ t . i \ ' ( : 4111-

1)licitv. ( I o ~ ~ t r i ~ ~ t , this wit.11 F ' i g ~ i r ( > 1.2. w l ~ i c h sl~ows k 1 1 w i t h wSry s i ~ u i I i \ r s i l l 1 o 1 1 ( ~ t t c s b 1 1 i

( I i l f ' ( m ~ 1 1 t t , ( x t ~ i r ( ~ . '1'11(?sc ( l i f F ( ~ r ~ i i ( w i ~ l o t i v i ~ t ( > t l i ~ two a 1 ) 1 ) r o a d i ( ~ s ( l v l i i ~ c i ~ t ~ ( ~ ( l i l l C l i ; ~ l ) t ( ~ s : 3

i u ~ t l 4 . W l l i l c tllo tost .urc c a ~ ~ l ) a r i s o l ~ o f ( : l~ ; lp tc : r 4 \ v o l ~ l d I)(: i l l ~ l ) o s s i l ) l c to ; ~ ~ ) p l p t o t l l e

( X ) I I ~ ( ) I I I X o f 1 7 i g l l r c : 1.2: t,ll(: s ~ I ; \ I ) ( ! 1 1 1 i l t ( ' l l i l l g of' (]IIi \]) t( :I ' :j \\'0111(1 O ( l l l i \ I I ~ i l l - s i i i t ~ d td)

( l i s ( * r i ~ i ~ i ~ ~ i ~ t i ~ ~ g I ) C ~ ~ W Y ? I I t l r sin~ilarlj: ~ l ~ i ~ ~ ) ( ~ ( l fisl~ of' I:iglur(! I ,:{.

' I ' ~ I ( ! 1 . ~ 0 1 1 1 ~ t , l l o ( k ill.(? , j l l ~ t i l p ) ~ ( ' ( 1 I l l ( > g o i l l 01' i l l l l l l l i l l i l t i l l g t l l ( ' i ~ ( : O l l l l ) I ( ! l l l ( ' l l t i l l ~

l l i l t ~ l l l ' ( ' , I t i l t 1lc>r 1 I l i l l l 1 . 1 ' ~ t 0 r i l l l k 1 l l (> i l ' ( ' f r ( ? ( : t i \ Y ' l K W l '(!I i l t iW' t 0 Oil(: i l l l O I !l('r. 1 l l( ' i l l ( (!Ill iO11

Curvdlurc Sculc Spacc Image locirtlons 0 1 zcro curvature po~nts

( : l i i t l ) t ( . ~ 4 tl(w.riI)c>s tliv 1 .cu)gn i t ion of fisll spccics i l l t'ra~ilc's fro111 ~ i ~ i ~ I ( ! r \ v a t ( ~ r vi( lvo. Fig-

I I I T I.:{ s l ~ o \ v s 2111 ( w \ ~ i i l ) l ( ~ o f t ~ v o li.qli to I)(! (:Iassifi(xl. 111 (x)~i i l ) i \ r isoi i \vill i 1 l i(\ ( ~ ) ~ i t o ~ i r s i i i

F ig t t rv 1.2. t ~ : ~ t l ~ r i t l >~1>l)(~i l l . i t l l (~(~ i s tli(! O I I I ~ c I is( . r i~t i i~~i t t , i \~(! ;IS~X!(;I. o f tli(, lisli t 1 1 ~ t\vo sl)(!(:ivs

Chapter 2

Previous Work

2.1 Shape matching methods

Curvature scale space

'l'lt(' ~ ' l l ~ V i l t l l ~ ( ! ~(;ilI(' S])iI(X1 (c;ss) illlil.gC [62! 5!): 6 1 . 121. ?I. ~Il i l l)(? l 'C>~)~(~~(?ll t i l t iOll I)ilX(I (111 >I

1)lot. o f t l l o zcrtrcmwsillgs o f t l l o c ~ ~ r v a t u r c f1111ctioi1 of a ( ~ I o s ~ ~ I - c o ~ ~ t o ~ ~ r ( : I I ~ V O i111(1or ( ! \ ~ ~ l u t ~ i w ~

wit.11 a (:a11asii111 of' 1)rogrcssivcly i~~c:rcwsi~~,q s l a ~ ~ t l u r t l cl( .vi i~tiol~ 0. !~111011g t110 i \ t , t , r i ~ ~ : l i ~ ~

i~sl)(!c:ts o f t l i i s s11i1l)o ro l ) l .c !so~~ta t io~~ arc; it.s i1l)ilit,~7 to c i ~ p t ~ ~ r c ~ 111111l,i-sc.id(! s l i i ~ p c~l~nrilc:tc!ristics

: I I I ( I its (\ffc(:t,iv(> ~ I I V ~ I S ~ ; I I I ( : ( > t o a l f i ~ ~ ( > t , r ; ~ ~ l s f o ~ . ~ ~ ) i l t i o ~ ~ s o f t11(> o r ig i~~a l ( . O I I ~ O I I I . . ' I ' l l (~ C S S

~ ( ~ I ) I . ( ' s ( ' I I ~ ~ ~ I ~ . ~ ~ I I II;IS I)(Y:II atlol)t(vl its a slli\l)(! cl(:scril)tor. i l l t , l ~ c ' h,fI-'11C:-7 stantlar(1 [M); 1'21.

( ' I I ; I I ) ~ ( , ~ : 3 l ) r ( ~ > ~ l t s O I I ~ i1111)rov(~1iio11t t.0 t f ~ ( : r ( , l ) r ( l ~ ( : ~ l t i ~ t i ( j ~ ~ i>II(I 11li\t(~11illg o f (-!ss ill1:1g(~s.

Curve segments

Eb~~ricr and wavelet descriptors

1 io~is o f 1110111<~1its. ii:(!r(3 ;1111()11g ( ~ l r l y (:01111)111c:r v i s i o i ~ ~ ~ ( ~ ~ I I I ~ ~ I ~ c s ~ l s ( ! ( I t o ( : l l ; l l ' i l ( ~ i ( ~ r i ~ ( : i 1 1 1 i l g ( ! ~

for l l l i ~ l ( . l l i ~ l f i [.l'L. 721. H I I [42] t l ( x : i . i l ) c x l a sc.1 o f i /~ ,o~r~c .n l i r ~ . w u i m t ~ t , o i l i ~ l ) r o ~ . o I I ~ O I I t l ~ :

Shock graphs

111 t I l k (.;IS(>; . ' l i ~ i w r * . r o k r s 1.0 t l l ~ i l l> i l i ty t o r ( ~ ~ ) r ( ! s ( ~ ~ l t t l i c I r t \ ~ i s f o r ~ i i i \ t i o ~ i wit 1 1 it 4 x ,4 ~ ~ l i ~ t r i s .

I)('li)l.lllill)l(' t('llll)Iilt(' lllil( ( . l l i l lg Illil~ h'O11 1 II(' ~ll l) , j l '( . :( Of i1 ( :O l lS i ( l ( ' l ' i l l )~~~ I)O(IJ' O f ('01111)11t(:r

visio~~ r c w ~ l r c h . 1~'isc:lllcr a l i t l k : l s ch l ; tgc~ [:{'.'I ~)rcw:~l tc! t l it f r a ~ ~ i c w w r k for t l l c k I I W o f this

Pictorial strnctlrrcs

2.2 Distance transforms

2 . 2 . 2 Distance trarlsfom~s gc~lcralizd t,o arbi t 1x.y functions

2.2.4 Fast gc?ric:ralizcd distance trarisfoi-111 algorithm

goto t i else

I, - / , . + I V [ k ] - I/

: [ I , ] I.- .s

t [ X + I ] -- +;x:

e r ~ c l i f

If (1 < 1) ~ I I ( , I I Ill(! 1)iIr;II)OIiI ( : o l l l i l i g fro111 (1 is l ) d ~ ) \ \ l t,li(! 011~: c o l ~ i h g fro111 p t o Ill(! l(?ft, o f

1 . l ~ : i n I ( w ( ~ ( ~ t i o 1 1 ])oil11 s. i111(l ;II)ov(> it. t .o t . 1 1 ~ r i g l i t . of' S.

' l ' l i t ? I o \ \ v r ( ~ I ~ \ T I O J ) ( ~ is ( ~ ) ~ i i l ) ~ i t c ( l by s c ( ~ ~ ~ ( ~ ~ i t . i a l l y f i r i ( l i1 ig t,Ii(l lo\v(:r c ? ~ i ~ ~ ( ~ l o ~ ) c o f l , l l c f i r s t ,

(1 j ) i I r i l I ) O I i l ~ . \ v i l l i 111(' 1 ) i l r i l l ) O I i l ~ o r ( I ( ! r ~ c l I )y t l l ( : i ~ I lo l . i%ol i t i l l grid I o ( : i ~ t , i o l ~ ~ . ' I ' l l (> ;11gor i t , l i l u

( ~ i l l ( ~ ~ ~ l ~ l t ( ~ ~ t l ~ l ( ~ o ~ i i l ) i l i ; l t o r i ; l l s t , r ~ i c t I I ~ C o f t lit: low^ ( ~ ~ i v c : l o l ) ( ~ . ( ~ l i ~ p l o y i n g t , \vo arrays 1 0 I w q )

t.lYl(:k Of 1 1 1 ~ i l ~ l . ~ ~ l l ~ ( ' l ~ l ( ' l l I ~ . '1'11(> ~ 1 0 ~ i ~ O l l t ~ l ~ gricl 10C:ll i011 o f t h ~ ! it'' 1)ilrilt)OIil . i l l t 11~: IO\W?l.

( w \ ~ ( ! l o l ) ( ~ is st OIWI in . I : F ] . ' 1 I i c l'il1ig(! i11 w l i i c l ~ t , l i c i'," 1 ) a r a I m l a . is I)do\v t11c ot IIWS is g i w x i

1 ) ~ ~ [ i ] i l l l ( I /7[i f f 1 . 'I 'll(: 1111111~)(~1' 01' 1 ) i l ~ i l l ) O l i l ~ i l l t.ll(' ~O\\Y:I' (!IIvcIOI)(' is ~1,01.('(1 i l l k .

N'II(~H ( : o ~ i s i ( l ( , r i ~ l g I l l ( > ~ ) i ~ ~ > \ I ) o l ; ~ a t (1. its i ~ i t ( : r w ( ~ l i o ~ ~ \vit11 t l i c ~ ) ~ i ~ a l ) o l ; l f ro111 ,o [k] is

fOllll(! ( / : [ k ] t l l C l ' ip l l l 1 l l O ~ t 1)ill'ilI)OIil i l l t l l ( ' IO\\Tl' (lll\l('lO1)(' ( 'O l l l~ ) l l t , (Y l SO f i l l ' ) . 'I'11(>1.(' ; lR! ~ \ v O

2.3 Dynamic programrrling on a tree structure

'I I I V r c ~ c ~ ~ l l - s i v c ~ I I ~ I ~ ~ I ~ P o f t 1 1 ~ f1111('t ioll:, L:, ( I , ) i l l l0 i i 'h il s i ~ l l p l ~ i1lgOrit11111 t o I ) ( > I IS<Y~ . If (1

1 s 1 ] I ( > I I I ~ I X ~ I I I I I I I I ( l c ' p t l ~ i l l 1 I I C t 1 (Y'. !'or ('<1(.11 1 1 0 ( ( 1 1 1 / wit 11 (1iq)t 11 ( I ( ~ I I I I ) I I ~ (' 1lJ ( I , ) w h ( ' 1 c o2

is 1 I N S l ) < ~ r ( > 1 1 1 o f i q J . S i l l ( (, t I I (WI LIT < I I I 1(1;1f I I O ( I V \ . t l ~ ( w i t flJ ( I , ) ('ill1 I ) ( > ( O I I I I ) I I ~ ( Y ~ 11si11g

2.4 Support vector i-naclrines (SVMs)

Chapter 3

Shape Retrieval with Eigen-CSS

Search

3.1 Introduction

I:ip,llr(> 3.1: San11)lc of' I ) ~ I I I I ( ~ ; I ~ (.II~.v('s.

3.2 Synopsis of CSS Matching by Contour Maxima

L ( I . 0 ) = I,()(.l.(t). ! / ( I ) ) R ! / ( I . ff) = ( X ( t . 0 ) . Y ( t . f f ) ) . (:M)

I ;o~ ( ! v c ~ ~ , v X ( ' ~ O - ( : I I I - V ~ ~ I , I I ~ C > p o i l ~ t : i .0.. t < ( t . 0 ) = 0 il111I ilti.(t? 0 ) / 8 t # 0. t 1 1 ~ ( - o r r ( \ s l ) o ~ ~ ( I i ~ ~ g

I o c * a t i o ~ ~ ( I . u) i l l 1 I I C I ) ~ I I ~ \ Y > J C:SS i111ag(1 is s ~ l l o 1 . 'LI IC\ ~ ~ i w k i i ~ g s of' 1 I I V z c : ~ , o - ( , ~ ~ r \ r i l t , ~ ~ r ~ !

l ) O i l l t , S fOl'l11 i l s(![ Of' ( ~ O l l t O l l ~ ~ . \ \ ' l l O ~ ( ' i ~ ~ ) ] ) ( ' ~ l ~ ~ I l 1 ( ~ ( ' ( ' i l ~ ) t l l r P ~ t l l C ~ 1 1 i l ~ ) C O f [ I l l ! ('IOSIY] (:llrV(?

F i g r l r c : :$.2 sl~ows il l1 o > c i ~ ~ ~ l l ) l c o f t l ~ c , s ~ ~ ~ o o t l ~ i l ~ g p r o ( : ( ! s s of' i l c:losc:tl I ) o I I I I ~ ~ ; ~ I ~ c : ~ ~ r v c ?

: ~ I I ( I its ( . ~ I , I , ( : S ~ ) ~ I ~ ( I ~ I I ~ CSS i ~ ~ ~ : i p , c , .

Curvature Scale Space image: locations of zero curvalure points

path length variable t ( 1 ) )

3.2.2 Matching by CSS Contour Maxima

Loop for all models Cost for model is the minimum of (

matchCSS(image, model), matchCSS(model, image),

matchCSS(mirror(image), model), matchCSS(model, mirror(image)))

End loop Rank models based on their costs

Fi~nction matchCSS(css1, css2)

Loop for a l l contour maximum pairs of cssl and css2 Align cssl and css2 by shifting cssl horizontally

Determine cost of the match End loop Return(minimum cost among all pairs)

End function

111 OlXl(!l' 1 0 f i l l t l t.11~: lllillillllllll C O S ~ Of tall(? lllilt.(:Il I)~t\\'(Y?ll WI1 illlilgC illl(I H I ~ I c ) ( I ( ! ~ ~ t110

il~gOl.i~llll1 1111151 ('Oll~i(l(!l. ilII pO~~ib l ( ! WilJJS Of illigllillg t.110 Il,%!]Il.-~(:d(? (:011tOlll. lllil,~illlil fl.O1ll

I ) o ~ ~ I (.:SS illli~gc~s: ;\11(1 ( Y ) ~ I I ] ) I I ~ ( ! ( ] I ( . ;lssoc:iiitc!tl (:os~,. Fol8 (;vc:ry possi\)l(! (:i\~l(li~li~l.(! 1)ilir of

r o ~ i l , o ~ ~ r ntaxi~tta. 1 I ~ C T C ar(- t LVO \Vitys t . 0 align t , h ~ t : cit,Itcr sl~ift i l~g t h i ~ ~ t i t g ~ (2% circ1~larl-y

in 1 1 1 ~ Itorizo111 ill clircc:t,iol~ or slliftil~g t . 1 1 ~ motld CSS. 13c(:;l.r1s(: of t , l ~ c > i ~ s y l l ~ ~ l l o t ~ i ( . t r c a t ~ ~ ~ ~ n t ,

o f t11c i111;lgc ( ' S S allcl ~ I I C ~ ~ o t l o l ClSS l y t,llc: a lgor i t ,h l~~, 1,0111 i ~ l i j i l ~ l ~ ~ c ~ ~ t 1 i i o t ~ l 1 0 ( 1 ~ l111lst I)(:

;ll f , (> l l l l ) t (V 1. iillt 1 1 I l ~ i r 2lssO~:i;ltLYl c:OSt.S l l l l l ~ t I ) ( > ~~t i l l l i l t ( : ( l ~(ll)ilI'iliL'Iy.

1J11Sort 1111atc1y~ t l t c a l ~ o v c p ro (w1 i11~ fails to ( I ck~f , t11c ~nirror- imag(~ of i.Ii(: i n p ~ i t il~iagc!

c:vc1l~ if suc:l~ is ill t,Ilc t1atal);wc~. 'I'l~c!roforc~: ttllc\ ;1lgorit;l1111 has lo 1 ~ p ( x 1 o l ~ c : o ag:ti~~ by

c . o l~ l l ) a r i~~g ~ , I I o nlil.ror(:(l C'SS illlilg(\ wit11 ill1 t11~ 1110(1(:1~. Fillilll~': ill1 (:os~,s of tilt! 111ilt,(.h ~ I I I ~ S ~ ,

I ) ( ) cwl~s i t l (w . t l t o c.i~I(.~tl;~tcb tllr c:lowr~c!s:: i l l a1)I)ctarnlic:c f?)r all ~~~otlcls .

'1 '11~ c:olic:ct~)tuaI lligll-lcvcl s t r l ~ c t ~ ~ r c o f tl lc! algorit . l l l~~ is sllowli as pscx~tlococlc! ill Algtr

r i t l ~ lu 3.1.

3.2.3 Class Matching Evaluation Method

3.3 Matching by Eigen-CSS

3.3.1 Eigenspaco: PCA via SVD

I) i~~~c:~~siol l i l l i ty r(ductio11 is t,11(! (Icrivat,ion of i t sct of Io~v(:r-~li11tc1~sio11~1.1 v(:(:tors fro111 liigli-

t l i~l~cwsioni~l chta., wllcrc~ tltc: lowc:r-tlimcnsio~~tll c:oortli~l;~tc.s still c a p t ~ l r c thc: rc la t , io~~s l~ ips

i ~ ~ l ~ ( : r ( x ~ t . in t . 1 1 ~ origilli~l ( I i ~ t i ~ s ~ t , . '1'11~ d~f;\.111t~ I I I C ~ , ~ I O ( ~ of ~ ~ i ~ ~ ~ ~ : l l s i o l l ~ ~ l i t ~ ~ rc(111(:t,io11 i l l th :

Eigcti-(SS ~ t ~ c l l i o ~ l . ~) r i i i (~ i~) i i l ( : O I I ~ ~ O I I ( : I I ~ aitalvsis (PCA) [47: 921. at, tcmpts t.o rq)r(:sc~it a

l:llgX! 1111111~)('l' Of ~ l i g ~ 1 - ~ ~ i l ~ l 1 ( : 1 1 ~ i o l l i l ~ fi:illlll'(! V C C ~ O I ' S i l l il ( I i l t , i l I ) i l~O i1S lill(!ill ' (:Olll~~illiitioll~ Of il

1111lc*li ~111;111(~1. 11111ii[)('i. of b i l ~ i ~ V(!(:(.ors (I'CJA is ~ o l l l ~ t ~ i l l l c ~ I . c~c \ I . I . (Y~ t o ilS tli(' I<iI~hll l I~l l -LO~~\~( '

l1,:11isfor111). 111 tc,ril~s of stori~yy s])a(:e?. this r(~(111ctio11 is a (l(?si~.i~l)lc goal. For cxa~~ i ] ) l c ! i f

1 Ii(:r(? i1rc 1 ;{ 1 sl1al)c.s i l l t l?1t,i1l)i1s~t alltl t,hc fc>at 11ro vc:c:t,ors arc! 2~14-tliinc~nsi011i11! wit,ho~lt

t l i~l lo~~sio~lal i t .y roclllc.t.iol~ it \vorlltl I)c ~locoss;~ry t,o st.orc: 1 : 3 1 tlitYc~rcnt 24;l-D vc!c:tct~x, om for

o;1(:11 s11i11)c in t lio tli~t.i\l)nsc. Ilowc~c:r, i f tlw set, of t l i ~ t , i l l > ; ~ ~ ~ f(:i~t.urc \;c:c:t,ors is rcl)rcsc~nt.cd

w\ I I I)JJ, for ~ : X I I ~ I I ) ~ ( ~ . ( . o ~ ~ ~ l ) i ~ ~ i ~ t i o ~ ~ s of 15 basis f ( : i ~ . t ~ ~ r ( ~ VC(:~.OYS? t.k1(:11 we! II(K:(~ 0 1 1 1 ~ 7 S ~ O ~ C : t,I~c:s(:

I5 I)asis vcvtol.s. i ~ l o ~ l g wil , l~ ;I, 15-vc~:tor of' wcigllts for oitc.11 sha~)c: i l l 111~ tli~.t,i\l.)i~s(:. \I7itl-

0111, (li~i~(~tisioii;llity i x v l ~ i ( ~ t io11, i t wo111(1 I)(: 1i(x:(~ssi11y to st,orc l:{l (Iiff'c~~cn(~ 2~l4-(Ii111(~11sioil;11

\XY~OI,S: wit11 ( l i~ i i (~ i~s io~i ;~ l i t~ ,v r ( ~ ( I ~ t ( ~ t i o ~ i 01i1y 15 scp1ri1tc 2 4 4 D v(!(,tors ail(1 131 15-11 v(:(~toi.s

i11.t: Iloc'c~ssary. If (.;1('11 ~ ( Y ' t o l (:lit 1.y is ;I :%2-l)ii f IOi l t ! tll(: 1111l'(:(111(~('(1 (I i l t i l I ) i lS(? \ ~ 0 1 1 1 ( 1 l'(!(lllir('

USV = X: (3A)

3.3.2 Marginal-Suru Feature Vectors

c = [C ~ ( 1 . j). C C ( 2 . i ) . . C ~ ( r . j ) ] '

3.3.3 P11asc Correl a t' .lo11

3.3.4 Mirror rcflectio~s

180' rotation vertical mirroring

k'o~, " I ~ o ~ ~ i z o ~ ~ t : ~ l n~irrorii~g"; \ v I w r c i ~ ~ t,l~(! ( X > I I ~ O I I I . is i11\:(\1,t(x1 fro111 top t o 1)ol 10111: t I I V CSS

i l i ~ + y ) is ~ ~ ~ ~ ( ~ l i i ~ i i ~ ( ~ ( l (>x(xx1)l 1.11i1I i t is IX:\Y:IXY~ Id'-rigl~t ;III(I sl~ift(:(l. '1'1111s :1gai11 \vc l - l i ~ \ ? ~ :

t l~a t t l w Fouric-I. t r a ~ ~ s f o r ~ ~ ~ 01' t l l v ~OW-SI I I I IS (1ow11 t h C O ~ I I I I I I I S t l i l l i ' r o111y I)\: i i 1 ) l ~ s c t a~i t l r

Obtain phase-correlated marginal-sum vector for each CSS

Form input matr ix for S V D

Perform S V D

Map each CSS in to 72' eigenspace

Stage 1: Ckrieral Eigenspace

Stagc 2: Specialized Eigerlspaccs

Obta in phase-correlated marginal-sum vector for each CSS

Form inpu t ma t r i x for S V D for each object category

Perform mul t ip le SVDs

M a p test CSS i n t o each R" eigenspace

T h e eigenspace tha t gives the closest reconstruction o f t h e test CSS

feature vector is t he best category

3.4 Experirncnts and Results

34 .1 Test Data Scts

1521. 'I'l~is cx)~lsisl-s of (: = 70 c:lassc>s (bone, chicken, cellular-ph, ...) o f 20 ol),j(.c.ts c1;lc:h

3.4.2 Ilnplelnelltation Details

CSS images

General Eigenspace

Specialized Eigenspaces

Global parameters

input: 0

I:ig~lrv :<.7: hlill (.liing r(~111( s for i l fis11 in (liltill)il~(! 1 . 11si11g 011r il lgoritl~l~i. '1'11(' tol)-l(:fl ( : I ~ ~ I V

sl~o\\;s 1 . 1 ~ : i1l1)11t fislt. ' 1 ' 1 1 ~ rcst. of t l ~ c fish sllow tl~c: Iwst I 5 nliltc:l~ rtwllt.s, ral~kcltl 1)y tllcir E~~c:lit l(wi tlistillic.c: t o 1 I IV i11pul fish i l l tlic c!igcmsl);lc:o sul)sl)ncc:. '1'111: fish's class ill)l)c!;~rs as

il 1ilI Wl OV(!l' c'?l('ll lkll.

CHAPTER 3. SHAPE RETRIEVAL VWlH EIGEN-CSS SEARCH

( i l l

Figure 3.8: (a): An image in datdxise 3 ('clloI)pcl.-Ol.gif'); (I ,) : 'l'hc st,aricl;~rtl-Icrigth corit,orir-. (c): Tlic corrcsporichg CSS image. (d); Thc c:orrcsponding fcaturc vector: thc pllasc- c:orrclat,cd 1na.rgina.1-srirri coniponcx~t is shown i l l l~lnc, mid t,lw row-~11111 is shown in grwn, diwhed.

(c,f): Vertical and Horizontal mirroring. Both genc:rate t,hc sairic foature vcctor (d).

- 0 0.2 0.4 0.6 0.8 1

1 - specificity

~ i g r l r c :i.l[): I,(!l't: I ' l O t 1 i l l g t11i ' ~ l l i l t ( : l l i l l g t i lV( ' r i lg0 fOl' i I i l f i l l ) i l ~ ( ! 1 1.5. I l l ( ! 11111111)i\1' Of' l ) i l ~ i ' ~

~ i s o ( l t o 1 b 1 ~ i i t 1 ~ : (\ip,(:iis~):il(:c. 11 igl~t: ' I I I ( \ ] ) l o t for (1il.t i l l ) t ~ s c ~ 2.

0 L 0 20 40 60 80 100 120

number of bases used

input: hammer

hammer

hammer

hammer

& hammer

ti==== hammer +

key

hammer

hammer

hammer

fountain

hammer

hammer

Bone

input: 11 11 11 11

C:(~llt~l~al &,cw-( 'SS 15

15 1 5

Sl)c~c~inlizctl Eigc.l~-( 'SS 1

2 5

3.4.5 Mot,ivat,ion for inarg i l ia l - sr~~r i feat rlrcls

3.5 Conclusion

input: 3

input. 3 -

Chapter 4

Recognizing Fish in Underwater

Video

4 . 1 Prcvious Work

4.2 Approach

Generate treestructured template models

Iteratively estimate transformations from query image to the template

Filter and sum the warped query images into feature vectors

Classify via SVMs

Shape contc!xt,s

0 1 1 1 . Il l( '( l l O ( l ('1111)10?.5 ~ l l i l l ) ( ' ('011I ( ' X ~ S ;IS i l l l i l g ( ' f('i11 111'('5 I)('('illlS(' 1111'). ill'(' \\Y'II-511it('(l f 0 ( ' i l l ) -

t l l l . i l l g I i l l l ' ~ ( ' - ~ ( ' i l l ( ' ill i i l l i l l l i ) l ' l l l i l t i o l l i l l i l l l i l g ( ' ~ ( > ~ l l i t ) i ~ ill:; S[)itIW' ( ' ( Ig( '5 . i1 ('01111110I1 (' l l i l l ' i l ( '-

Distance transforms

Dynamic prograrnrning for efficient minimization

Iterative warping

Si11w tliv fisli arc t11c sa111c: ~ I I > I , [ ) P 1)11t Ililvv (Iiff'crvl~t 111;1rIii~1gs. LW ( Y I I ~ ) I ( . ) v a t , ( !xt~~r(~-l~i~si : ( l

(:Ii~ssifi(:~lt,ioi~ t ~ l ~ t , l l o ~ l . OH(:(: I I I C ( I U ( Y J ~ i ~ ~ ~ i ~ g c s I I ~ W l ) ( ~ > l i l ~ r t ~ ~ s f o r ~ ~ ~ ~ ~ l hit o (>st i l i i i~ t , (>( l i~ligli-

I I I C I I ~ . with the t(~nplat.(! t h y arc proc(?ss(xl t.o (xtract , ts(:xt~m: p rop r t i c s . First. (WCII i~l lagc

is c:o~~volvc?tl wit11 il ;I-pixcl-tall vc:rtic:;~l c:cntrnl tlifforcl~c:c? 1wrllc.l. '1'11(: ~llol.iv;>t.ioll k)r vc!rt,i-

(I(;riv;~tivc filtrring is t , l~a t aft,(:r s ~ ~ : ( ~ s s f ~ l l warj)ilig! t,lw vvrtiixl (lir(l(:t,io~~ ( : i~ l ) t ,~ l r~ :~ tlic

~ ~ ~ o s t i ~ l ~ i ~ g ( : iriforliiiltion. Next,. tlic filtoi. rc:sl)ollso is Iialf-wnvc rc,c,tilic\tl i o i ~ ~ o i t l c:al~c:c:llatio~i

tluril~fi sul)socl~lont s l ) ;~ t i ;~ l i~ggrc!g;lt,ion. N;I(:II I ~ a I f - ~ v ~ ~ v c ( : O ~ I I I ) O I I ( \ I I ~ of 1 I I C liltcr IT~I )OIIS( !

is s11111111(:(1 iut,o 7-1)ixcl ~ ( l l l i ~ r ~ sc(:t,iol~s. l ~ i ~ i a l l j ~ : all of 1,11(: (:o~ill)il~c(l f i l l (:r r (~s l )o l~s(~s arc

c ~ ~ I I ( : ~ I ~ . ( ~ ~ I ~ I ~ . c ! ( ~ into a fcat,~lrc v ~ c t o r as input, for a ~~~~~~~~~~t, v ~ c t o r ~ n a c l ~ i ~ ~ o (SVAI) (:lassifit;r.

SVRIs arc. b i l ~ i ~ r y classificw. Ilowcvcr: i l l o ~ l r 11iot11od t.11cxc arc: t.wo I . ( '~ l l l ) l i l t~~ , ollc for

w c l ~ t,y1)(! of fisl~, m c l (.:+(:I1 q11~1y il.~~ilgc' is LV;II ' I ) (Y~ I.0 1)0th t ( ' l ~ ~ l ) l i ~ t ( ' ~ . ' l ' l~ i s lli(:illlS thilt ~ i e

11:lw~ tivo S V N s wlios(~ o11tp11ts ~i(:(:(l l o I)(! ( : o ~ ~ ~ l ) i l ~ ( ~ ( l tJo get, :I fi11i11 <~lassifici~tiol~ ( lcc is io~~.

011r s i t ~ l t ~ . t i o l ~ is n silr~l)lific.tl t.c\rsiol~ of t11c I I I I I ~ ~ i-SVM 1)ro1)10111 of' [;{I . If 1 x ) t l i SVMs ikgrC(:

011 ;I c.l;lssifici~tiol~ tlccision: tJloil 1111 is woll. If tJlc two SVhIs ;lssctrt o1q)ositc: c:li~~sifii:ilt.iol%

I llcw tllc: t l(~cisio~l of tile SVM with t 11o grcatcr i11)solut.c tlistil11c:c t o its sc~l)nrill,illg hypc:rl)ln~lo

is ti1kv11 to I)(> t j I~c t , r~w one.

- - . - . - - - - - . - --

query warped filter responses , to ternplate I

- + /find

correspondences and warp

find correspondences L a n d warp

Classification decision

template 2 query warped to template 2 filter responses I

4.3 Results

linear S4'% !to% polynomial 81 % Mi'%.

4.4 Conclusion

Chapter 5

Conclusion

5.2 Future work

5.3 Conclusion

Bibliography

[ I (j] 13. By1111 i111(1 S.-W. I.,(!(:. Appliwtio~is of' slipport. \:(!(:tor ~ l ~ i ~ ( . l ~ i i l ~ ~ for 1 ~ 1 1 t.cr11 r(xwgi1i- tioil: i t s11rwy. 111 S.-1%'. T A T ali(1 A. Vcrri; (~ I i to~-s ; I'(ldLwrt, Ii(,(.o!~r~.iLior~, wi./,lt. Support I / ( ~ t o r Mac:l~~ir~.c..;: 17i~s t I r t ~ t c : ~ ~ r ~ l , i o l I/Vor~k,s/t,op (SViI./ '02). \:0111111(\ 2:LSHj2002. p:~gcs 21:3 2;{7. Sl)riiigcxi,. Alrg~ist 10 2002.

[IS] I Icmg-Pi~~g Cllan. K l ~ i ~ i o Doi. Sii11ri111,jit Cialllotra. Carl .J. Vyl)orliy. H ~ l ) ( ~ r ~ I i t c ~ h l t ~ l ~ o ~ i . ;111tl I'ctc~ b1. .Joki(li. I~nngc. f(wtlirc. analysih i t l l t l (.~~~ll)~lt(:r-i~i(l(>(l (liilgllo& il l digitid I il(liOgr~1)Il~. I. i I l~t ,or~~i~t( ' ( l (I~t(\( ' t io~i of ~l~i('ro('i~l(.ifi(.i~f ioils i l l i l l i~i t~l i~Ogri~l) l~~. I I ~ C ~ Z ( Y I L Ph lj.szc4s, l4(4) :5:M 548. 1057.

12.11 h h u k S. I ~ c w . 71 ' i~~i K. 1 , ~ . N K I A I I ( I ~ ( w I i ow . S l~ i~ l ) ( ' r(~h-i(w11 wit 11 (+~\II-C'SS s(wr(,li, ' i 1 i i 1 1 c o t 1 ' 2 0 - 7 S o a s J i i 2005. http: //f as. sf u. ca/ pub/cs/TR/2005/CMPT2005-07.pdf.

[2S] l'otlro F. l:c~lzoilsz\\7ull) i111el Dui\ic,l I:'. II~lttc~~loc~llc:r. Dista11c:c: ~ ~ ; I I I S ~ ' ~ I . I I I S of' ~ i t ~ ~ ~ l ) l ( t ( l f '~i~ic:tio~is. 'I'oc:llnic:d 12opor.t '1'122004- l!)(i:l. Cornc>ll IJ~iiwrsit.y, 2004. http: //people . cs.uchicago.edu/"pff/papers/dt.pdf.

[:I01 P.F. l : c : l ~ c ! ~ ~ ~ z ~ i ~ l l j a ~ ~ t l D.P. EI~ittc~~loc:l~c:r. Pic:torii\l s t , r ~ ~ c + ~ ~ r w for ol,,jvc,t ~ u ~ o g ~ ~ i t i c ) n . 17~le1.7~(1.tio71.(1.1 , Jown(1 .1 of Co711,put~r I/'~sZO~L 61 ( 1) :55 79, 2005.

r ! [iU] kl . FiscI~l(!r i111~1 I t . E I sc : l~ l ag~ . 1 IIC ~,cl)rc:s(!~~t,~~t,ioli i111(1 ~ i ~ a t c l ~ i i ~ g of' 1)ict ori;lI st r11c:t I~I , ( ! s .

1EEE 'li-c~~~sact.ion.s 076 fi711,p1~ters, C'-22( 1):67 92, 1973.

[46\ '1'. . J o t ~ c l ~ i l ~ ~ s . . / I ~ U ( I , I I , ( : V . ~ , i r ~ I<(TII,P/ i W ~ ; t / ~ ~ d s , S I L ~ ~ O I . / , V(Y;LO~. Lc/~,rr~,in!j. (:11a11)t(!r 11: h l i~k- itig L:~rg(>-S(,alo SVhf L(:arl~ing Practical. h,llrI'-l'r(w. 1999.

[SO] I . 1<1111t,tll. I,. 1,ol)istij. .I. I t i~~ l l~ i l l l . ~ i~ i~ , . iltlcl A . Visi~. h111lfisc.alc Fo~t r ic t tlcsc.ripl.or for s l l a p ~ c:l;~ssiIic:nliou. 111 l'rvccctli7rgs oJ' Lh(t 12tI1. J ~ L / , ( : T . I I , ( I . / % O ~ L ( I ~ (,'or~:fi:u:n(t 0 1 1 IIIIMI,!/I: A ~ I M I ~ / : I ~ s ~ , s (ICIA '03). ppys 5:$6 5/11! 200:{.

[73] Rat*li(~l Ross, Rolm l)irtl-\~~:~t(:li(~r: ;11i i~~ tc l l ig t~~i t vi(l(!v s y s l t ~ ~ i i l l ; I I ~ A ~ , k ~ ~ i s ; i s I ) ; ~ ~ O I I soal.c.lic:s for ; I I ~ c~l~lsivc! I)irtl, http: //www . technologyreview . com/Inf otech/l8205/ page 1 / . 'l?d~moLo!j;g l?(:ui(w. F c l ) r ~ ~ ~ y 2007.

[78] S t q ) l ~ ~ t t SC: David I,ow(:! ; ~ t d . l i ~ i ~ Lit,t,lc. C ; lo l~~ l lt)(:i~li~i~t.ioti 1tsi11g (list,i~ici.ivc f(!i~.t.~ll-(?~. 111 P r o ( w d ~ 7 1 q ~ I! f /./LC 2002 IEF,'E/RS.J Fnt rmsrr tion,o.l C ~ 0 7 1 . j t i r v 7 ~ ~ ( ~ on. Intelligent Xo hols a r d S;ij.stcvri,s~ ~);lgc:s 226 231: 0ctolx:r 2002.

IS%] I>i~ltl)ia Scw~;uii. <:llristol)lic! Sa.int.-.l(:a~i, Carl Frhlicot.: 'l'liicrry Bo~lw~ilmis, ant1 Picrrc C~ortrtc>ll(~~~~oiit . . St.i.wl~~rr(rl. Syrrttwtic. r m d Stal is l ic~d Pali.r:m I t~~-o!j~~il . io~i~: ( : l i ;~p t c~ Alive fislics sl)c?c:ic!s c:lr;~t,ac,lc.rizi~tioli ft,o~ii vitl(3o socp~clic:os. 1,cxturc Not,os i r t Colnj)~lt,er Sci- c,~i(,c,. S~)riligc?r, 2002.

[Mi] Pit111 S ~ ~ C ~ C I I S ! I';~~(:itl FII ;~: i t t i (1 A I I ~ ~ ( ! \ v .I. H ~ I I I S ~ I I . ( '!o~ii~)~tt;~.tiott;~l st~rat.(~giw ~ O T object, I , P C O ~ I ~i t i o~ i . 11 (,'07ii,p l/di~l,!j A%L7V~:ij,~ (Ct?lJR)! 24:s (2: h 1i1 r(:h 1!)$)2,

Index