My Research Work New

8/9/2019 My Research Work New

1/40

Tracking with 3D Data

M. Usman Butt

Department of Electrical and Computer Engineering

10 July, 2012


2/40


3/40

Objectives

To propose a novel solution for robust objectdetection and tracing using 3D !depth" informationand other features# if necessar$# such as te%ture#edges etc&

To design and develop a program implementing thescheme&

To investigate and identif$ the problems for real'time implementation of the scheme for objectdetection and tracing&

To develop the scheme on an ()*+ or a *),platform for real'time implementation&

To evaluate and validate the performance of thes$stem b$ comparing with e%isting state'of'the'art

s$stems&


4/40

Previous Work

Stereo with-o Intensit$ and binar$ template(Beymer et al., Frame-rate, 1999)

o.olour and pattern(Darell et al., IJCV, 2000)

o.olour and motion(Yang et al., IJIST, 2005)

o Intensit$# 3D world point clouds !plan view"(Baha!r" et al., #$$l. Intell"gen%e, 200&)

oSparse set of feature points and ernel'basedclustering

(Ca" et al., 'attern e%!gn"t"!n, 2010)


5/40

M !""roach

Onl$ 3D information from a high resolution disparit$ mapgenerated b$ implementation of S$mmetric D$namic)rogramming Stereo !SD)S" algorithm on ()*+

Object detection from SD)S disparit$ map

Tracing using the 0alman 1lter

+ssumptions + suitable bacground disparit$ map is available

Objects of interest can be an$ rigid !e&g& vehicles" or non'rigid !e&g&human" object# but object sies must lie in a pre'de1ned range

ach traced object 1rst appears as an isolated object and moves

upright Object speed is less than 45 m s '4 !Ol$mpic sprinter" for human

tracing

The ground plane is basicall$ 6at and 1%ed

.ameras are tilted at 1%ed angle to the ground plane


6/40

#mmetric Dnamic Programming#tereo $#DP#%Matches scan line b$ scan line

)roduces disparit$ map seen b$ a.$clop8an e$e

9as a nice mathematical propert$o +n$ change in disparit$ along a

scan line is accompanied b$ onemonocular point for each unitchange in disparit$• : *+ 1, 2, , * are M; or M< pi%els

Objects separated from their

bacground are outlined with bandsof M; or M< points


7/40

#mmetric Dnamic Programming#tereo $#DP#%

+n object>s 3D structure is represented b$ severalcontours

+ e$ problem

oDetermine the contour best outlining the object

Major Drawbac

o

Streaing artefacts

Disparit$ map .ontoursSphere

-*


8/40

#tereo &ardware #stem

SD)S algorithm implemented on ()*+ as wellas *),

4 Mpi%els at 35 fps

Disparit$ range 5'42?


9/40

'a(man)i(ter

#DP#!(gorithm$)P*! or

*PU%

Object Detection and Tracking

<image

;image

Dispmap

Occl

map

Thresholding disp

map

MOOA

.ontour*eneration

3DObjectmodel

DetermineBC !object"contours


10/40

'a(man)i(ter

#DP#!(gorithm

$)P*!%


<image

;image

Dispmap

Occl

map

Thresholding disp

map

MOOA

.ontour*eneration

3DObjectmodel



11/40

'a(man)i(ter

#DP#!(gorithm

$)P*!%


<image

;image

Dispmap

Occl

map

Thresholding disp

map

MOOA

.ontour*eneration

3DObjectmodel



12/40

'a(man)i(ter

#DP#!(gorithm

$)P*!%


<image

;image

Dispmap

Occl

map

Thresholding disp

map

MOOA

.ontour*eneration

3DObjectmodel



13/40

Mor"ho(ogica( O"en O"eration$MOO%Before MOO +fter MOO


MOO


14/40

'a(man)i(ter

#DP#!(gorithm

$)P*!%


<image

;image

Dispmap

Occl

map

Thresholding disp

map

MOOA

.ontour*eneration

3DObjectmodel



15/40

'a(man)i(ter

#DP#!(gorithm

$)P*!%


<image

;image

Dispmap

Occl

map

Thresholding disp

map

MOOA

.ontour*eneration

3DObjectmodel



16/40

'a(man)i(ter

#DP#!(gorithm

$)P*!%


<image

;image

Dispmap

Occlmap

Thresholding disp

map

MOOA

.ontour*eneration

3DObjectmodel



17/40

Object Detection !(gorithm

.ompare the candidate contours at adjacentdisparities and determine the binocularl$ visible Gtrue>contours representing the object outline using rules-

If the centroid of B,, lies inside B-1, # then B, isenclosed b$ B-1, # then

If the heights of both the contours are same withinthe tolerance# !h, 3h-1, 4 h" and

The internal contour area ratio# #, #-1, 6 0.9, then

B, is an object outline


18/40

'a(man)i(ter

#DP#!(gorithm

$)P*!%


<image

;image

Dispmap

Occlmap

Thresholding disp

map

MOOA

.ontour*eneration

3DObjectmodel



19/40

'a(man)i(ter

#DP#!(gorithm

$)P*!%


<image

;image

Dispmap

Occlmap

Thresholding disp

map

MOOA

.ontour*eneration

3DObjectmodel



20/40

'a(man)i(ter

#DP#!(gorithm

$)P*!%


<image

;image

Dispmap

Occlmap

Thresholding disp

map

MOOA

.ontour*eneration

3DObjectmodel



21/40

'a(man)i(ter

#DP#!(gorithm

$)P*!%


<image

;image

Dispmap

Occlmap

Thresholding disp

map

MOOA

.ontour*eneration

3DObjectmodel



22/40

Object Detection !(gorithm

(O< +.9 OBH.T I )


23/40

'a(man)i(ter

#DP#!(gorithm

$)P*!%


<image

;image

Dispmap

Occlmap

Thresholding disp

map

MOOA

.ontour*eneration

3DObjectmodel



24/40

'a(man)i(ter

#DP#!(gorithm

$)P*!%


<image

;image

Dispmap

Occlmap

Thresholding disp

map

MOOA

.ontour*eneration

3DObjectmodel



25/40

ki


26/40

Tracking

0alman (ilter predicts the object position in the ne%t frame

+deNuate for simple linear model

(aster computationall$ simple )redicted data helps to

o Identif$ heads cut oP due to SD)S streas

o (ind missing objects !occluded or buried in noise" in the frame

Search in a small area around the predicted position for best match withthe template

4e-erence Tracking Point .entroid (: %, Y %, ; % )

o +n obvious choice but# for articulated objects!person"# centroid shifts as the object deforms

(eet (: / , Y / , ; / )o *round contact point changes signi1cantl$ as

people wal

o ,suall$# feet cut oP b$ bacground subtraction

9ead (: h, Y h, ; h )

o +lmost alwa$s visible


27/40

4e-erence Point +hoice

4e-. Tracking Point 5$ * c , + ( , ( &

670m 6 8 0m 690m

+entroid

5&5/ 5&5R 5&4@

)eet 5&5@ 5&5? 5&47

&ead 5&57 5&5/ 5&4/


28/40

4esu(ts

#cenario !


29/40

4esu(ts

#cenario !


30/40

4esu(ts

#cenario B


31/40

4esu(ts

#cenario B


32/40

4esu(ts

#cenario +


33/40

4esu(ts

#cenario +

'2 '2 '4 '4 5 4 4 2 2

+ +!)red" B B!)red" . .!)red"

7 m

9 m

'5&2

'5&4

5&5

5&45&2

+ B .

)rame :

Deviation$7% m

'2

'4

5

4

2

+!K" +!)redSK"

B!K" B!)redSK"

)rame :

7 m

'5&='5&/'5&2 5&5 5&2 5&/ 5&=5

45

25

35

/57 Devition

+

B

.

'5&2

'5&4

5&5

5&4

5&2

+ B .

)rame :

Deviation$8% m

5

4

4

2

2

3+!L" +!)redSL"

B!L" B!)redSL"

.!L" .!)redSL"

)rame :

8 m

'5&='5&/'5&2 5&5 5&2 5&/ 5&=5

7

45

47 8 Devition

+

B

.

'4&5

'5&7

5&5

5&7

4&5

+ B .

)rame :

Deviation$9% m

45

47

25

27

35 +!M " +!)redSM "B!M " B!)redSM "

)rame :

9 m

'5&='5&/'5&25&5 5&2 5&/ 5&=5

4

2

3

/9 Devition

+

B

.

a" d

"

g"

b" e" h"

c" f" j"


34/40

4esu(ts


35/40

4esu(ts

#cenario D


36/40

4esu(ts

#cenario D


37/40

4esu(ts

9eight !L"#m

+ctual Measured

Subject + /.;< /.=>


38/40

4esu(ts

Per-ormance

Time# s

Bacgroundsubtraction

Detection Tracing!0("

Total

Scenario +!Single)erson"

//


39/40

4esearch Progress to Date

Designed and developed isolated object detection and tracing techniNue

from depth map

Developed and integrated 0alman (ilter !0(" into the tracing scheme

+cNuired video seNuences for several outdoor scenarios with the help of.IT< group to test the developed techniNue

Implemented and tested the techniNue on the video seNuences withisolated objects

Based on anal$sis of results of isolated object tracing# improved thedetection and tracing techniNue to handle the occlusion problem

Tested the improved version on the video seNuences with the presence ofocclusion

+ttended one of the Doctoral Sills )rogramme Induction Da$s

Submitted full thesis proposal

)ublished a paper-

A. . B8tt an J. A!rr", reci$e .racking u$ing /ig( e$olutioneal-time tereo, Image an V""!n C!m$8t"ng Ee ;ealan(IVCE;2011), #8%lan, Ee ;ealan, IVCE;, 2011, $$. 1=@-1=>.


40/40

)uture Work

Identi1cation of bottlenecs for real'time implementation

)aralleliation of the algorithm for real'time implementation

Stud$ of architecture of diPerent hardware platforms !*),and ()*+" for suitabilit$ of paralleliation

Development of algorithm on suitable platform !()*+ or

*)," for real'time processing Testing of real'time performance of the algorithm

(urther improvement in the algorithm based on resultanal$sis

valuation of the algorithm

My Research Work New

Documents

Transcript of My Research Work New