Using S40 to build mobile robot vision

Dr. Michael Montvelishsky

1Friday, November 19, 2010

L R

D

To make machine visual depth perception, using two images captured from slightly different

viewpoints as an input.

The Goal is

2Friday, November 19, 2010

Minimum disparity search, using SAD (Summ of Absolute Differences) as

similarity measure

sad k l rx y i j i k j

j y w

y w

i x w

x w

, , ,( ) = +

=

+

=

+

12

12

12

=

12

d n k sad n sad kx y x y x y, , ,: { | : ( ) ( )}

The Algorithm is

3Friday, November 19, 2010

MinimumDisparitySearch

sad3 3 0, ( ) sad3 3 1, ( ) sad3 3 2, ( ) sad3 3 15, ( )

4Friday, November 19, 2010

Required modules:

• Input driver

• Output driver

• Storage for 2x2 input pixel lines (FIFOs)

• 16 SAD calculators

• ARG MIN module

5Friday, November 19, 2010

Application Map

! "#$#%

&#$#%

'#$#%

(#$#%

)#$#%

*#$#%

+,-./"&

0,-./"'

1,-./"(

"!23$4

""#$#%

"&#$#%

"'#$#%

"(#$#%

")#$#%

"*#$#%

"+,-./""

"05$43-

"1,-./")

&!63$4

&"7822

&&9$6:

&',-./&

&(,-./'

&),-./*

&*,-./+

&+,-./"!

&05$43;

&1%8<

'! '";8##

'&,-./!

'',-./"

'(,-./(

'),-./)

'*,-./0

'+,-./1

'05$43=

'15$43.

&(100*+1/4, <%<-2/2%-.> '!/?

6Friday, November 19, 2010

SSD Map

! "#$#%

&#$#%

'#$#%

(#$#%

)#$#%

*#$#%

+#$#%

, -

"! ""./00

"&1$23

"'#$#%

"(#$#%

")#$#%

"*#$#%

"+#$#%

", "-

&! &"4456!

&&4456"

&'4456&

&(4456'

&)4456(

&*4456)

&+4456*

&,4456+

&-4456,

'! '"7$869

'&7$86:

''4456#

'(44563

')44565

'*4456;

'+44569

',4456:

'-4456-

(""(!&,'684 <%<:060%:5= '&6>

7Friday, November 19, 2010

SAD vs SSD

SAD SSD

8Friday, November 19, 2010

Comparison

Algorithm Platform Size FPS Power(Watt)

Efficiency(Pix/J)

SSD CSX 700 640x480 160 9 5.4 e6

SSD S40 384x288 15 0.080 20.7 e6

SAD S40 384x288 25 0.075 36.9 e6

9Friday, November 19, 2010

Prototype board and Sanyo AM-5308 Amorphous Solar Cell (117mW with

Vop=1.9V and Iop=61.5mA)

!"#$%&%%!!"#$%&'()*&+(,-#

• '()*%"%+,-.%"/0%1,2'*%3"))$45%)*$%678879(/#%:"38$%;*79;%)97%$<"=>8$;%76%3"))$45%8(6$%)(=$%";%6?/@)(7/%76%)*$%0$>)*%=">%4")$A%

",.-*#/#01,23-*#%'#.,++*45#-('*#'%4#67'38#,&)#9'38#858+*2:#

;*3+<#2,3#!2,=*#>?@A16@@#3(1*-8#/9#)*3+<#B,-C*8D#B="#$%@7=>?)")(7/%4")$%C6>;D%!79$4%@7/;?=>)(7/%C='D%E"))$45%8(6$%C0"5;D%

&F%-F%G&,F%2%HI%GHJ,F%

! :*$%.$45%879%>79$4%6$")?4$;%76%)*$%0$.$87>$0%*"409"4$%="K$;%()%>7;;(38$%)7%?;$%L78"4%@$88;%";%$()*$4%)*$%="(/%74%)*$%;$@7/0"45%;7?4@$%M@*"4#(/#%)*$%3"))$45N%76%>79$4%674%)*$%;5;)$=,%:*$%L"/57%OPQF+JI%O=74>*7?;%L78"4%R$88%MHH-='%9()*%S7>TH,US%"/0%B7>T2H,F=ON%(;%@?44$/)85%?;$0%674%)*$%>4776%76%)*$%@7/@$>),%

%

E(=:#?#"<*#;*B*-%3*)#34%%'#%'#F%&F*3+#<,4)G,4*#F%23,4*)#G(+<#H,&5%#H%-,4#$*--:#10Friday, November 19, 2010

Intelligent Vehicles

11Friday, November 19, 2010