Lec Structured Light2

7/28/2019 Lec Structured Light2

1/60

StructuredStructuredLight

Based

Light

Based

CS635Spring2010

.

DepartmentofComputerScience

PurdueUniversity


2/60

Ac uirin

D namicScenesAc uirin

D namic

Scenes

Ac uisition:ca tureasmuchinformationas ossibleinonetoafewframes

Byexploiting

coherence

yexp o ngsevera c anne s o n orma on e.g.,co or, n rare ,e c


3/60

Ac uirin

D namicScenesAc uirin

D namic

Scenes

InfraredLight,

Frueh

and

Zakhor,

PROCAMS

2005

Usesingleframesandinfraredillumination

RapidShapeAcquisitionUsingColorStructuredLightandMultipassDynamic

Programming,

Zhang

et

al.,

3DPVT

2002

Fast3DScanningwithAutomaticMotionCompensation,Weiseetal.,CVPR

Usephaseshiftingandmotioncompensationoverafewframes

ea me o e cqu s on , us n ew cze a .,

Usepatternsproducinglocalcorrespondencesoverafewframesandmerge


4/60

Capturing2DDepthandTextureofTimeCapturing2DDepthandTextureofTime

ary ng

cenes

s ng

tructure

n rare

g tary ng

cenes

s ng

tructure

n rare

g t


5/60

Line definedat middle of

IR strip

How do you know which line is which? Ideas?


6/60

H-line sweeps up/down at2Hz and enables an orderingof (a subset of) the V-linesand thus ermits theircorrespondence


7/60

Grab color image

Foreground

segmentation, andense ep

interpolation

Put it all together

IR camera at 30Hz, color camera at 10Hz

(probably faster today)


8/60

RapidShapeAcquisitionUsingColorStructuredRapidShapeAcquisitionUsingColorStructured

g tan

u tg t

an

u t

pass

ynam c

rogramm ngpass

ynam c

rogramm ng

Recall:howdowe

corres ondlines?


9/60


g tan

u tg t

an

u t

pass

ynam c

rogramm ngpass

ynam c

rogramm ng

DefinelinesatthetransitionsfromcolorA


10/60


g tan

u tg t

an

u t

pass

ynam c

rogramm ngpass

ynam c

rogramm ng

Resolution.Why?


11/60


g tan

u tg t

an

u t

pass

ynam c

rogramm ngpass

ynam c

rogramm ng

Onl havethreecolorchannels R G B andcanonly

robustly

differentiate

strong

color

changes

Thisreducesthenumberofcolorstouse,and

Oftenresultsinambiguityinthecolorcoding


12/60


g tan

u tg t

an

u t

pass

ynam c

rogramm ngpass

ynam c

rogramm ng

Challenges Givenacolorcode,howtodobest

correspondthestripes?

Withtheaboveinmind,howdowedesignagoodcolorcode?


13/60


g tan

u tg t

an

u t

pass

ynam c

rogramm ngpass

ynam c

rogramm ng





14/60

Solution DynamicProgramming


15/60

(rectifiedimages)


16/60


17/60

= Hiii ,...,, 2

2

1

1

Multiple match

hypotheses

),(ij

eqsSimilarityscore(ofcolor)between

edgeei

andtransitionqj

is

H

== kij kk eqs1 ,core

o

e

en re

ma c

sequence


18/60

D namic ro rammin ob ectiveis: maxar f

However,thespaceallpossible isverylarge:O(MN)

Solution?

ssumemono on c y o e ep or er ng :

Hiii ...21

Great!But

this

monotonicity does

not hold

in

what

situation?

Occlusions!Ohwell

But itholds

for

individual

fragments,

which

we

can

combine


19/60

D namic ro rammin ob ectiveis: maxar f

Letoptimal becalled *

=* ),()( 1,1*

ijij eqsf +

0 ifj=0ori=0

)( 1,*

ijf

)( ,1 ijf max

ffoundthrougharecursivesearchandsomeoptimizationstofurtherreducethesearchspace(e.g.,assumeatmostsmall

depthchangesfromonecolumntoanother)


20/60


g tan

u tg t

an

u t

pass

ynam c

rogramm ngpass

ynam c

rogramm ng





21/60

,

isa

cyclic

sequence

of

a

given

alphabet

A with

size

n inA appearsasasequenceofconsecutive

charactersexactly

once

B(k,n) haslengthkn

, B(2,3)= 00010111or 11101000


22/60

,,

Canalsobeconstructedb takin

a

Hamiltonian

path

of

an

n

dimensionalDeBruijngraphover

. .,

Hamiltonian

athmeans

each

vertexisvisitedonce)


23/60

= =

000 isuseless

Colorse uenceiscreatedb = XOR d

XORingeffectivelyflipsbitsusingdj

p0 isa

chosen

initial

color

(e.g.,

100)

Want3letterssequencesdj tobeunique

Inpracticeabout125stripesissufficient

Thus,a

B(5,3) is

adequate


24/60


25/60

Fast3DScanningwithAutomaticFast3DScanningwithAutomatic

MotionCompensationMotion

Compensation

method

Usesphaseshiftingandmotioncompensation


26/60

Fast3DScanningwithAutomaticFast3DScanningwithAutomatic

MotionCompensationMotion

Compensation


27/60

(moving)

object

ro ector camera


28/60

,

correlationbased

stereo

is

used

to

Ana t ona mo cat on spropose tohandlediscontinuities(whichalsoplague

stan ar

p ases t ng


29/60

(slidesandvideosofthissectionbySyzmon Rusinkiewicz @Princeton


30/60

RealRealTime3DModelAcquisitionTime3DModelAcquisition

PipelinePipeline

3D Scanne3D Scanne3D Scanne3D Scanne

AlignmentAlignmentAlignmentAlignmentView PlanningView Planning

Mer inMer inMer inMer inDone?Done?

HumanHuman

DisplayDisplayDisplayDisplay


31/60


PipelinePipeline

3D Scanne3D Scanne

AlignmentAlignmentView PlanningView PlanningView PlanningView Planning

Mer inMer inDone?Done?Done?Done?

a enge:a enge:Real TimeReal Time

DisplayDisplay


32/60


PipelinePipeline

3D Scanne3D Scanne

AlignmentAlignmentAlignmentAlignmentView PlanningView PlanningView PlanningView Planning

Part I:Part I:

Mer inMer inMer inMer inDone?Done?Done?Done?

StructuredStructured--LightLightTriangulationTriangulation



33/60


PipelinePipeline




art :art :Fast ICPFast ICP



34/60


PipelinePipeline




Voxel GridVoxel Grid

DisplayDisplay


35/60

ObjectObject

LaserLaser

(x,y)(x,y)

CameraCamera


36/60

Correspondenceproblem:

which

stripe


37/60

ss gn meco es

tostripe

boundaries

Performframetoframe

tracking

of

correspondingboundaries

Propagateilluminationhistory

Illumination history = (WB),(BW),(WB)Illumination history = (WB),(BW),(WB)

CodeCode

[HallHolt&Rusinkiewicz,ICCV2001]


38/60

lotsof

black/white

edges

at

each

frame

boundariesthatcantbeseendirectly


39/60

0000 1101

1010 0111

10000101

11111111 00100010

1011 0110

00010001 11001100

0100 1001

00111110

[Hall[Hall--Holt & Rusinkiewicz, ICCV 2001Holt & Rusinkiewicz, ICCV 2001


40/60


41/60

ProjectProject CaptureCapture FindFind MatchMatch ComputeCompute

DLPprojector

illuminates

scene

@

60

Hz.

CodeCode ImagesImages BoundariesBoundaries BoundariesBoundaries RangeRange

SynchronizedNTSCcameracapturesvideo

.


42/60


PipelinePipeline




art :art :Fast ICPFast ICP



43/60

movingobjects

,

toeachotherasobjectmoves


44/60

onescan,

minimize

distance

to

closest

point


45/60

an era e o n a gnmen

IteratedClosest

Points

(ICP)

[Besl

&

McKay

92]


46/60

Inthis

pipeline,

scans

close

together

minimum

~

Pointtoplaneminimization

ro ec on

ase

ma c ng Withthesetweaks,runningtime~milliseconds

[Rusinkiewicz&Levoy,3DIM2001]

R lR l Ti 3D M d l A i itiTi 3D M d l A i iti


47/60


PipelinePipeline




Voxel GridVoxel Grid

DisplayDisplay


48/60

tobe

able

to

see

holes

accumulateslinearlywithtime

Stan ar g qua tymerg ngmet o s:processingtime~1minuteperscan


49/60


50/60


51/60


52/60

++


53/60

,


54/60

18 cm.18 cm.


55/60


56/60

Quality/speedtradeoff

,

Offlinepostprocessingforhighqualitymodels

Globalregistration

Highqualitymerging(e.g.,usingVRIP[Curless96])


57/60


58/60


59/60

obtainingcomplete,

hole

free

models

Piecesofpipelineselectedforrealtimeuse:

Structuredlightscannerformovingobjects

FastICPvariant

Simplegrid

based

merging,

point

rendering


60/60

Couldbe

made

equivalent

with

careful

Ultimatelimitationsonquality:focus,texture

Duetonoise,miscalibration,degenerategeometry

e uce , u no e m na e , y anc orscans

PossiblycombineICPwithseparatetrackers

Lec Structured Light2

Documents

Transcript of Lec Structured Light2