Nonrigid Surface Modelling and Fast Recoverylyu/student/phd/jackie/Jackie_Term2_ppt.pdf · ed an...
50
1 Nonrigid Surface Modelling and Fast Recovery Zhu Jianke Supervisor: Prof. Michael R. Lyu Committee: Prof. Leo J. Jia and Prof. K. H. Wong Department of Computer Science and Engineering The Chinese University of Hong Kong May 11, 2007
Transcript of Nonrigid Surface Modelling and Fast Recoverylyu/student/phd/jackie/Jackie_Term2_ppt.pdf · ed an...
1
Non
rigid
Surf
ace
Mod
ellin
gan
d Fa
st R
ecov
ery
Zhu
Jian
ke
Sup
ervi
sor:
Pro
f. M
icha
el R
. Lyu
Com
mitt
ee: P
rof.
Leo
J. J
iaan
d P
rof.
K. H
. Won
g
Dep
artm
ent o
f Com
pute
r Sci
ence
and
Eng
inee
ring
The
Chi
nese
Uni
vers
ity o
f Hon
g K
ong
May
11,
200
7
2D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
3D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Out
line
•In
trodu
ctio
n
•Tw
o-st
age
Sch
eme
with
Act
ive
App
eara
nce
Mod
els
for n
onrig
id s
urfa
ce re
cove
ry
•P
rogr
essi
ve F
inite
New
ton
App
roac
h to
non
rigid
su
rface
det
ectio
n
•C
oncl
usio
n
4D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Intr
oduc
tion
•N
onrig
idsu
rface
mod
ellin
gan
d de
tect
ion
are
esse
ntia
lly th
e co
mpu
ter v
isio
n ta
sks
in a
var
iety
of a
pplic
atio
ns:
–im
age
alig
nmen
t–
med
ial i
mag
ing
–au
gmen
ted
real
ity–
hum
an c
ompu
ter i
nter
actio
n –
digi
tal e
nter
tain
men
t
5D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Non
rigid
Surf
ace
Mod
ellin
g
•D
ata
embe
ddin
g: P
rinci
pal C
ompo
nent
Ana
lysi
s–
Act
ive
App
eara
nce
Mod
els
–A
ctiv
e S
hape
Mod
els
–3D
Mor
phab
leM
odel
s•
Phy
sica
l mod
el: F
inite
Ele
men
t Mod
el (F
EM
)–
Act
ive
Con
tour
s (S
nake
s)•
Thin
-Pla
te S
plin
e–
TPS
-RP
M–
Sha
pe C
onte
xt–
Gau
ssia
n M
ixtu
re M
odel
s ba
sed
poin
ts s
et re
gist
ratio
n
6D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Non
rigid
Surf
ace
Rec
over
y
•Ap
pear
ance
-bas
ed M
etho
d–
Als
o kn
own
as d
irect
met
hod
–Lu
cas-
Kan
ade
algo
rithm
bas
ed m
etho
ds [L
K IJ
CA
I’81]
–A
ctiv
e A
ppea
ranc
e M
odel
s [C
oote
sP
AM
I’01]
–3D
Mor
phab
leM
odel
s [V
ette
r Sig
grap
h’99
, PA
MI’0
1]–
2D+3
D A
AM
s[X
iao
CV
PR
’04]
•Fe
atur
e-ba
sed
Met
hod
–S
hape
con
text
[Bel
ongi
eP
AM
I’02]
–TP
S-R
PM
[Hui
CV
IU’0
3]–
Ker
nel c
orre
latio
n [E
CC
V’04
]–
Gau
ssia
n m
ixtu
re m
odel
s [J
ian
ICC
V’0
5]–
Sem
i-im
plic
it op
timiz
atio
n sc
hem
e [P
iletC
VP
R’0
5, IJ
CV
’07]
7D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Par
t I
A T
wo-
stag
e S
chem
e fo
r Non
rigid
Sur
face
R
ecov
ery
with
Act
ive
App
eara
nce
Mod
els
8D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Part
I: S
truc
ture
•M
otiv
atio
n
•Ex
tend
ed A
ctiv
e A
ppea
ranc
e M
odel
s (A
AM
s) fi
tting
•Tw
o st
age
sche
me
for n
onrig
id s
urfa
ce re
cove
ry–
Offl
ine
cons
truct
ion
of 3
D s
hape
mod
el–
Estim
ate
3D p
ose
and
non-
rigid
sha
pe p
aram
eter
s
•Ex
perie
men
tal R
esul
ts
•S
umm
ary
of P
art I
9D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Mot
ivat
ion
•N
onrig
id s
hape
reco
very
for A
ugm
ente
d R
ealit
y–
Rig
id O
bjec
t •
L. V
acch
etti
et a
l. (P
AM
I‘04)
pro
pose
d an
effi
cien
t sol
utio
n fo
r 3D
rig
id o
bjec
t tra
ckin
g•
Two
2D A
AM
s ap
proa
ch fo
r rig
id o
bjec
t pos
e es
timat
ion
–N
on-r
igid
Obj
ect
•V
.Bla
ndz:
3D
Mor
phab
le M
odel
s•
J.A
hlbe
rg: 3
D A
AM
with
gen
eric
Mod
el.
•Ji
ng X
. (C
VP
R‘0
5) 2
D+3
D A
AM
10D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Act
ive
App
eara
nce
Mod
els
•A
AM
sis
def
ined
by
its s
hape
and
text
ure
•Th
e 2D
sha
pe a
nd te
xtur
e ar
e co
ntro
lled
by a
sta
tistic
al m
odel
. The
y ca
n be
re
pres
ente
d as
a b
ase
plus
a li
near
com
bina
tion
of v
aria
tions
:
•D
irect
App
eara
nce
Mod
els
11D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Exte
nded
AA
Ms
Fitti
ng A
lgor
ithm
•O
bjec
t: ||
I –Im
||•
Idea
: the
pro
pose
d ap
proa
ch p
redi
cts
shap
e di
rect
ly fr
om te
xtur
e
whe
re r
= gi
-gm
, is
the
resi
dual
imag
e
12D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Exte
nded
AA
M F
ittin
g
13D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
AA
M F
ittin
g Sa
mpl
e
The
AA
Ms
are
built
up
with
140
stil
l fac
e im
age
belo
ngin
g to
20
indi
vidu
als,
7 im
ages
for e
ach.
The
fitti
ng e
xper
imen
t is
perfo
rmed
on
an
AA
M w
ith 1
4 sh
ape
para
met
ers,
68
text
ure
para
met
ers,
and
363
35 c
olor
pi
xels
.
14D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Two-
stag
e Sc
hem
e N
onrig
idSu
rfac
e R
ecov
ery
•Tr
aini
ng a
nd b
uild
ing
offli
ne b
asis
Acq
uire
the
2D s
hape
of o
bjec
ts u
sing
the
AA
M fi
tting
alg
orith
m,t
hen
cons
truct
the
3D s
hape
bas
is.
•O
nlin
e tra
ckin
g–
Ste
p 1.
2D
AA
M fi
tting
and
trac
king
–S
tep
2. e
stim
ate
the
3D p
ose
and
shap
e pa
ram
eter
s si
mul
tane
ousl
yvi
a lo
cal b
undl
e ad
just
men
t by
build
ing
up th
e po
int c
orre
spon
denc
es
betw
een
2D a
nd 3
D.
15D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Bui
ldin
g O
fflin
e 3D
Mod
el
•3D
sha
pe
•W
eak-
pers
pect
ive
proj
ectio
n
16D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Bui
ldin
g O
fflin
e 3D
Mod
el
•Fa
ctor
izat
ion
met
hod
[Bre
gler
CV
PR
’00,
CV
PR
’01]
17D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Onl
ine
Alg
orith
m
•Th
e op
timiz
atio
n pr
oble
m c
an b
e de
rived
as:
whe
re
deno
tes
the
proj
ectio
n of
3D
sha
pe g
iven
the
para
met
ers
A, R
and
T.
18D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Expe
riem
enta
l Res
ults
I
Pen
tium
III 1
GH
z C
PU
, 20
0ms
per i
mag
e of
siz
e 35
2x28
8. A
AM
fitti
ng ta
kes
40m
s an
d 3D
re
cove
ry s
tep
take
s 74
ms.
The
AA
M w
ith 1
0 sh
ape
para
met
ers,
52
text
ure
para
met
ers.
6
cam
era
para
met
ers
and
6 3D
sha
pe p
aram
eter
s.
19D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Expe
rimen
tal R
esul
ts II
•D
eter
min
e th
e nu
mbe
r of 3
D s
hape
bas
is
20D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Aut
omat
ic F
ace
Mod
ellin
gSc
hem
e
21D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Ren
derin
g in
Diff
eren
t Vie
w
22D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Re-
text
urin
g fa
ce
23D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Dis
cuss
ion
•R
igid
vs.
Non
-Rig
id–
The
prio
r mod
el e
mpl
oyed
by
L. V
acch
etti
et a
l. is
onl
y fo
r rig
id o
bjec
ts o
r def
orm
able
ob
ject
s w
ith s
mal
l var
iatio
ns.
–P
. Mitt
rapi
yanu
mic
et a
l. do
not
take
full
adva
ntag
e of
AA
M’s
defo
rmat
ion
pow
er•
Offl
ine
vs. O
nlin
e–
In c
ontra
st to
the
offli
ne a
lgor
ithm
s, o
ur p
ropo
sed
met
hod
is a
ble
to w
ork
onlin
e by
ex
ploi
ting
the
3D s
hape
mod
els
that
can
be
cons
truct
ed o
fflin
e ef
fect
ivel
y by
AA
M
track
ing
•A
dvan
tage
s fo
r AR
app
licat
ions
–G
ener
ic v
s. p
erso
n sp
ecifi
c: h
andl
e la
rge
text
ure
varia
tions
, fitt
ing
to d
iffer
ent
indi
vidu
als
–C
ombi
ned
2D+3
D A
AM
: wea
k-pe
rspe
ctiv
e m
odel
•D
isad
vant
ages
and
Fut
ure
Wor
k–
does
not
take
full
adva
ntag
e of
3D
info
rmat
ion
for s
peed
ing
up A
AM
con
verg
ence
.–
Larg
e ro
tatio
n m
ay b
e co
mpe
nsat
ed b
y th
e 3D
line
ar m
ode,
ther
efor
e, th
e es
timat
ed
pose
is n
ot s
o ac
cura
te.
–Tr
aini
ng th
e 3D
AA
M w
ith th
e al
igne
d 3D
sha
pes
inst
ead
of 2
D s
hape
s.
24D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Sum
mar
y of
Par
t I
•A
nov
el tw
o-st
age
sche
me
for o
nlin
e no
n-rig
id s
hape
reco
very
tow
ard
Aug
men
ted
Rea
lity
appl
icat
ions
usi
ng A
AM
s.
•O
btai
n un
brok
en p
oint
cor
resp
onde
nces
acr
oss
mul
tiple
fram
es to
co
nstru
ct 3
D s
hape
mod
els
•P
rovi
de 2
D to
3D
ver
tex
corr
espo
nden
ces
in th
e on
line
track
ing.
•A
n ef
ficie
nt a
lgor
ithm
is p
ropo
sed
to e
stim
ate
both
3D
pos
e an
d no
n-rig
id s
hape
par
amet
ers
via
loca
l bun
dle
adju
stm
ent.
25D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Par
t II
Pro
gres
sive
Fin
ite N
ewto
n A
ppro
ach
to
Non
rigid
Sur
face
Det
ectio
n
26D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Part
II: S
truc
ture
•M
otiv
atio
n•
Pro
gres
sive
Fin
ite N
ewto
n A
ppro
ach
to n
onrig
id s
urfa
ce d
etec
tion
–2D
non
rigid
sur
face
mod
el–
Feat
ure-
base
d no
nrig
id s
urfa
ce re
cove
ry–
Fini
te N
ewto
n fo
rmul
atio
n–
Opt
imiz
atio
n•
Expe
riem
enta
l Res
ults
–C
ompu
tatio
nal e
ffici
ency
and
non
rigid
sur
face
det
ectio
n–
Aug
men
ted
real
ity–
Med
ical
imag
e re
gist
ratio
n•
Sum
mar
y of
Par
t II
27D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Mot
ivat
ion
•N
onrig
idsu
rfac
e de
tect
ion:
rec
over
ing
the
expl
icit
surfa
ce w
ith a
few
de
form
atio
n pa
ram
eter
s an
d fin
ding
out
the
corr
ect c
orre
spon
denc
es
from
noi
sy d
ata
sim
ulta
neou
sly.
•U
nlik
e th
e rig
id o
bjec
t, it
is d
iffic
ult t
o di
rect
ly e
mpl
oy a
robu
st e
stim
ator
to
rem
ove
the
spur
ious
mat
ches
for n
onrig
idsu
rface
det
ectio
n.
•Th
e ite
rativ
e m
etho
ds, s
uch
as T
PS
-RP
M [C
VIU
’03]
and
Sha
pe C
onte
x[P
AM
I’02]
, are
eith
er s
ensi
tive
to in
itial
con
ditio
ns a
nd p
aram
eter
cho
ices
, or
invo
lve
too
man
y ite
ratio
ns a
nd a
com
plex
opt
imiz
atio
n pr
oced
ure.
•S
emi-i
mpl
icit
met
hod:
an
auto
mat
ed a
ppro
ach,
and
can
be
appl
ied
for t
he
real
-tim
e A
ugm
ente
d R
ealit
y. [
Pile
tet a
l CV
PR
’05,
IJC
V’0
7]
28D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Rob
ust M
etho
ds
•R
AN
SA
C
•P
RO
SA
C
•M
-est
imat
or
•H
ough
tran
sfor
m
29D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Whe
re
a
re th
e ba
ryce
ntric
coor
dina
tes
for t
he p
oint
m.
2D N
onrig
idSu
rfac
e M
odel
30D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Non
rigid
Surf
ace
Rec
over
y
•E
nerg
y fu
nctio
n
–Ee
(s) i
s th
e su
m o
f the
wei
ghte
d sq
uare
err
or re
sidu
als
for t
he m
atch
ed p
oint
s.
–E
r(s)
is th
e re
gula
rizat
ion
term
that
repr
esen
ts th
e su
rface
def
orm
atio
n en
ergy
–is
a re
gula
rizat
ion
coef
ficie
nt.
31D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Reg
ular
izat
ion
Term
•E
r(s)
, als
o kn
own
as in
tern
al fo
rce
in S
nake
s, is
co
mpo
sed
of th
e su
m o
f the
squ
ared
sec
ond-
orde
r de
rivat
ives
of t
he m
esh
verte
x co
ordi
nate
s.
whe
re K
is a
spa
rse
and
band
ed m
atrix
whi
ch is
de
term
ined
by
the
stru
ctur
e of
the
mes
h m
odel
.
32D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Rob
ust E
stim
ator
•D
efin
e th
e re
sidu
al e
rror
•Th
e ro
bust
est
imat
or
M1
cont
ains
the
inlie
rmat
ches
M2
is th
e se
t of t
he o
utlie
rsTh
e or
der n
det
erm
ines
the
scal
e of
the
resi
dual
.
33D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Fini
te N
ewto
n Fo
rmul
atio
n
•Th
e ro
bust
est
imat
or fu
nctio
n is
not
con
vex
•Th
e as
soci
ated
pen
alty
func
tion
appr
oxim
atio
n pr
oble
m
beco
mes
a h
ard
com
bina
tiona
l opt
imiz
atio
n pr
oble
m
•Ta
ckle
this
pro
blem
und
er th
e fin
ite N
ewto
n op
timiz
atio
n fra
mew
ork
34D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Fini
te N
ewto
n Fo
rmul
atio
n
•An
aug
men
ted
vect
or:
•Th
en,
whe
re (u
, v) a
re th
e co
ordi
nate
s of
m1.
Re-
writ
e th
e er
ror t
erm
as
whe
re q
is th
e nu
mbe
r of o
utlie
rs
35D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Fini
te N
ewto
n Fo
rmul
atio
n
•D
efin
e ve
ctor
b a
s:
•M
atrix
A (N
by
N):
•E
nerg
y fu
nctio
n
36D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Fini
te N
ewto
n Fo
rmul
atio
n
•Th
e fin
ite g
radi
ent o
f the
ene
rgy
func
tion
E w
ith
resp
ect t
o s
•H
essi
an
•R
ewrit
e gr
adie
nt
37D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Fini
te N
ewto
n Fo
rmul
atio
n
•U
pdat
e eq
uatio
n:•
Set
r =
1, a
nd o
btai
n th
e lin
ear e
quat
ion
38D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Opt
imiz
atio
n
•C
oars
e-to
-fine
sch
eme
to d
eal w
ith la
rge
outli
ers
–Su
ppor
t
of ro
bust
est
imat
or is
pro
gres
sive
ly d
ecay
ed
at a
con
stan
t rat
e–
is k
ept c
onst
ant
–R
esul
t is
used
as
the
initi
al s
tate
for n
ext m
inim
izat
ion
–S
top
whe
n
is
clo
se to
the
expe
cted
pre
cisi
on–
Rep
ort a
suc
cess
ful d
etec
tion
whe
n th
e nu
mbe
r of i
nlie
rm
atch
es is
abo
ve a
giv
en th
resh
old.
39D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Opt
imiz
atio
n
•S
tart
from
a s
uffic
ient
ly la
rge
supp
ort
–To
avo
id g
ettin
g st
uck
at lo
cal m
inim
a–
Nee
ds a
few
iter
atio
ns to
com
pens
ate
for t
he e
rror
s du
e to
pos
e va
riatio
ns•
Mod
ified
RA
NS
AC
–C
lose
d-fo
rm s
olut
ion
–D
raw
from
pro
gres
sive
ly la
rger
set
s of
top-
rank
ed
corre
spon
denc
es–
Sam
ple
size
40D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Expe
rimen
tal S
etup
–In
ord
er to
regi
ster
the
mes
h m
odel
con
veni
ently
, a m
odel
im
age
is a
cqui
red
whe
n th
e no
nrig
idsu
rface
con
tain
s no
de
form
atio
n.–
A ra
ndom
-tree
s ba
sed
met
hod
[Lep
etit
PA
MI’0
6] is
use
d to
bu
ild th
e co
rres
pond
ence
s be
twee
n th
e m
odel
imag
e an
d in
put
imag
e.–
A s
et o
f syn
thet
ic d
ata
is u
sed
to s
elec
t the
par
amet
ers
–Th
e be
st re
gula
rizat
ion
coef
ficie
nt is
foun
d by
grid
sea
rchi
ng.
–n
= 4,
dec
ay ra
te is
0.5
–P
entiu
m-4
3.0
GH
z P
C w
ith 1
GB
RA
M–
A D
V c
amer
a si
ze o
f 720
x576
41D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Com
puta
tiona
l Effi
cien
cy
•Th
e co
mpl
exity
of t
he p
ropo
sed
met
hod
is m
ainl
y de
term
ined
by
the
num
ber o
f ver
tices
N•
Ano
ther
fact
or is
the
num
ber o
f inl
ierm
atch
es•
Pro
pose
d m
etho
d: 8
iter
atio
ns, a
roun
d 18
fram
es p
er s
econ
d•
Sem
i-im
plic
it m
etho
d [C
VP
R’0
5, IJ
CV
’07]
: 40
itera
tions
, ab
out 9
fram
es p
er s
econ
d on
Cof
fee
mat
vid
eo
42D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Cof
fee
mat
[Vid
eo d
emo]
43D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Re-
text
urin
g a
T-sh
irt [V
ideo
dem
o]
44D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Pape
r [Vi
deo
dem
o]
45D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Med
ical
imag
e
46D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Sum
mar
y
•A
nov
el p
rogr
essi
ve s
chem
e to
sol
ve th
e no
n-rig
id s
urfa
ce
dete
ctio
n pr
oble
m•
In c
ontra
st to
the
prev
ious
sem
i-exp
licit
met
hod,
we
dire
ctly
sol
ve
the
unco
nstra
ined
qua
drat
ic o
ptim
izat
ion
prob
lem
by
an e
ffici
ent
fact
oriz
atio
n •
Mod
ified
RA
NS
AC
sch
eme
can
hand
le h
igh-
dim
ensi
onal
spa
ces
with
noi
sy d
ata.
•W
e ha
ve c
ondu
cted
ext
ensi
ve e
xper
imen
tal e
valu
atio
ns o
n di
vers
e ob
ject
s w
ith d
iffer
ent m
ater
ials
.•
The
prop
osed
met
hod
is v
ery
fast
and
robu
st, a
nd c
an h
andl
e la
rge
defo
rmat
ions
and
illu
min
atio
n ch
ange
s.
47D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Con
clus
ion
•Tw
o di
ffere
nt a
ppro
ache
s to
non
rigid
surfa
ce re
cove
ry h
ave
been
inve
stig
ated
•A
ppea
ranc
e-ba
sed
met
hod:
–U
sing
mor
e in
form
atio
n, m
ay b
e m
ore
accu
rate
–O
fferin
g im
age
codi
ng c
apab
ility
–Te
nd to
be
com
puta
tiona
l exp
ensi
ve–
Eas
y to
stu
ck a
t the
loca
l opt
ima
•Fe
atur
e-ba
sed
met
hod
–A
utom
atic
sol
utio
n–
Ess
entia
lly fa
st–
Taki
ng a
dvan
tage
of t
he a
dvan
ces
in fe
atur
e m
atch
ing
and
obje
ct
reco
gniti
on–
Err
ors
occu
r in
the
regi
on la
ckin
g te
xtur
e
48D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Futu
re w
ork
•Fu
sion
app
roac
h
•E
xplo
re n
ew re
gula
rizat
ion
met
hod
•Fi
nd fa
st a
nd a
ccur
ate
feat
ure
mat
chin
g al
gorit
hm
•In
clud
ing
mor
e fe
atur
es, s
uch
as e
dges
and
si
lhou
ette
49D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
The
End
Tha
nk y
ou
50D
epar
tmen
t of C
ompu
ter S
cien
ce a
nd E
ngin
eerin
g,
Facu
lty o
f Eng
inee
ring,
Th
e C
hine
se U
nive
rsity
of H
ong
Kong
Ref
eren
ces
•T.
Coo
tes,
G. E
dwar
ds, a
nd C
. Tay
lo. A
ctiv
e ap
pear
ance
mod
els.
IEE
E T
rans
. on
Pat
tern
A
naly
sis
and
Mac
hine
Inte
llige
nce,
23(
6), J
une
2001
.•
J. X
iao,
S. B
aker
, I. M
atth
ews,
and
T. K
anad
e. R
eal-t
ime
com
bine
d 2d
+3d
activ
e ap
pear
ance
mod
els.
In P
roc.
Con
f. C
ompu
ter V
isio
n an
d P
atte
rn R
ecog
nitio
n, v
olum
e 2,
pa
ges
535–
542,
200
4.•
L. V
acch
etti,
V. L
epet
it, a
nd P
. Fua
. Sta
ble
real
-tim
e 3d
trac
king
usi
ng o
nlin
e an
d of
fline
in
form
atio
n. IE
EE
Tra
ns. O
n P
atte
rn A
naly
sis
and
Mac
hine
Inte
llige
nce,
26(
6), O
ct. 2
004.
•J.
Pile
t, V
. Lep
etit,
and
P. F
ua. F
ast n
on-r
igid
sur
face
det
ectio
n, re
gist
ratio
n, a
nd re
alis
tic
augm
enta
tion.
Int’l
J. C
ompu
ter V
isio
n, 2
007.
•J.
Zhu
, S. C
. Hoi
, E. Y
au, a
nd M
. R. L
yu. A
utom
atic
3d
face
mod
elin
g us
ing
2d a
ctiv
e ap
pear
ance
mod
els.
In P
roc.
13th
Pac
ific
Con
f. C
ompu
ter G
raph
ics
and
App
licat
ions
, pag
es
133–
135,
200
5.•
J. Z
hu, S
. C. H
oi, a
nd M
. R. L
yu. R
eal-t
ime
non-
rigid
sha
pe re
cove
ry v
ia a
ctiv
e ap
pear
ance
m
odel
s fo
r aug
men
ted
real
ity. I
n P
roc.
Eur
opea
n C
onf.
Com
pute
r Vis
ion,
pag
es 1
86–1
97,
2006
.•
J. Z
hu a
nd M
. R. L
yu. P
rogr
essi
ve fi
nite
new
ton
appr
oach
to re
al-ti
me
nonr
igid
surfa
ce
dete
ctio
n. In
Pro
c. C
onf.
Com
pute
r Vis
ion
and
Pat
tern
Rec
ogni
tion,
200
7.•
J. Z
hu, S
. C. H
oi, a
nd M
. R. L
yu. A
mul
ti-sc
ale
tikho
nov
regu
lariz
atio
n sc
hem
e fo
r im
plic
it su
rface
mod
ellin
g. In
Pro
c. C
onf.
Com
pute
r Vis
ion
and
Pat
tern
Rec
ogni
tion,
200
7.
