Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image...
-
Upload
vernon-robbins -
Category
Documents
-
view
220 -
download
0
Transcript of Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image...
![Page 1: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/1.jpg)
Computer and Robot Vision II
Chapter 15Motion and Surface Structure from
Time Varying Image Sequences
Presented by: 傅楸善 & 王林農0917 533843
[email protected]指導教授 : 傅楸善 博士
![Page 2: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/2.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.1 Introduction
Motion analysis involves estimating the relative motion of objects with respect to each other and the camera given two or more perspective projection images in a time sequence.
![Page 3: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/3.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.1 Introduction (cont’)
Real-world applications:
industrial automation and inspection, robot assembly, autonomous vehicle navigation, biomedical engineering, remote sensing, general 3D-scene understanding
![Page 4: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/4.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.1 Introduction (cont’)
object motion and surface structure recovery from: observed optic flow point correspondences
![Page 5: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/5.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.2 The Fundamental Optic Flow Equation
(x, y, z): 3D point on moving rigid body (u, v): perspective projection on the image pla
ne f: camera constant (u, v): velocity of the point (u, v)
![Page 6: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/6.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.2 The Fundamental Optic Flow Equation (cont’)
take time derivatives of both sides
yields the fundamental optic flow equation:
y
x
z
f
v
u
yzzy
xzzx
z
f
v
u2
z
y
x
v
u
f
f
zv
u 0
0
1
.
.. .. .
.
.
.
.
.
![Page 7: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/7.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.2 The Fundamental Optic Flow Equation (cont’)
general solution: (λ is a free variable)
f
v
u
v
u
f
z
z
y
x
0
.
.
.
.
.
![Page 8: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/8.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.2.1 Translational Motion
Known: N-point optic flow field:
Unknown: corresponding unknown 3D points: all points moving with same but unknown velocity (x, y, z)
can be solved up to a multiplicative constant
Nnnnnn vuvu 1)},,,{(
. .
Nnnnn zyx 1)},,{(
. . .
![Page 9: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/9.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.2.2 Focus of Expansion and Contraction
Known: 3D motion is translational one 2D projected point (u, v) has no motion:
thus translational motion is in a direction along the ray of sight
0vu. .
![Page 10: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/10.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.2.2 Focus of Expansion and Contraction (cont’)
focus of expansion (FOE): if 3D point field moving toward camera
FOE: motion-field vectors radiate outward from that point
focus of contraction (FOC): if 3D point field moving away from camera
FOC: vectors radiate inward toward diametrically opposite point flow pattern of the motion field of a forward-moving observer
![Page 11: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/11.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 12: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/12.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.2.3 Moving Line Segment
Known: fixed distance between two unknown 3D points
translational motion with common velocity
(x, y, z) corresponding optic flow:
),,(),,,( 222111 zyxzyx
. . .
)v,u,v,(u),v,u,v,(u 22221111
. . . .
![Page 13: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/13.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.2.3 Moving Line Segment (cont’)
Unknown: : two unknown 3D points
common velocity: (x, y, z)
),,(),,,( 222111 zyxzyx
. . .
![Page 14: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/14.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.2.4 Optic Flow Acceleration Invariant
Since differentiating general solution in Sec 15.2 and s
olve for (x, y, z)
zf .
.. .. ..
f
v
u
f
zv
u
f
zv
u
f
z
z
y
x
0
2
0
..
..
..
..
..
.
.. .
.
![Page 15: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/15.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.3 Rigid-Body Motion
Rigid-body motion: no relative motion of points w.r.t. one another
Rigid-body motion: points maintain fixed position relative to one another
Rigid-body motion: all points move with the body as a whole
![Page 16: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/16.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.3 Rigid-Body Motion (cont’)
R(t): rotation matrix T(t): translation vector p(0): initial position of given point R(0)=I, T(0)=0 p(t): position of given point at time t
![Page 17: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/17.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.3 Rigid-Body Motion (cont’)
Rigid-body motion in displacement vectors:
velocity vector: time derivative of its position:
)()0()()0( tTptRP
)()0()()0( tTptRP .. .
![Page 18: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/18.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.3 Rigid-Body Motion (cont’)
Since
(a) translational-motion field under projection onto hemispherical surface only translational-component motion useful in determining scene structure
(b) rotational-motion field under projection onto hemispherical surface rotational-motion field provides no information about scene structure
)()()()()()()()(
)]()()[()0(11
1
tTtTtRtRtptRtRtp
yTtptRp
. . .
![Page 19: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/19.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 20: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/20.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.3 Rigid-Body Motion (cont’)
we can describe rigid-body motion in instantaneous velocity by
)()()()( tktpttp .
![Page 21: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/21.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.3 Rigid-Body Motion (cont’)
: angular velocities in three axes : translational velocities in three axes from rigid-body-motion equation
wzx ,,
zyx kkk ,,
zyx
yxz
xzy
z
y
x
z
y
x
kxy
kzx
kyz
k
k
k
z
y
x
k
z
y
x
z
y
x
.
.
.
![Page 22: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/22.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.3 Rigid-Body Motion (cont’)
and perspective projection equation
we can determine an expression for z:
f
v
u
f
z
z
y
x
.
zyx kuvf
zz )(
.
![Page 23: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/23.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.3 Rigid-Body Motion (cont’)
after simplification
z
y
x
z
y
x
k
k
k
u
v
f
uv
f
uf
f
vf
f
uv
z
uz
u
z
fz
f
v
u
22
22
0
0
..
![Page 24: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/24.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.3 Rigid-Body Motion (cont’)
image velocity: expressed as sum of translational field and rotational field
(x, y, z): 3D coordinate before rigid-body motion in displacement vectors
(x’, y’, z’): 3D coordinate after rigid-body motion in displacement vectors
: rotation angles in three axes : translation in three axes
),,( zyx ),,( zyx ttt
![Page 25: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/25.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.3 Rigid-Body Motion (cont’)
Rigid-body motion in displacement vectors:
z
y
x
t
t
t
z
y
x
RRR
z
y
x
yxyxy
zxzyxzxzyxzy
zxzyxzxzyxzy
z
y
x
xyz
coscoscossinsin
sinsincossincoscoscossinsinsinsincos
sinsincossincossincoscossinsincoscos
)()()(
'
'
'
![Page 26: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/26.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.3 Rigid-Body Motion (cont’)
motion in displacement vector and instantaneous velocity is different:
e.g. moon encircling earth instantaneous velocity: first order approximation of
displacement vector first order approximation: when small,
0sinsin,1cos,sin
![Page 27: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/27.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.3 Rigid-Body Motion (cont’)
first order approximation: when time=1 thus x=(x’ - x)/1 first order approximation:
.
zzyyxxzzyyxx tktktk ,,,,,
zxy
yzx
xyz
z
y
x
xy
xz
yz
tyxz
txzy
tzyx
t
t
t
z
y
x
z
y
x
1
1
1
'
'
'
![Page 28: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/28.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
joke
![Page 29: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/29.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.4 Linear Algorithms for Motion and Surface Structure from Optic Flow
15.4.1 The Planar Patch Case : arbitrary object point on planar
patch at time t : central projective coordinates of p(t) o
nto image plane z= f
)]'(),(),([)( tztytxtp
)](),([ tvtu
]'),(),([)(
)(
)(
)()(
)(
)()(
ftvtuf
tztp
tz
tyftv
tz
txftu
![Page 30: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/30.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.4.1 The Planar Patch Case
: instantaneous velocity of moving image point
: optic flow image point : instantaneous rotational angular
velocity : instantaneous translational velocity
)](),([ tvtu)](),([ tvtu
. .
)](),(),(),([ tvtutvtu . .
)(t
)(tk
![Page 31: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/31.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.4.1 The Planar Patch Case (cont’)
unit vector n(t): orthogonal to moving planar patch rigid planar patch motion represented by rigid-moti
on constraint:
1)()'(
)()()()(
tptn
tktptwtp.
![Page 32: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/32.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.4.1 The Planar Patch Case (cont’)
from above two equations:
Let
Rigid-motion constraint could be written as
pknpknpp
'
pknpwp
)'('
0
0
0
,,
'
12
13
23
321
![Page 33: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/33.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.4.1 The Planar Patch Case (cont’)
denote the 3 x 3 matrix by W and its three row vectors by
W: called planar motion parameter matrix since skew symmetric
'kn',',' 321
''' nkknWW
![Page 34: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/34.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.4.1 The Planar Patch Case (cont’)
above equation can be written as
from perspective projection equations:
taking time derivatives of these equations we have
vzzv
uzzu
fy
x
v
u
f
z
y
x
z
y
x
W
z
y
x
1..
.
...
.
.
.
![Page 35: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/35.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.4.1 The Planar Patch Case (cont’)
substitute equations into above equations:
from third row
substitute z to obtain optical flow-planar motion equation
v
u
f
v
u
v
u
f
f
v
u
f
zz
f
v
u
z
fz
vzzv
uzzu
32
1
3
3
2
1
'1
'
'
'
'
'
'
. .
. ..
.
.
.
.
![Page 36: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/36.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.4.1 The Planar Patch Case (cont’)
we have 2N linear equations: n=1,…,N:
optic flow-planar motion recovery: first solve W then find
n
nn
n
n
n
v
u
f
v
u
v
u
f 32
1 '1
'
'
..
.
nk,,
![Page 37: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/37.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.4.2 General Case Optic Flow-Motion Equation
1. set up optic flow-motion equation not involving depth information
2. solve it by using linear least-squares technique
![Page 38: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/38.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.4.3 A Linear Algorithm for Solving Optic FlowMotion Equations
![Page 39: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/39.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.5.4 Mode of Motion, Direction of Translation, and Surface Structure
mode of motion: whether translation k=0 or not direction of translation: direction of k surface structure: relative depth when k 0
![Page 40: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/40.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.4.5 Linear Optic Flow-Motion Algorithm and Simulation Results
motion and shape recovery algorithms should answer three questions:
minimum number of points to compute motion and shape
what set of optic flow points violate rank assumption e.g. collinearity…
What’s the accuracy of estimated motion from noisy optic flow?
![Page 41: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/41.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
joke
![Page 42: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/42.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.5 The Two View-Linear Motion Algorithm
![Page 43: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/43.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.5.1 Planar Patch Motion Recovery from Two Perspective Views: A Brief Review
Two View-Planar Motion Equation imaging geometry for two view-planar motion rigid planar patch in motion in half-space z< 0
![Page 44: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/44.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 45: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/45.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.5.1 Planar Patch Motion Recovery from Two Perspective Views: A Brief Review (cont’)
: arbitrary object point before motion : same object point after motion : central projective coordinates of
f : camera constant
),(),,(
)',,(
)',,(
2211
2222
1111
vuvu
zyxp
zyxp
21, pp
2
22
2
22
1
11
1
11
,
,
zyfvz
xfu
zyfvz
xfu
![Page 46: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/46.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.5.1 Planar Patch Motion Recovery from Two Perspective Views: A Brief Review (cont’)
R0: 3 X 3 rotational matrix, R0’R0=I,|R0|=1
t0: 3 X 1 translational vector
n0: 3 X 1 normal vector
![Page 47: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/47.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.5.1 Planar Patch Motion Recovery from Two Perspective Views: A Brief Review (cont’)
Rigid-body-motion equation relates p1 to p2 as follows:
planarity constrains p1 by
combining two equations produces planar rigid-body-motion-equation
10002
10
0102
)'(
1'
pntRp
pn
tpRp
![Page 48: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/48.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.5.2 General Curved Patch Motion Recovery from Two Perspective Views A Simplified Linear Algorithm
discard planar patch assumption, consider general curved patch
![Page 49: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/49.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.5.3 Determining Translational Orientation
![Page 50: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/50.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.5.4 Determining Mode of Motion and Relative Depths
![Page 51: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/51.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.5.5 A Simplified Two View-Motion Linear Algorithm
![Page 52: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/52.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.5.6 Discussion and Summary
when no noise appears: algorithm extremely accurate
when small noise appears: it works well except mode of motion incorrect
![Page 53: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/53.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.6 Linear Algorithm for Motion and Structure from Three Orthographic Views
Ullman (1979) showed that for the orthographic case four-point correspondences over three views are sufficient to determine the motion and structure of the four-point rigid configuration
![Page 54: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/54.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Shimon Ullman, The Interpretation of Visual Motion
The MIT Press Cambridge MA. 1979
![Page 55: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/55.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.6 Linear Algorithm for Motion and Structure from Three Orthographic Views
to infer depth information: translation needed in perspective projection
to infer depth information: rotation useless in perspective projection
to infer depth information: rotation needed in orthographic projection
to infer depth information translation useless in orthographic projection
![Page 56: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/56.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.6.1 Problem Formulation
image plane stationary three orthographic views at time
(x, y, z): object-space coordinates of point P at t1
(x’, y’, z’): object-space coordinates of point P at t2
(x”, y”, z”): object-space coordinates of point P at t3
(u, v): image-space coordinates of P at t1
(u’, v’): image-space coordinates of P at t2
(u”, v”): image-space coordinates of P at t3
321 ,, ttt
![Page 57: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/57.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.6.1 Problem Formulation (cont’)
: rotation matrix : translation vector
(x’, y’, z’)’ = R(x’, y’, z’)+Tr
(x”, y”, z”)” = S(x”, y”, z”)+Ts
1313
3333
)(,)(
)(,)(
sisrir
ijij
sTtT
sSrR
![Page 58: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/58.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.6.1 Problem Formulation (cont’)
Known: four image-point correspondences
Unkown:
4,3,2,1),,,(
),(),,(
4,3,2,1),","()','(),(
izyx
TSTR
iuuuuuu
iii
sr
iiiiii
![Page 59: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/59.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.6.1 Problem Formulation (cont’)
note that with orthographic projections
therefore it is obvious that tr3, ts3 can never be determined
we are trying to determine:
)","()","(
)','()','(
),(),(
yxvu
yxvu
yxvu
4,3,2,,2,1,,,, 1 izzittSR isiri
![Page 60: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/60.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.6.2 Determining ),,,(),,,,(,, 32312313323123133333 ssssrrrrsr
![Page 61: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/61.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.6.3 Solving a Unique Orthonormal Matrix R
![Page 62: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/62.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.6.4 Linear Algorithm to Uniquely Solve R, s, a3
![Page 63: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/63.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.6.5 Summary
Given two orthographic views, one cannot finitely determine the motion and structure of a rigid body, no matter how many point correspondences are used, as shown by Huang.
![Page 64: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/64.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.7 Developing a Highly Robust Estimator for General Regression
![Page 65: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/65.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.7.1 Inability of the Classical Robust M-Estimator to Render High Robustness
Classical robust estimator, such as M-, L-, or R-estimator:
1. optimal or nearly optimal at assumed noise distribution
2. relatively small performance degradation with small number of outliers
3. larger deviations from assumed distribution do not cause catastrophe
MF-estimator with new property much stronger than property 3
relatively small performance degradation with larger deviations from assumed distribution
![Page 66: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/66.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.7.2 Partially Modeling Log Likelihood Function by Using Heuristics
MF-estimator: combine Bayes statistical decision rule with heuristics
MF-estimator: robust regression more appropriate model-fitting
![Page 67: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/67.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.7.3 Discussion
M-, L-, R and MF-estimator: all residual based
![Page 68: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/68.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.7.4 MF-Estimator
![Page 69: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/69.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.8 Optic Flow-Instantaneous Rigid-Motion Segmentation and Estimation
formulate optic flow-single rigid-motion estimation into general regression
![Page 70: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/70.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.8.1 Single Rigid Motion
![Page 71: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/71.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.8.2 Multiple Rigid Motions
![Page 72: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/72.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
joke
![Page 73: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/73.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.9 Experimental Protocol
![Page 74: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/74.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.9.1 Simplest Location Estimation
![Page 75: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/75.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.9.2 Optic Flow-Rigid-Motion Segmentation and Estimation
![Page 76: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/76.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.10 Motion and Surface Structure from Line Correspondences
![Page 77: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/77.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.10.1 Problem Formulation
Cartesian reference system-central projection
![Page 78: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/78.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.10.1 Problem Formulation (cont’)
![Page 79: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/79.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.10.1 Problem Formulation (cont’)
l: line in D space L: projection of the line on image plane z = f z = f : image frame : known plane line L is in; projective plane of l : set of lines in 3D space : lines moved by rigid motion (R’ , T’)’ at time t’ : lines moved by rigid motion (R” , T”)” at time t”
},...,{
},...,{
},...,{
""1
''1
1
k
k
k
ll
ll
ll
![Page 80: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/80.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.10.1 Problem Formulation (cont’)
: projections of lines
; respective projective planes
}"{}{}{
}{}{}{
k1k1k1
k1k1k1
",...,,',...,',,...,
L",...,L",L',...,L',L,...,L
![Page 81: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/81.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.10.1 Problem Formulation (cont’)
Known: K triples of line correspondences in three views
Unkown: rotations and translations: 3D lines
kiLLL iii ,...,1,"'
",',",' TTRR
Kili ,...,1,
![Page 82: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/82.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.10.2 Solving Rotation Matrices R’, R” and Translations T’,R”
![Page 83: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/83.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.10.3 Solving Three-Dimensional Line Structure
![Page 84: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/84.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.11 Multiple Rigid Motions from Two Perspective Views
15.11.1 Problem Statement imaging geometry for two-view-motion
![Page 85: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/85.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 86: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/86.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.11.1 Problem Statement
How many good point correspondences are needed in order to apply the nonlinear least-squares estimator?
![Page 87: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/87.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.11.2 Simulated Experiments
![Page 88: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/88.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.12 Rigid Motion from Three Orthographic Views
![Page 89: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/89.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.12.1 Problem Formulation and Algorithm
same as Sec. 15.6, instead of linear algorithms, formulate model-fitting problem
![Page 90: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/90.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.12.2 Simulated Experiments
![Page 91: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/91.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.12.3 Further Research on the MF-Estimator
two problems to be solved for MF-estimator to be practically useful:
distance problem requirement for a good initial approximation
![Page 92: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/92.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
difficulty of motion and shape recovery: ambiguity of displacement field
Fuh. Ph.D. Thesis, Fig 4.1
![Page 93: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/93.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 94: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/94.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.13 Literature Review
15.13.1 Inferring Motion and Surface Structure
![Page 95: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/95.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.13.1 Inferring Motion and Surface Structure
classifications for methods of inferring 3D motion and shape use of individual sets of feature points use of local optic flow information about a single p
oint use of the entire optic flow field
![Page 96: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/96.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.13.1 Inferring Motion and Surface Structure
Despite all the results obtained over the years, almost none of these inference techniques have been successfully applied to feature-point correspondences calculated from real imagery
![Page 97: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/97.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.13.2 Computing Optic Flow or Image-Point Correspondences
problem source contains abundant information occlusion boundaries specular points near a focus of expansion noise and digitization effects in image formation
![Page 98: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/98.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.13.2 Computing Optic Flow or Image-Point Correspondences (cont’)
motion parallax: apparent relative motion between objects and observer
points in observer’s direction of translation remain relatively unchanged information available to a moving observer
![Page 99: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/99.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 100: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/100.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.13.2 Computing Optic Flow or Image-Point Correspondences (cont’)
impart time dimension to image data spatiotemporal image data block
![Page 101: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/101.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 102: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/102.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.13.2 Computing Optic Flow or Image-Point Correspondences (cont’)
motion field: assignment of vectors to image points representing motion
angular velocity of fixed scene: inversely proportional to distance pilot in straight-ahead level flight on an overcast day
![Page 103: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/103.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 104: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/104.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 105: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/105.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.13.2 Computing Optic Flow or Image-Point Correspondences (cont’)
motion field of pilot looking straight ahead in motion direction
zero image velocity: at approach point and at infinity (along horizon)
![Page 106: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/106.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 107: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/107.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.13.2 Computing Optic Flow or Image-Point Correspondences (cont’)
motion field of pilot looking to the right in level flight
focus of expansion here: at infinity to the left focus of contraction here: at infinity to the righ
t of the figure
![Page 108: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/108.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 109: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/109.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
15.13.2 Computing Optic Flow or Image-Point Correspondences (cont’)
spatiotemporal image data acquired by a camera,- caption -
straight streaks at block top due to translating parallel to image plane
![Page 110: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/110.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 111: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/111.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
B.K.P, Horn, Robot Vision, The MIT Press, Cambridge, MA, 1986
Chapter 12 Motion Field & Optical Flow optic flow: apparent motion of brightness patt
erns during relative motion
![Page 112: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/112.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.1 Motion Field
motion field: assigns velocity vector to each point in the image
Po: some point on the surface of an object
Pi: corresponding point in the image
vo: object point velocity relative to camera
vi: motion in corresponding image point
![Page 113: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/113.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.1 Motion Field (cont’)
ri: distance between perspectivity center and image point
ro: distance between perspectivity center and object point
f’: camera constant z: depth axis, optic axis object point displacement causes correspondi
ng image point displacement
![Page 114: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/114.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.1 Motion Field (cont’)
![Page 115: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/115.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.1 Motion Field (cont’)
Velocities:
where ro and ri are related by
![Page 116: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/116.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.1 Motion Field (cont’)
differentiation of this perspective projection equation yields
![Page 117: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/117.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
joke
![Page 118: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/118.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.2 Optical Flow
optical flow need not always correspond to the motion field
(a) perfectly uniform sphere rotating under constant illumination:
no optical flow, yet nonzero motion field (b) fixed sphere illuminated by moving light
source: nonzero optical flow, yet zero motion field
![Page 119: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/119.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 120: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/120.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.2 Optical Flow (cont’)
not easy to decide which P’ on contour C’ corresponds to P on C
![Page 121: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/121.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 122: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/122.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.2 Optical Flow (cont’)
optical flow: not uniquely determined by local information in changing
irradiance at time t at image point (x, y)
components of optical flow vector
![Page 123: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/123.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.2 Optical Flow (cont’)
assumption: irradiance the same at time
fact: motion field continuous almost everywhere
![Page 124: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/124.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.2 Optical Flow (cont’)
expand above equation in Taylor series
e: second- and higher-order terms in cancelling E( x, y, t), dividing through by
![Page 125: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/125.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.2 Optical Flow (cont’)
which is actually just the expansion of the equation
abbreviations:
![Page 126: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/126.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.2 Optical Flow (cont’)
we obtain optical flow constraint equation:
flow velocity (u, v): lies along straight line perpendicular to intensity gradient
![Page 127: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/127.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 128: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/128.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.2 Optical Flow (cont’)
rewrite constraint equation:
aperture problem: cannot determine optical flow along isobrightness contour
![Page 129: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/129.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.3 Smoothness of the Optical Flow
motion field: usually varies smoothly in most parts of image
try to minimize a measure of departure from smoothness
![Page 130: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/130.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.3 Smoothness of the Optical Flow (cont’)
error in optical flow constraint equation should be small
overall, to minimize
![Page 131: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/131.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.3 Smoothness of the Optical Flow (cont’)
large if brightness measurements are accurate
small if brightness measurements are noisy
![Page 132: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/132.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.4 Filling in Optical Flow Information
regions of uniform brightness: optical flow velocity cannot be found locally
brightness corners: reliable information is available
![Page 133: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/133.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.5 Boundary Conditions
Well-posed problem: solution exists and is unique
partial differential equation: infinite number of solution unless with boundary
![Page 134: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/134.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.6 The Discrete Case
first partial derivatives of u, v: can be estimated using difference
![Page 135: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/135.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 136: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/136.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.6 The Discrete Case (cont’)
measure of departure from smoothness:
error in optical flow constraint equation:
to seek set of values that minimize
![Page 137: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/137.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.6 The Discrete Case (cont’)
dieffrentiating e with respect to
![Page 138: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/138.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.6 The Discrete Case (cont’)
where are local average of u, v (9 neighbors? )
extremum occurs where the above derivatives of e are zero:
![Page 139: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/139.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.6 The Discrete Case (cont’)
determinant of 2x2 coefficient matrix:
so that
![Page 140: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/140.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.6 The Discrete Case (cont’)
suggests iterative scheme such as
new value of (u, v): average of surrounding values minus adjustment
![Page 141: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/141.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 142: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/142.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.6 The Discrete Case (cont’)
first derivatives estimated using first differences in 2x2x2 cube
![Page 143: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/143.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 144: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/144.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.6 The Discrete Case (cont’) consistent estimates of three first partial deriv
atives:
![Page 145: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/145.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.6 The Discrete Case (cont’)
four successive synthetic images of rotating sphere
![Page 146: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/146.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 147: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/147.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.6 The Discrete Case (cont’)
estimated optical flow after 1, 4, 16, and 64 iterations
![Page 148: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/148.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 149: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/149.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.6 The Discrete Case (cont’)
(a) estimated optical flow after several more iterations
(b) computed motion field
![Page 150: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/150.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 151: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/151.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
12.7 Discontinuities in Optical Flow
discontinuities in optical flow: on silhouettes where occlusion occurs
![Page 152: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/152.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 153: Computer and Robot Vision II Chapter 15 Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw.](https://reader037.fdocuments.in/reader037/viewer/2022102909/56649ee55503460f94bf4e46/html5/thumbnails/153.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Project due April 18
implementing Horn & Schunck optical flow estimation as above
synthetically translate lena.im one pixel to the right and downward
Try 10 1, 0.1, of