Organizing Optic Flow
18
Organizing Optic Flow Cmput 610 Martin Jagersand
description
Organizing Optic Flow. Cmput 610 Martin Jagersand. Last lecture: Questions to think about. Compare the methods in the paper and lecture Any major differences? How dense flow can be estimated (how many flow vectore/area unit)? How dense in time do we need to sample?. - PowerPoint PPT Presentation
Transcript of Organizing Optic Flow
Organizing Optic Flow
Cmput 610
Martin Jagersand
Last lecture:Questions to think about
Compare the methods in the paper and lecture
1. Any major differences?
2. How dense flow can be estimated (how many flow vectore/area unit)?
3. How dense in time do we need to sample?
Organizing different kinds of motion
Two examples:
1. Greg Hager paper: Planar motion
2. Mike Black, et al: Attempt to find a low dimensional subspace for complex motion
Remember last lecture:
Over determined equation system
Im = Mu Can be solved in e.g. least squares sense
using matlab u = M\Im
...à
@t@Im
...
0
@
1
A =
......
@x@Im
@y@Im
......
0
@
1
A î xî y
ò ó
3-6D Optic flow
Generalize to many freedooms (DOFs)
Im = Mu
Know what type of motion(Greg Hager, Peter Belhumeur)
u’i = A ui + dPlanar Object => Affine motion model:
It = g(pt, I0)
Mathematical Formulation
Define a “warped image” g– f(p,x) = x’ (warping function), p warp parameters– I(x,t) (image a location x at time t)– g(p,It) = (I(f(p,x1),t), I(f(p,x2),t), … I(f(p,xN),t))’
Define the Jacobian of warping function
– M(p,t) =
Consider “Incremental Least Squares” formulation– O(p, t+t) = || g(pt,It+t) – g(0,I0) ||2
@p@gh i
Model– I0 = g(pt, It ) (image I, variation model g, parameters p)– I = M(pt, It) p (local linearization M)
Define an error
– et+1 = g(pt, It+ ) - I0
Close the loop
– pt+1 = pt - (MT M)-1 MT et+1 where M = M(pt,It)
Estimating motion parameters
M is N x m and is time varying!
A Factoring Result
Suppose I = g(It, p) at pixel location u is defined as I(u) = I(f(p,u),t)
And = L(u)S(p)
Then
M(p,It) = M0 S(p) where M0 = M(0,I0)
@u@fà ãà 1
@p@f
Numerical Solution
G is m x N, e is N x 1S is m x m
O(mN)operations
In general, solve– [ST G S] p = M0
T et+1 where G = M0TM0
constant!– pt+1 = pt + p
If S is invertible, then– pt+1 = pt - S-T G et+1 where G = (M0
TM0)-1M0T
Numerical Solution
In general, solve– [ST G S] p = M0
T et+1 where G = M0TM0
constant!– pt+1 = pt + p
If S is invertible, then– pt+1 = pt - S-T G et+1 where G = (M0
TM0)-1M0T
G is constant!
S is small and time varyingLocal asymptotic
stability!
In general, solve
– [ST G S] p = M0T et+1 where G = M0
TM0 constant!
– pt+1 = pt + p
If S is invertible, then–
pt+1 = pt - S-T G et+1 where G = (M0TM0)-1M0
T
Stabilization Revisited
G is constant!
S is small and time varyingLocal asymptotic
stability!
On The Structure of M
u’i = A ui + dPlanar Object -> Affine motion model:
X Y Rotation Scale Aspect Shear
M(p) = @g=@p
Organizing flowfields
Express flow field f in subspace basis m
Different “mixing” coefficients a correspond to different motions
Example:Image discontinuities
Mathematical formulation
Let:
Mimimize objective function:
=
Where
ExperimentMoving camera
4x4 pixel patches
Tree in foreground separates well
Experiment:Characterizing lip motion
Very non-rigid!
