Pyramidal Implementation of Lucas Kanade Feature Tracker Jia Huang Xiaoyan Liu Han Xin Yizhen Tan.
Pyramidal Implementation of Lucas Kanade Feature Tracker
description
Transcript of Pyramidal Implementation of Lucas Kanade Feature Tracker
![Page 1: Pyramidal Implementation of Lucas Kanade Feature Tracker](https://reader036.fdocuments.in/reader036/viewer/2022081420/56814740550346895db47d2a/html5/thumbnails/1.jpg)
Pyramidal Implementation of Lucas Kanade Feature Tracker
Jia HuangXiaoyan Liu
Han XinYizhen Tan
![Page 2: Pyramidal Implementation of Lucas Kanade Feature Tracker](https://reader036.fdocuments.in/reader036/viewer/2022081420/56814740550346895db47d2a/html5/thumbnails/2.jpg)
Abstract
IntroductionTracking algorithm
Lucas-Kanade algorithm Iterative implementation
Tracking features analysis Feature lost Feature selection
![Page 3: Pyramidal Implementation of Lucas Kanade Feature Tracker](https://reader036.fdocuments.in/reader036/viewer/2022081420/56814740550346895db47d2a/html5/thumbnails/3.jpg)
Objective For a given point u in image A, find its
corresponding location v = u + d in image B.
Image A Image B
d
![Page 4: Pyramidal Implementation of Lucas Kanade Feature Tracker](https://reader036.fdocuments.in/reader036/viewer/2022081420/56814740550346895db47d2a/html5/thumbnails/4.jpg)
Residual function and Window size
2( ) ( , ) ( ( , ) ( , ))y yx x
x x x x
u wu w
x y x yx u w y u w
d d d A x y B x d y d
To find the location Minimize residual function:
: Integration window size
Small integration window Higher accuracy
Larger integration window Higher robustness
Nature tradeoff:
,x yw w
![Page 5: Pyramidal Implementation of Lucas Kanade Feature Tracker](https://reader036.fdocuments.in/reader036/viewer/2022081420/56814740550346895db47d2a/html5/thumbnails/5.jpg)
Pyramid Implementation of LK algorithm Calculate a set of pyramid representations of original image Apply traditional tracking algorithm for each level Results of current iteration is propagated to next iteration Key point: the same window size is used for each level
Top View Side View
![Page 6: Pyramidal Implementation of Lucas Kanade Feature Tracker](https://reader036.fdocuments.in/reader036/viewer/2022081420/56814740550346895db47d2a/html5/thumbnails/6.jpg)
Lucas-Kanade algorithm(1)
At the level L, we define images A and B: ( , ) [ 1, 1] [ 1, 1]x x x x y y y yx y p w p w p w p w
( , ) [ , ] [ , ]x x x x y y y yx y p w p w p w p w
( , ) ( , )LA x y I x y
( , ) ( , )L L Lx yB x y J x g y g
2( ) ( , ) ( ( , ) ( , ))y yx x
x x y y
p wp w
x y x yx p w y p w
v v v A x y B x v y v
![Page 7: Pyramidal Implementation of Lucas Kanade Feature Tracker](https://reader036.fdocuments.in/reader036/viewer/2022081420/56814740550346895db47d2a/html5/thumbnails/7.jpg)
Lucas-Kanade algorithm(2) At the optimum, the first derivative of
After first order Taylor expansion
Components in the equation above
( )| [0 0]
optv v
v
v
,
( )( ) ( ( , ) ( , ) )
y yx x
x x y y
p wp w
x p w y p w
v B B B Bv A x y B x y v
v x y x y
( , ) ( , ) ( , )I x y A x y B x y T
x
y
I B BI
I x y
![Page 8: Pyramidal Implementation of Lucas Kanade Feature Tracker](https://reader036.fdocuments.in/reader036/viewer/2022081420/56814740550346895db47d2a/html5/thumbnails/8.jpg)
2
2
1 ( )
2
y yx x
x x y y
T p wp wxx x y
x p w y p w yx y y
I II I Ivv
I II I Iv
Lucas-Kanade algorithm(3) Two derivative images are expressed:
With these notation, we can get:
The optimum optical flow vector is ( , ) ( 1, ) ( 1. )( , )
2x
A x y A x y A x yI x y
x
( , ) ( , 1) ( . 1)
( , )2y
A x y A x y A x yI x y
y
1optv G b
bG
![Page 9: Pyramidal Implementation of Lucas Kanade Feature Tracker](https://reader036.fdocuments.in/reader036/viewer/2022081420/56814740550346895db47d2a/html5/thumbnails/9.jpg)
Pyramidal diagram
Inner loop: K-level K initialized to 1, assume that the previous
computations from iterations 1,2,...,k-1 provide an initial guess
The new translated image according to
Iterative scheme of LK algorithm(1)
. , 3me g L
0mL
1 1 1[ ]k k k Tx yv v v
1kv
( , ) [ , ] [ , ]x x x x y y y yx y p w p w p w p w 1 1( , ) ( , )k k
k x yB x y B x v y v
![Page 10: Pyramidal Implementation of Lucas Kanade Feature Tracker](https://reader036.fdocuments.in/reader036/viewer/2022081420/56814740550346895db47d2a/html5/thumbnails/10.jpg)
Iterative scheme of LK algorithm(2) The goal: to compute the residual pixel motion vector
, that minimizes the error function
Image mismatch vector , where the image difference delta I k defined as:
New pixel displacement guess is computed for the next iteration step k+1:
[ ]k k kx y
2( ) ( , ) ( ( , ) ( , ))y yx x
x x y y
p wp wk k k kx y k x y
x p w y p w
A x y B x y
( , ) ( , )
( , ) ( , )
y yx x
x x y y
p wp wk x
k
x p w y p w k y
I x y I x yb
I x y I x y
( , ) ( , ) ( , )k kI x y A x y B x y
kbthk
1k k kv v
![Page 11: Pyramidal Implementation of Lucas Kanade Feature Tracker](https://reader036.fdocuments.in/reader036/viewer/2022081420/56814740550346895db47d2a/html5/thumbnails/11.jpg)
Iterative scheme of LK algorithm(3)
On average, 5 iterations are enough At the 1st iteration (k=1), the initial guess is set
to zero
The final solution for the optical flow vector is
Outer loop: L-levelThe vector d is propagated to the next level
L-1 and overall procedure is repeated L-1, L-2, …, 0
1
KL K k
k
v d v
0 [0 0]T
![Page 12: Pyramidal Implementation of Lucas Kanade Feature Tracker](https://reader036.fdocuments.in/reader036/viewer/2022081420/56814740550346895db47d2a/html5/thumbnails/12.jpg)
Declaring a Feature Lost
Several cases of lost feature the point falls outside of the image image patch around the tracked point varies
too much between image A and image B too large displacement
How to solve it combine a traditional tracking approach with an affine image matching
![Page 13: Pyramidal Implementation of Lucas Kanade Feature Tracker](https://reader036.fdocuments.in/reader036/viewer/2022081420/56814740550346895db47d2a/html5/thumbnails/13.jpg)
Feature Lost Example(1)
Image A Image B
![Page 14: Pyramidal Implementation of Lucas Kanade Feature Tracker](https://reader036.fdocuments.in/reader036/viewer/2022081420/56814740550346895db47d2a/html5/thumbnails/14.jpg)
Feature Lost Example(2)
Image A Image B
![Page 15: Pyramidal Implementation of Lucas Kanade Feature Tracker](https://reader036.fdocuments.in/reader036/viewer/2022081420/56814740550346895db47d2a/html5/thumbnails/15.jpg)
Feature Selection
Intuitive To select the point u on image A good to track.
Process steps: Compute the G matrix and λm
Call λmax the maximum value of λm Retain the pixels that have a λm value larger than a percentage of λmax Retain the local max. pixels Keep the subset of those pixels so that the minimum distance between pixels is larger than a threshold
![Page 16: Pyramidal Implementation of Lucas Kanade Feature Tracker](https://reader036.fdocuments.in/reader036/viewer/2022081420/56814740550346895db47d2a/html5/thumbnails/16.jpg)
Example of LK Feature Tracking
Image A Image B
![Page 17: Pyramidal Implementation of Lucas Kanade Feature Tracker](https://reader036.fdocuments.in/reader036/viewer/2022081420/56814740550346895db47d2a/html5/thumbnails/17.jpg)
More Examples
Image BImage A
![Page 18: Pyramidal Implementation of Lucas Kanade Feature Tracker](https://reader036.fdocuments.in/reader036/viewer/2022081420/56814740550346895db47d2a/html5/thumbnails/18.jpg)
Summary
Lucas-Kanade Feature Tracker is one of the most popular versions of two-frame differential methods for motion estimation
Iterative implementation of the Lucas-Kanade optical flow computation provides sufficient local tracking accuracy.
![Page 19: Pyramidal Implementation of Lucas Kanade Feature Tracker](https://reader036.fdocuments.in/reader036/viewer/2022081420/56814740550346895db47d2a/html5/thumbnails/19.jpg)
Thanks for your attention
Any question?