Lecture25 Tracking
-
Upload
waqas-ahmed -
Category
Documents
-
view
223 -
download
0
Transcript of Lecture25 Tracking
-
8/2/2019 Lecture25 Tracking
1/26
Tracking
-
8/2/2019 Lecture25 Tracking
2/26
Definition of Tracking
Tracking: Generate some conclusions about the motion of
the scene, objects, or the camera, given a
sequence of images.
Knowing this motion, predict where things are
going to project in the next image, so that wedont have so much work looking for them.
-
8/2/2019 Lecture25 Tracking
3/26
Why Track?
Detection andrecognition are expensive
If we get an idea ofwhere an object is in
the image because we
have an idea of the
motion from previous
images, we need lesswork detecting or
recognizing the object.
Scene
Image Sequence
Detection+ recognition
Tracking
(less detection,no recognition) New
person
A
B
C
A B
C
-
8/2/2019 Lecture25 Tracking
4/26
Tracking a Silhouette by
Measuring Edge Positions Observations are positions of edges along normals to tracked contour
-
8/2/2019 Lecture25 Tracking
5/26
Why not Wait and Process the
Set of Images as a Batch? In a car system, detecting and tracking
pedestrians in real time is important.
Recursive methods require less computing
-
8/2/2019 Lecture25 Tracking
6/26
Implicit Assumptions of
Tracking
Physical cameras do not move instantlyfrom a viewpoint to another
Object do not teleport between places
around the scene
Relative position between camera and scene
changes incrementally
We can model motion
-
8/2/2019 Lecture25 Tracking
7/26
Related Fields
Signal Detection and Estimation Radar technology
-
8/2/2019 Lecture25 Tracking
8/26
The Problem: Signal Estimation
We have a system with parameters Scene structure, camera motion, automatic zoom
System state is unknown (hidden)
We have measurements
Components of stable feature points in the
images.
Observations, projections of the state.
We want to recover the state components fromthe observations
-
8/2/2019 Lecture25 Tracking
9/26
Necessary Models
System Dynamics
Model
Measurement
Model (projection)
(u, v)
Previous StateNext State
StateMeasurement
We use models to describe a priori knowledge about
the world (including external parameters of camera)
the imaging projection process
-
8/2/2019 Lecture25 Tracking
10/26
State variable
a
A Simple Example of Estimation
by Least Square Method Goal: Find estimate of state
such that the least squareerror between measurements
and the state is minimum
a
=
=n
i
i axC1
2)(2
1
==
===
n
i
i
n
i
i anxaxaC
11
)(0
==n
i
ixn
a1
1
aMeasurement
x
x
txi
a xi-a
ti
-
8/2/2019 Lecture25 Tracking
11/26
Recursive Least Square Estimation We dont want to wait until
all data have been collected
to get an estimate of thedepth
We dont want to reprocessold data when we make a
new measurement
Recursive method: data at
step i are obtained from
data at step i-
1
a
State variablea
Measurement
x
x
t
a
xi
1 ia ia
-
8/2/2019 Lecture25 Tracking
12/26
Recursive Least Square Estimation 2
Recursive method: data at
step i are obtained fromdata at step i-1
x
t
a
xi
)(1
11 += iiii axiaa
i
i
k
k
i
k
ki xi
xi
xi
a 1111
11
==+==
=
=1
1
1
1
1
i
k
ki x
i
a
iii xi
ai
ia
1
1 1 +
=
-
8/2/2019 Lecture25 Tracking
13/26
Recursive Least Square Estimation 3
)(1 11 += iiii ax
i
aaEstimate at step i
Predicted
measure
Innovation
GainActual
measure
Gain specifies how much
do we pay attention
to the difference
between what we expected
and what we actually get
-
8/2/2019 Lecture25 Tracking
14/26
Least Square Estimation of the
State Vector of a Static System
1. Batch method
a
H1 Hi
H2
x1
xi
x2
aH = iix a
H
H
H
=
nnx
x
x
......
2
1
2
1
aHx =
measurement equation
XHH)(HaT1T =
Find estimate that minimizesa
a)H(Xa)H(XT =
2
1C
We find
-
8/2/2019 Lecture25 Tracking
15/26
Least Square Estimation
of the State Vectorof a Static System 2
2. Recursive method
a
H1 Hi
H2
x1
xi
x2
) -1iiii-1ii aH(XKaa +=
with
Calculation is similar to calculation of running average
Now we find:
We had: )(1
11
+=iiii
axi
aa
1)( =
=T
iii
T
iii
HHP
HPK
Gain matrixInnovation
Predicted measure
-
8/2/2019 Lecture25 Tracking
16/26
Dynamic System
i
i
i
i
AV
X
a
ixwAA
tAVV
tVXX
ii
iii
iii
+=
+=
+=
1
11
11
+
=
wA
VX
tt
A
VX
i
i
i
i
i
i
00
100
1001
1
1
1
State of rocket
Measurement
waa -1ii +=
[ ] V
A
V
X
x
i
i
i
i +
= 001 Vxi
+=i
aH
State equation for rocket
Measurement equation
Noise
Tweak factor
-
8/2/2019 Lecture25 Tracking
17/26
) -1iiiii-1iii aH(xKaa +=
Recursive Least Square
Estimation for a Dynamic System(Kalman Filter)
-1i-1iii waa +=
iiii naHx+=
),0(~ ii Qw N
)R,0(~n ii N
-1i-1i-1i-1i
-1i
T
i-1iii
-1i
Tiii
Tiii
)P'HK(IP
QPP'
RHP'HHP'K
=
+=
+= )(
State equation
Measurement equation
Tweak factor for model
Measurement noise
Prediction forxi
Prediction for ai
Gain
Covariance matrix for
prediction error
Covariance for estimation error
-
8/2/2019 Lecture25 Tracking
18/26
-1i-1i-1i-1i
-1i
T
i
-1i
i
i
-1
i
T
iii
T
iii
)P'HKIP
QafP
afP'
RHP'HHP'K
=
+
=
+=
(
)(
)))( -1iiii-1ii af(H(xKafa +=
Estimation when System Model
is Nonlinear(Extended Kalman Filter)
-1i-1ii wafa += )( iiii vaHx +=State equation Measurement equation
Jacobian Matrix
Differences compared toregular Kalman filter are
circled in red
-
8/2/2019 Lecture25 Tracking
19/26
Predict next state as using
previous step and dynamic model
Predict regionsof next measurements using
measurement model anduncertainties
Make new measurements xi in
predicted regions
Compute best estimate of next state
Tracking Steps
(u, v)Measurement
Prediction
region
-1iia
),( i-1iii P'aH N
) -1iiiii-1iii aH(xKaa +=Correction of predicted state
-
8/2/2019 Lecture25 Tracking
20/26
Recursive Least Square Estimation for
a Dynamic System (Kalman Filter)
x
t
Measurementxi
State vectoraiEstimation ia
-
8/2/2019 Lecture25 Tracking
21/26
Tracking as a Probabilistic
Inference Problem
Find distributions for state vectorai and formeasurement vectorxi. Then we are able to
compute the expectations and
Simplifying assumptions (same as for HMM)
ix
)|(),,,|( -1ii-1i21i aaaaaa PP =L
)|()|()|()|,,,( ikijiiiji axaxaxaxx PPPP KK =
(Conditional independence ofmeasurements given a state)
(Only immediate past matters)
ia
-
8/2/2019 Lecture25 Tracking
22/26
Tracking as Inference
Prediction
Correction
Produces same results as least square approach if
distributions are Gaussians: Kalman filter See Forsyth and Ponce, Ch. 19
= -1i-1i1-1i-1ii-1i1i axxaaaxxa dPPP ),,|()|(),,|( LL
=
i-1i1iii
-1i1iiii1i
axxaax
xxaaxxxa
dPP
PPP
),,|()|(
),,|()|(),,|(
L
L
L
-
8/2/2019 Lecture25 Tracking
23/26
)( 11 i-iii-i afhxKafa +=
Kalman Filter for 1D Signals
11 i-i-i wafa +=
iii vahx+=
)0(~ ,qNwi
)0(~ ,rNvi
111
1
2
12
')1(
)(
i-ii-
ii
-iii
phKp
qpfp'
rp'hhp'K
=
+=
+=
State equation
Measurement equation
Tweak factor for model
Measurement noise
Prediction for xi
Prediction for ai (a priori estimate)
Gain
Standard deviation for
prediction errorSt.d. for estimation error
-
8/2/2019 Lecture25 Tracking
24/26
Applications: Structure from Motion
Measurement vector components: Coordinates of corners, salient
points
State vector components:
Camera motion parameters
Scene structure
Is there enough equations?
N corners, 2N measurements
N unknown state components from structure
(distances from first center of projection to3D points)
6 unknown state components from motion
(translation and rotation)
More measurements than unknowns for every
frame if N>6 (2N > N + 6)
xi
Batch methods Recursive methods
(Kalman filter)
-
8/2/2019 Lecture25 Tracking
25/26
Problems with Tracking
Initial detection If it is too slow we will never catch up
If it is fast, why not do detection at every frame?
Even if raw detection can be done in real time, tracking
saves processing cycles compared to raw detection.
The CPU has other things to do.
Detection is needed again if you lose tracking
Most vision tracking prototypes use initial
detection done by hand(see Forsyth and Ponce for discussion)
-
8/2/2019 Lecture25 Tracking
26/26
References Kalman, R.E., A New Approach to Linear Prediction
Problems, Transactions of the ASME--Journal of BasicEngineering, pp. 35-45, March 1960.
Sorenson, H.W., Least Squares Estimation: from Gauss to
Kalman, IEEE Spectrum, vol. 7, pp. 63-68, July 1970. http://www.cs.unc.edu/~welch/kalmanLinks.html
D. Forsyth and J. Ponce. Computer Vision: A Modern
Approach, Chapter 19.http://www.cs.berkeley.edu/~daf/book3chaps.html
O. Faugeras.. Three-Dimensional Computer Vision. MIT
Press. Ch. 8, Tracking Tokens over Time.