Travel Map Amsterdam, Berlin, Copenhagen Europe Amsterdam Berlin Copenhagen.
Visual Tracking by Cluster Analysis Arthur Pece Department of Computer Science University of...
-
Upload
edwin-hoover -
Category
Documents
-
view
216 -
download
0
description
Transcript of Visual Tracking by Cluster Analysis Arthur Pece Department of Computer Science University of...
Visual Tracking by Cluster Analysis
Arthur PeceDepartment of Computer Science
University of [email protected]
Tracking by image differencing
The problem: detecting and tracking moving objects, with no prior knowledge about the objects or the camera, except that the camera is fixed.
Image differencing
[see movie]• Non-zero grey-level differences are all over
the place !• Moving targets generate clusters which are
not compact• There is no well-defined threshold to
discriminate between targets and background
Usual strategy
A sequence of algorithms is applied to the difference image:
• Thresholding• Morphological operators• Connected-component analysis• Feature detection• matching from frame to frame or Kalman
filtering
An alternative: cluster tracking• Find clusters of pixels significantly different
from the reference image• Optimize cluster parameters by the EM
clustering algorithm• Remove, merge or split clusters on the basis
of cluster parametersNone of the operations on the previous slide(except - possibly - Kalman filtering)
Generative model• The camera is looking at an unknown number of
moving objects, each of which generates a cluster of pixels in the image
• Moving objects generate clusters with bivariate Gaussian distributions in image coordinates
• There is no correlation between colors of the moving objects and colors of the background, so grey-levels differences in the moving clusters have a uniform distribution
Background model
• For consistency, the background is also considered as a generator of pixels
• The pixels originating from the background have uniform distribution over the image
• Grey-level differences in the background are likely (but not certain) to be close to zero: a double-exponential distribution gives a good fit
Generative model (re-phrased)• The image is generated by a density mixture, in
which each pixel is a data point in 3D (2 image coordinates plus grey level)
• The background is a cluster with uniform distribution in image coordinates and Laplacian (2-sided exponential) distribution in grey-level differences
• Moving objects are clusters with bivariate Gaussian distributions in image coordinates and uniform distribution in grey-level differences
A 2-D example
Background adaptation• The background image, subtracted from the
current image, is not constant• If it were known which pixels belong to the
background, it would be possible to update their grey levels by low-pass filtering
• Since we only have probabilities of pixels belonging to the background, we use these probabilities
Background adaptation• Each reference pixel value is a low-pass version of
the corresponding pixel in the image sequence• The pixel is updated in proportion to the
probability that it is a background pixel, i.e. the time constant of adaptation is inversely proportional to the probability that the pixel originates from the background.
Cluster merging
Two clusters are merged if the expected decrease of the log-likelihood after merging, is less than a threshold.
For computational convenience, the decrease of log-likelihood is assumed to be equal to the decrease of the EM functional.
Cluster merging (continued)
The expected change of EM functional after merging depends on
• the number of pixels in the 2 clusters;• the similarity in densities between the 2 clusters;• the distance between the 2 clusters;• the similarity in shape of the 2 clusters.
Example: too far for merging
Example: close enough for merging
The 2 clusters do not fit into a single ellipse
The 2 clusters fit into a single ellipse
Cluster removal
A moving cluster is removed if the expected decrease of EM functional after merging it into the background, is less than a threshold.
The expected change of EM functional depends on the parameters of the moving cluster:
• the number of pixels originating from the cluster;• the density of pixels in the Gaussian ellipse;• the average grey-level difference.
Cluster generationIf there are lots of pixels close together, which are
unlikely to be generated from any of the clusters (including the background cluster), a new cluster is generated
Cluster splittingA cluster is split if the distribution of pixels
belonging to the cluster is significantly different from the expected distribution, in any of its sections
Iterating over an image sequenceFor each frame:• All clusters are initialized with the same
parameters as in the previous frame• Next, the image is inspected for generation of new
clusters • Clusters are tested for splitting • The EM clustering algorithm is applied to update
the cluster parameters• Clusters are tested for merging or removal• Finally, the background is updated
Cluster parameters
• For the target clusters: center and covariance of the Gaussian ellipsoid
• For the background cluster: background image and exponential parameter of the distribution of grey-level differences
• In addition, number of clusters and numbers of pixels for each cluster
Parameters of the method
• Thresholds for the generation, splitting, removal/merging of clusters
• Initial estimate of the covariance of a new cluster
• Time constant of local background adaptation
• Convergence criterion for EM
PETS 2000 test sequence
PETS 2001 test sequence 3:1
Conclusions
• The method is limited by occlusion• Merging can be reduced by learning the
grey-level distribution for each target cluster, but not much (PETS 2001)
• Deciding which pixel belongs to which cluster is never necessary