Online Updating Appearance Generative Mixture Model for Meanshift Tracking

Jilin Tu, Thomas Huang

Elec. and Comp. Engr. Dept.Univ. of Illinois at Urbana and ChampaignUrbana, IL 61801{jilintu, huang}@ifp.uiuc.edu

Hai Tao

Elec. Engr. Dept.Univ. of Calif. at Santa CruzSanta Cruz, CA 12345tao@soe.ucsc.edu

Content

• Problem Statement• Overview of tracking object with appearance variations• Overview of Meanshift Tracking • Meanshift tracking with online appearance updating• Generative model for histogram appearance updating• Experiments• Summary

Problem Statement

• Our Goal:– To track object of vast (but typical)

appearance variations by online appearance updating.

– To avoid drifting problem which is typical in most tracking algorithm with online model updating mechanism.

– To Infer object status by modeling the appearance variations.

Overview: tracking of object with appearance variations

• Subspace Model– Black1998

• Eigentracking: Model the object appearances by PCA.• Unable to incorporate novelties.

– Ross2004• Eigentracking with incremental PCA.• Handles appearance novelties better.

– Ho2004• Tracking with L-Infinity norm subspace.• Handles appearance novelties the best, but no theoretical gurranttee for avoiding drifting problem.

• Mixture model– Jepson2003

• Model the appearance variations as mixture of stable appearance, outliers and two frame variations caused by optical flow. • Drifting problem is not solved.

Overview of Meanshift Tracking– Meanshift(Chen1995):

• An iterative procedure for finding the data density modes (or cluster centers) of an ensemble of data samples.

• By shifting toward the mean of the samples in the vicinity.

Overview of Meanshift Tracking• Meanshift tracking

– Setting: The likelihood of the object appearance is represented by ensemble of likelihood pixel locations in a likelihood image. – Naïve Intuition (CAMSHIFT): Follow the cluster center using meanshift while the cluster center of the likelihood image moves.– Approach(Comaniciu2000): Maximizing the similarity of the histogram in the tracking window and the static histogram of the object appearance.

Overview of Meanshift Tracking• Meanshift tracking in a nut shell

Overview of Meanshift Tracking• Meanshift tracking properties

– Pros:• Realtime• Histogram appearance model tolarates minor appearance variations.

– Cons:• One single static histogram does not capture the appearance variations of the object.

Meanshift Tracking with online appearance updating• The framework

– The idea: updating the histogram with the constraint of key appearances.

Generative model for histogram updating

• The generative model

EM rules for histogram updating

• Expectation

• Maximization

Online EM rules for histogram updating

• Expectation of Log-likelihood taking into the consideration of past observations

where• Online Maximization Rule

Histogram Updating Rule

• Given the generative model we have

)}({ * nh

}ˆ)}),({|({ wnzmp

)}({ nz

1w)}({ )1( nhL

)}({)},({ )2()1( nhnh LR

)}({ )2( nhR

2w

Experiments

• Application: Human head tracking with coarse head pose inference. – Head appearance changes substantially when

human turns his/her head. – Usually only near frontal head pose provides

more information about the person’s identity, facial expression, eye gaze, lip movement, etc.

– It is important to track the head and detect near frontal head pose simultaneously.

Experiments

Tracking with L-Infinity Subspace Updating

Experiments

Tracking by Meanshift Tracker

Experiments

Meanshift Tracking with online updating

Experiments

Meanshift Tracking with online updating: Another example

Pose estimation accuracy 77%

Summary

• Our tracker with online appearance updating– More robust comparing to the standard Meanshift tracker.– Better tracking accuracy.– Avoid drifting problem.– Can infer the object state in relation to the appearance variation. – Acquisition of more than one key appearance of the object sounds annoying, but is practical in most online/offline tracking tasks.

Questions