Research Update and Future Work Directions

– Jan 18, 2006 –

Ognjen ArandjelovićRoberto Cipolla

Overview

Research update:

1. Face recognition from video for

i. User authentication

ii. Multimediaretrieval/organization

2. Acquisition conditions-adaptiveimage filtering

3. Local manifold illumination-invariants

AFR from Video: Authentication (ECCV)

Key ideas:

• Sequence re-illumination algorithm

• Offline learning: generic effects of illumination across human face shape variation

Addressed invariance to:

1. Illumination

2. Pose

3. User motion pattern

AFR from Video: Authentication (ECCV)

Key results:

• Average recognition 99.7% on 171 people (over 1300 sequences)

• Excellent generalization, even across race

• Interesting findings on image filters for AFR

Future work:

• Efficiency improvement (more compact representations of FMMs…)

• Smarter use of image filters (different research direction)

Automatic Cast Listing in Films (CVPR)

Visually defined clustering – on face appearance manifolds

Key ideas:

• Similarities between people exhibit coherence – exploited by working in the Manifold Space (each point a manifold)

• Iterative unsupervised learning, bootstrapped using offline training

Automatic Cast Listing in Films (CVPR)

Key results:

• Algorithm needs more testing – only preliminary results in

• Very promising improvement over simple clustering (inter-manifold distance thresholding)

“Simple clustering” results

My clusters

Single cluster

A New Look at Filters for AFR (FG)

Key ideas and methods:

• Recognition performances of raw and filtered data negatively correlate (ECCV results)

• Learn how to optimally combine raw input and filtered data

• Implicit learning of the severity of data acquisition conditions

• We propose a heuristic, iterative algorithm

A New Look at Filters for AFR (FG)

A summary of the results:

Local Manifold Illumination Invariant (ICPR)

Method overview:

• Consider the generative function of the face appearance manifold

• We show that the angles between hyperplanes of small head motion are invariant under illumination changes

• Manifold is represented as a redundant set of locally linear patches

Probabilistic Extension of MSM (ICPR)

MSM limitations:

• Information loss with subspace dimensionality choice

• Within subspace, all directions treated the same – decreased SNR

Key idea:

• Find the most probable “mutual mode”

Efficiently computed similarity:

Colour invariants for AFR

Key ideas:

• Colour used extensively for detection applications – very little research on its use for recognition

• Step 1: Model non-linear response of the photometric sensor

• Step 2: Recover model parameters

• Step 3: Camera/illumination invariants

AFR for Content-Based Retrieval and Synthesis

Combine:

• Face recognition

• Texture/Segmentation

• Local features-basedretrieval

• Image mosaicing

Retrieval query interface tool