Tracking-dependent and interactive video projection (Big Brother project)

Team

Pierre Bretéché

Matei Mancas

Jonathan Demeyer

Thierry Ravet

Donald Glowinski

Gualtiero Volpe

System Architecture

Summary

• Robust tracking

• Modeling human motion attention of a visual scene

Summary

• Robust tracking

Tracking should be robust to severe illumination changes !

• Modeling human motion attention of a visual scene

Robust tracking: data acquisition

Color tracking of red hats

IR tracking of lights on the top of the red hats

Robust tracking: data fusionProjective transform

Robust tracking: data fusion

Selected points (Matlab)

Transformation matrix (EyesWeb)

Image warping (EyesWeb)

Robust tracking: data fusionRED: color tracking / GREEN : IR tracking / BLUE : Final fused tracking

Summary

• Robust tracking

• Modeling human motion attention of a visual scene

Summary

• Robust tracking

• Modeling human motion attention of a visual scene

Regions of interest should be dynamically highlighted and visual effects corresponding to outstanding events should be displayed

Attention: in space (blob speed)HOT RED : most important / DARK RED : less important

Global contrast !

Attention: in time (blob speed)HOT RED : really interesting / BLACK : really boring

Rarity on 4 seconds

Attention: in time (quantity of motion)HOT RED : really interesting / BLACK : really boring

Rarity on 4 seconds

ConclusionSummary :

• Multi-blob real time tracking (on IR images and color images)• Image registration using projective warping• Data fusion using a weighted combination of blob positions based on confidence levels for each modality• Demonstration of motion attention in space (instantaneous) and in time (short-time memory) -> motion is not necessarily salient : it depends on the context• Use of blob speed and silhouette quantity of motion features

So we worked hard ...

Conclusion... but we will work even harder :

• Refinement of confidence level for each modality• Better hardware is needed: adapted optics for the cameras, and a color camera with the same characteristics than the IR camera: easy modality registration• A computer with several firewire ports: easy modality synchronization• Use of other features for attention for tracked paths (direction, acceleration, direction variation, trajectory curvature, …) and for gesture expressivity (energy, internal/external quantity of motion, …)• Long-time memory attention (flickering lights, commonly used paths, …) to get higher level motion segmentation• More efficient instantaneous attention (space) which works also with moving cameras and for surprising behaviors in crowds …

Thank you !