ETISEO

Benoît GEORIS and François BREMOND

ORION Team, INRIA Sophia Antipolis, France

Lille, December 15-16th 2005

2

Fair Evaluation

Unbiased and transparent evaluation protocol

Large participation

Meaningful evaluation

3

Fair Evaluation

Meaningful evaluation Etiseo video sequences: dedicated to specific

problems, with several levels of difficulties

Definition of specific ground truth, reference data and metrics taking advantage of contextual information

For each partner, presentation of results for problems (one after one) that will be studied by ETISEO with variations from easy to hard

For a specific problem, global comparison of partner performances (e.g., sensitivity of tracking wrt occlusion)

4

First Evaluation Cycle Analysis Evaluation of the participation:

Questionnaire & video clips Number of participants and focus

From simple (common) conditions: no occlusion, middle view, well contrasted, no artifact, no light change, color, mono-camera

To complex conditions: Detection Classification Tracking Multiple cameras Events

What can be modified for the 2nd evaluation cycle? Video clips & ground truth (now) Reference data, metrics & comparison (later)