ARDA VACEARDA VACE

Advanced Research and Development Advanced Research and Development Activity (ARDA) Video Analysis and Activity (ARDA) Video Analysis and

Content Extraction (VACE)Content Extraction (VACE)

Video Analysis and Content Extraction Video Analysis and Content Extraction (VACE)(VACE)Project DescriptionProject Description

• Goal: Develop revolutionary Goal: Develop revolutionary advances in automatic video advances in automatic video extraction, understanding, and multi-extraction, understanding, and multi-modal-fusion. modal-fusion.

• Funded in 3, 2-year phases. (we are currently in the middle of phase 2.

• Multi-domain: meetings, Broadcast Multi-domain: meetings, Broadcast News, UAV (Unmanned Air Vehicle), News, UAV (Unmanned Air Vehicle), Surveillance/Ground reconSurveillance/Ground recon

• NIST is supporting VACE meeting NIST is supporting VACE meeting video needs and evaluation oversightvideo needs and evaluation oversight

SponsorSponsor• ARDAARDAParticipantsParticipants

ProgressProgress//InsightsInsights• Building corpus and evaluation Building corpus and evaluation

infrastructure for video extraction infrastructure for video extraction community by applying knowledge community by applying knowledge from speech from speech recognition/understanding evaluationrecognition/understanding evaluation..

• Developing diagnostic metrics with Developing diagnostic metrics with the mind of moving toward a single-the mind of moving toward a single-number metric.number metric.

Future PlansFuture Plans• 2005 Core evaluations2005 Core evaluations

– face detect/trackingface detect/tracking– Text detect/track/recognitionText detect/track/recognition– person detect/trackingperson detect/tracking– hand detect/trackinghand detect/tracking

• 2006 2006 – event detectionevent detection

•BAE•BBN•USC •UMD•TASC•USF•UCF

•Virginia Tech•U Illinois Urbana Champaign•Salient Stills•IBM/Columbia•Carnegie Mellon U•Pittsburgh Pattern Recognition•Advanced Interfaces

IssuesIssues

• Having “enough” data of the types being Having “enough” data of the types being evaluated.evaluated.

• Ground Truth (Currently we use ViPER Ground Truth (Currently we use ViPER [developed at Univ. Maryland])[developed at Univ. Maryland])– Temporal/SpatialTemporal/Spatial– bounding box or point referencebounding box or point reference– object definitionobject definition

• MetricsMetrics• FundingFunding• Definition of PurposeDefinition of Purpose

Evaluation Domain Sub-Forums• Broadcast News (in coordination with TRECVID)• Meetings • Surveillance • UAV • Tasks:

– Develop evaluation recommendations for 2006+ event detection tasks

• These include task definitions, metrics, schedule, source data, and necessary annotations

• Chairs work with both research community and interested partners to satisfy Sponsor and Research needs

• Identify existing research sour data to be used or list realistic requirements, ensuring that corpora meet above needs

• produce task list and required data resources– Provide recommendations to Core Evaluation Forum– NIST will create mail lists and can set up conference calls

• Forums should hold biweekly teleconferences as necessary to maintain momentum

– Forum chairs will give report at Fall Workshop

DomainTask

Meetings Broadcast News UAV Surveillance

Text Detection (English) - Y - -

Text Tracking (English) - Y - -

Text Recognition (English) - Y - -

Text Detection (Chinese) - - - -

Text Tracking (Chinese) - - - -

Text Detection (Arabic) - Y - -

Text Tracking (Arabic) - Y - -

Face Detection Y Y - Y

Face Tracking Y Y - Y

Hand Detection Y Y - Y

Hand Tracking Y Y - Y

People Detection Y - Y Y

People Tracking Y - Y Y

Vehicle Detection - - Y Y

Vehicle Tracking - - Y Y

Planned Core Eval Tasks/Domains (34 evaluations)Planned Core Eval Tasks/Domains (34 evaluations)

http://www.nist.govhttp://www.nist.gov

Information Technology LaboratoryInformation Technology Laboratoryhttp://www.itl.nist.govhttp://www.itl.nist.gov

itlab@nist.govitlab@nist.gov

Information Access DivisionInformation Access Divisionhttp://www.itl.nist.gov/iaui/http://www.itl.nist.gov/iaui/

john.garofolo@nist.govjohn.garofolo@nist.gov

rachel.bowers@nist.gov

ARDAARDAhttp://www.ic-arda.orghttp://www.ic-arda.org