AHRC ICT Strategy Projects Meeting 14/12/05

ICT Tools for Searching, Annotation and Analysis of Audio-Visual Media

Lancaster University(Institute for the Contemporary Arts & Institute for Cultural Research):

Alan Marsden

Adrian Mackenzie

Adam Lindsay

Oxford University (Phonetics Laboratory):

John Coleman

Greg Kochanski

Harriet Nock

AHRC ICT Strategy Projects Meeting 14/12/05

Motivation

• Increasing access to and interest in audio-visual materials

• Emerging technologies: Speech annotation and retrieval

Music information retrieval

Video annotation

• Benefits to arts & humanities researchers Facilitating existing types of research?

Enabling previously intractable research?

Engendering new types of research?

AHRC ICT Strategy Projects Meeting 14/12/05

Domain

• Speech, music & video All extended in time

Use similar technologies and often occur in combination

• Searching, annotation & analysis But not focusing exclusively on these activities

• Materials Collections and sources of audiovisual materials

Developments reported in technological literature

Investigation of specific technologies

Contacts with arts & humanities researchers—the ultimate beneficiaries

AHRC ICT Strategy Projects Meeting 14/12/05

Audiovisual collections• Over 300 UK film collections (Enser+Sandom, 2001 citing 1997 figure)

Academic, commercial, private, public

• British Library Sound Archive Over 550,000 hours of audio and moving imagery in multiple categories

• Specialised archives and corpora: Imperial War Museum South-East Film and Video Oxford Performances of Greek and Roman Drama SFSU Poetry Center (readings by original poets) Oxford Devon English Oxford East Timorese oral histories etc.

• Web: Public-domain/Creative Commons audiovisual digital libraries (e.g. open-video,

freesound) Institutional archives of digital research material, lectures, meetings (e.g. MIT

dSpace, Tate Modern Online Events Archive) Audio blogs (podcasts), video blogs, moblogs (from portable devices) Commercial music download and subscription services (e.g. iTunes, Naxos Music

Library)

AHRC ICT Strategy Projects Meeting 14/12/05

Technology Landscape

• Goal: identify current and emerging technologies of potential use to humanities researchers

• Survey: Literature Technologist interviews (9 so far)

• Broad classes of technology: Analysis and annotation e.g. audio type (speech, music, …), speaker, topic Search and browse Summarise individual items or collections E.g. blinkx.tv, FERRET, match (next)

• Challenges: Identifying limitations and/or pre-requisites to deployment within arts & humanities E.g. digitisation, data quality and system adaptation requirements, language

issues, deployment requires research project/services engagement, copyright issues

AHRC ICT Strategy Projects Meeting 14/12/05

A&H Researcher Interviews• Goal: identify researcher needs and potentially helpful

technologies

• Phase 1: short interviews to identify interested fields (currently 10 scheduled) Current audiovisual and ICT usage Canned demonstrations of existing Web tools (post-interview handout

lists actual sites for later exploration, if wished) Potential usage scenarios in research (and possibly teaching) Anecdotal egs: “ramadan advertising” or “overconsumption during

ramadan”, finding novel speech or film examples for class

• Phase 2: in-depth interviews to identify usage scenarios Identify model scenarios for tool deployment Format dependent upon Phase 1 output

• Challenges: Identifying arts & humanities researchers with both interest and time

to meet

AHRC ICT Strategy Projects Meeting 14/12/05

Outcomes• Report

Existing tools and technologies Likely tools and technologies within 5 years Usage scenarios Impediments to research (e.g. access and copyright

issues) Areas for future development

• Website Contents as in report, but searchable and linked

• Weblog communication and recording tool for dual-site project material will be mutated into website and report publicly accessible: ict4av.lancs.ac.uk