Post on 17-Jan-2015
description
User Requirements in Audiovisual SearchA Quantitative Approach
Danish NadeemRoeland Ordelman
Robin AlyErwin Verbruggen
AUDIOVISUAL ARCHIVE
BUSINESS ARCHIVE DUTCH PUBLIC BROADCASTERS
DIGITIZATION
CLARIAH PRESENTATIE 11 September 2013
5
800.000 hrs
?METADATA
Document levelvs
Segment level
CLARIAH PRESENTATIE 11 September 2013
8
TOOLS
9
USEFUL?
Developer/ICT researcher
User
CLARIAH PRESENTATIE 11 September 2013
10
FEEDBACK
Access toAudiovisualArchives
Cont
ent
Technology
Users
ARCHIVAL METADATASUBTITLES
AUDIO ANALYSIS:SPEECH/SPEAKER RECOGNITION
pose
beach
Luther King
Obama
Visual analysis
AUDIOVISUAL SEARCH
VIDEO HYPERLINKING
Broadcast professionals
Broadcast Professionals
Re-use
Media Archivists Annotate
Research & Education
Broadcast Professionals
Hergebruik
Media Archivists (documentalisten)
Beschrijven
Journalists Research
Academic researchers Investigate
Education Illustrate
Home user
Public EntertainmentEdutainment
21
Use Scenarios&
System Requirements
Interview & elicitation sessions
Mock-up creation & evaluation
Prototype evaluation
System evaluation
Surveys & log analysis
Qualitative
Qualitative
QualitativeQuantitative
Quantitative/Qualitative
Quantitative approach• Categorization of user requirements
• According to different user groups
• Prioritize implementation of system features
1. 2. 3.
Concept Mapping
• Data collection
• Multidimensional Scaling
• Hierarchical Cluster Analysis
Method• Phase 1: “Idea generation”
– Caveat: unfamiliarity with A/V search– Interviews, requirements elicitation
experiments using system mock-ups– professionals, researchers, home user– 71 “statements”
Method (2)• Phase 2: “Sorting & Rating”
– 47 participants, evenly spread over 3 groups– Categorize according to perceived similarity
• Create categories and label with description
– Rate according to desirability• 1-5: undesirable – highly desirable
Categories vizualisation
Categories
User-defined search functions
Translates into:1. having options for search manipulation and
result representation:– change sort order of results (date, relevance)– Use filters and activate them at any point during
search– Use boolean operators
2. the system should be clear and guide users in a rather complex search process
Advanced search
• Users are eager to make use of advanced search options:– Search using example image– Events, persons, speakers, visual objects– Using multiple metadata sources, e.g.,
• Archival metadata• Speech transcripts• Visual labels
• Less interest in “technical” metadata– Shot types, camera movement, b/w-color
Help functions
• Slight miss-alignment with what developers understand as “help functions”
• Next to “access to help manuals”:– System should be fast – Clear, user-friendly, intuitive UI – Access to video – Clear message when no results are found – Consistent results
USER GROUP COMPARISON
DEVELOPMENT IMPLICATIONS
PRO
RESEARCH
HOME
Conclusions
• Quantitative approach using concept mapping– Useful approach to get some more grip on user
requirements and differences between user groups
– Helps to steer development process of advanced user type specific A/V search tools
– User defined labels may not always be in-line with formal definitions used by developers which may cause confusions, e.g.,
• Help functions vs. clear, smooth and consistent system
Conclusions (2)
• Clear interest of users in 1. advanced search options 2. segment-level access points to AV by means of
rich multi-modal annotations (speech, speakers, events, objects)
3. Clear, transparent, smoothly working system
Upcoming
STUDY HOME USERS
&VIDEO HYPERLINKING
@BBC
http://www.axes-project.eu
Information Retrieval for Information Serviceshttp://www.project-infiniti.nl/