August 12, 2004IAML - IASA 2004 Congress, Olso1 Music Information Retrieval, or how to search for...
-
Upload
morgan-clarke -
Category
Documents
-
view
224 -
download
2
Transcript of August 12, 2004IAML - IASA 2004 Congress, Olso1 Music Information Retrieval, or how to search for...
August 12, 2004 IAML - IASA 2004 Congress, Olso 1
Music Information Retrieval, or how to search for (and maybe find) music and do away with
incipits
Michael FingerhutMultimedia Library and
Engineering BureauIRCAM – Centre Pompidou
August 12, 2004 IAML - IASA 2004 Congress, Olso 2
Why Music Information Retrieval
Increased availability of musical contents in digital form (remotely)
Networks
... actions, methods and procedures for recovering stored data to provide information on music.
Need for…
Faster methods for processing contents and producing "meaning"
Computing power
Increased availability of musical contents in digital form (locally)
Storage
ImplicationTechnical Advance
pattern extraction, form recognition…
synchronization, summarization
indexation
pitch, voice extraction
work
publicationspaper
symbolic
OCR
score
about
sound
audio programs
text
fingerprint
structure
taste, mood
user
musicology
data
multimediaaudio
data
rights owner
DR
M
annotation
performer
composer
performance (live, recorded)
info
rmat
ion
mu
sic
sto
red
dat
a
metadata
genre
automatic
playlists
librarian
lyrics
abstract
digital
symbolic
physical
semantic
author
interaction
cognitive, social
theory
feature extraction,
identification
motives
conc
ept
→ s
ign
→ s
igna
l →
si
gn
→
conc
ept
textual
features
characteristicsmusical
acoustical
interpretation similarity form orchestration
melody key harmonypolyphony tempo instrumentation
chordssegments dynamics rhythm
…durationpitch timbre intensity spatial room
analysis
A (much simplified) MIR map
similarity
metadata
August 12, 2004 IAML - IASA 2004 Congress, Olso 4
A typology of MIR
Preprocessing• OCR, digitization, compression• Encoding, notation• Feature extraction• Segmentation• Instrument recognition• Voice recognition
Indexation• Identification• Clustering• Classification
Extraction• Melody, key, harmony, rhythm…
Structural analysis• Polyphony• Repetition• Similarity• Summarization
Organization• Databases, systems, networks• Compression• Synchronization• Metadata
Search• Objective criteria
• Metadata indices (name, title, period, genre, instrumentation…)
• Full-text (with or without semantic tags)• Query by example (audio excerpt, melody,
contour, rhythm, tonality, harmony…)• Similarity• Acoustical characteristics
• Subjective criteria• Mood• Taste
Retrieve, deliver, use• Browsing• Playlists• Using and reusing (annotate, combine, transform)• Rights management (recognition, watermarking…)
Usability• Evaluation• User studies
August 12, 2004 IAML - IASA 2004 Congress, Olso 5
Database of 100,000 documents
Common methods in MIR
Modeling• Start from a theory
• Look for patterns Statistical
• Look for patterns
• Build a theory Evaluation
• Relevance (recall, precision)
• Performance (speed, friendliness)
• Standardized test collections
1000 relevant documents 400 retrieved
documents
700 100300
Recall: 30% (300 out of 1000)
Precision: 75% (300 out of 400)
query
August 12, 2004 IAML - IASA 2004 Congress, Olso 6
MIR as a multidisciplinary domain
000 – General• 000 Generalities & computer science• 001 Knowledge
• 001.4 Research
• 003 Systems• 004 Data processing
• 004.5 Storage • 004.6 Interfacing and communications
• 005 Computer programming• 005.4 Systems programming and programs • 005.7 Data in computer systems
005.8 Data security
• 006 Special computer methods • 006.3 Artificial intelligence • 006.4 Computer pattern recognition
• 006.5 Computer sound synthesis • 020 Library and information sciences• 025 Library operations
• 025.3 Bibliographic analysis and control • 025.4 Subject analysis and control • 025.5 Services to users • 025.6 Circulation
100 – Philosophy and Psychology• 150 Psychology• 152 Perception
• 152.1 Sensory perception • 152.3 Movements and motor functions, facial expressions, laterality • 152.4 Emotions and feelings • 152.5 Physiological drives • 152.8 Quantitative threshold, discrimination, reaction-time studies
• 153 Mental processes and intelligence• 153.7 Perceptual processes
300 Social sciences• 300 Sociology and anthropology
• 302.2 Communication
• 306 Culture and Institutions
• 310 General Statistics
• 340 Law• 341.7 Law of international cooperation
• 380 Commerce
• 384 Communications; Telecommunications
500 – Natural Science and Mathematics• 510 Mathematics
• 516 Geometry
600 – General Technology• 620 Engineering & allied operations
• 620.2 Sound and related vibrarions
• 621.3 Electric, electronic, magnetic, communications, computer engineering
• 681 Precision instruments & other devices
700 – The Arts• 780 Music
• 780.7 Education, performances
• 780.92 musicians, composers
• 781 General principles & musical forms• 781.1 Aesthetics, appreciation, taste
• 781.2 Elements of music
• 781.3 Composition
• 781.4 Techniques of music
• 781.5 Kinds of music
• 781.8 Musical forms
August 12, 2004 IAML - IASA 2004 Congress, Olso 7
Take One, or SummarizingDave Brubeck’s Take Five (5m25s)
all 1:47:35 A small bit of each segment
each 0:14:82 One segment of each type
longest 0:03:25 Longest segment
most freq. 0:03:25 Most frequent segmentsimilarity matrixThe warmer the color (red = warmest), the more similar
1. Find which parts sound like other parts (timbre similarity)
2. Extract segments
3. Produce summaries
6
5
4
3
2
1
Courtesy of Geoffroy Peeters, Ircam
0m0s
0m0s
5m25s
5m25s
August 12, 2004 IAML - IASA 2004 Congress, Olso 8
A tool to catalog and extract audio CD contents for online distribution
Automatic identification of CDs• Compute CDDB of the CD
Metadata retrieval and correction• Query Internet CDDB for metadata
• Allow correction
Extraction and compression Transfer to a Web server
August 12, 2004 IAML - IASA 2004 Congress, Olso 9
August 12, 2004 IAML - IASA 2004 Congress, Olso 10
August 12, 2004 IAML - IASA 2004 Congress, Olso 11
August 12, 2004 IAML - IASA 2004 Congress, Olso 12
August 12, 2004 IAML - IASA 2004 Congress, Olso 13
August 12, 2004 IAML - IASA 2004 Congress, Olso 14
August 12, 2004 IAML - IASA 2004 Congress, Olso 15
August 12, 2004 IAML - IASA 2004 Congress, Olso 16
August 12, 2004 IAML - IASA 2004 Congress, Olso 17
August 12, 2004 IAML - IASA 2004 Congress, Olso 18
August 12, 2004 IAML - IASA 2004 Congress, Olso 19
What’s the use…
Increased collections• Quantity
• Variety Help in organizing
• Cataloguing and indexing Help in finding
• Search and retrieval
Contribute knowledge• Metadata, indexation
• Relevance
• Users Getting involved
• ISMIR annual conferenceswww.ismir.net• Be informed
• Publish, speak
• Review
• Organize