Mnri web page
-
Upload
alberto-casas-rodriguez -
Category
Documents
-
view
71 -
download
0
Transcript of Mnri web page
![Page 1: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/1.jpg)
1
Music Notation,Music Representation,
AND
Intelligence
Donald ByrdSchool of Music, Indiana University
3 February 2005minor rev. 5 February
![Page 2: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/2.jpg)
2
Overview
• Music representations: from abstract to concrete (notation)
• Try to assume just the right things: e.g., no knowledge of music or notation
• For motivation, focus on real-world content-based music-IR situations
• Organization of the TalkI. Motivation: Why is this Important and/or
Interesting?II. Representation and SemanticsIII. Music Notation, Representation, and IntelligenceIV. Conclusions
![Page 3: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/3.jpg)
3
You Are Here
I. Motivation: Why is this Important and/or Interesting?
II. Representation and SemanticsIII. Music Notation, Representation, and
IntelligenceIV. Conclusions
![Page 4: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/4.jpg)
4
Audio-to-Audio Music “Retrieval”
• “Shazam - just hit 2580 on your mobile phone and identify music”
• Query: • Match:• Fantastically impressive to many people• Have they solved all the problems of music
IR? No, (almost) none!• Reason: intended signal & match are
identical => no time warping, let alone higher-level problems (perception/cognition)
![Page 5: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/5.jpg)
5
Similarity Scale for Content-Based Music IR
1. Same music, arrangement, performance, & recording (Shazam)2. Same music, arrangement, performance; different recording3. Same music, arrangement; different performance, recording4. Same music, different arrangement; or different but closely-
related music, e.g., simpler variations (Mozart, etc.), minor revs. (OMRAS, etc.)
5. Different & less closely-related music: freer variations (Schumann, etc.), extensive revisions (AI)
6. Music in same genre, etc. (AI?)7. Music influenced by other music (AI!)
Relationship categories describing what’s in common between items whose similarity is to be evaluated (from closest to most distant)
• For material in notation form, distinctions among (1), (2), and (3) don’t apply: it’s just “Same music, arrangement”
![Page 6: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/6.jpg)
6
OMRAS Polyphonic Audio Music IR: A Task that Needs Note Representation
• Query (audio -> MIDI -> audio)
• Match (original audio recording)
• Started with recordings of Bach preludes and fugues• Did polyphonic (several notes at once) music recognition
• Polyphonic audio -> events is an open research problem•Converted results to MIDI, used as queries against database of c. 3000 pieces in MIDI form• One of worst-sounding cases: Prelude in G Major from Well-Tempered Clavier, Book I•Outcome: the actual piece was ranked 1st!
• Models built from notation database, but note data only
![Page 7: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/7.jpg)
7
Basic Representations of Music & Audio
Audio (e.g., CD, MP3): like speech
Time-stamped Events (e.g., MIDI file): like unformatted text
Music Notation (sheet music): like HTML text
Digital Audio
Time-stamped Events
Music Notation
![Page 8: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/8.jpg)
8
Basic Representations of Music & Audio
Audio Time-stamped EventsMusic Notation
Common examples CD, MP3 file Standard MIDI FileSheet music
Unit Sample Event Note, clef, lyric, etc.
Explicit structure none little (partial voicing much (complete
information) voicing information)
• Converting to form with less explicit structure (to left): moderately difficult
• Converting to form with more explicit structure (to right): very difficult
• MIDI = Musical Instrument Digital Interface: simple, very standard low-bandwidth protocol (from early 1980’s)
![Page 9: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/9.jpg)
9
Music-IR Problems that Needs More Structure
• Joan Public’s problem: find a song, given some of the melody and some lyrics– Needs notes and text (lyrics)– Common question for music librarians, esp. in public
libraries
• Musicologist’s problem: authorship/origin of works in manuscripts– Full symbolic data is important, even “insignificant”
details of notation (John Howard)
![Page 10: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/10.jpg)
10
You Are Here
I. Motivation: Why is this Important and/or Interesting?
II. Representation and SemanticsIII. Music Notation, Representation, and
IntelligenceIV. Conclusions
![Page 11: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/11.jpg)
11
Representation, from Abstract to Concrete
• Cf. Basic Representations of Music & Audio• Abstract: represention: semantics only• Intermediate: syntax (mapping rules)?• Concrete
– for use by computers: encoding– for use by humans: if visual, notation (involves
graphics and/or typography)
• Analogous to knowledge representation vs. data structure
![Page 12: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/12.jpg)
12
Semantics in Music
• Denotation (explicit, well-defined)...• vs. Connotation (implicit, ill-defined)• In text
– Two “definitions” of pig:• 1. Ugh! Dirty, evil-smelling creatures, wallowing in filthy
sties! (Hayakawa)• 2. Mammal with short legs, cloven hoofs, bristly hair, and
a cartilaginous snout used for digging (Amer. Heritage)
– Prose is “mostly” denotation– Poetry is art => connotation much more
important
• Music is always art, & only connotation!• Major issue for content-based music IR
![Page 13: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/13.jpg)
13
From Representation to Notation
• Choosing a representation inevitably introduces bias
• Given a representation, choosing notation inevitably introduces more bias
• Important to consider the purpose (R. Davis et al; Wiggins et al)
• For huge body of important music, we have no choice: notation is CMN (Conventional Music Notation)!– Really “CWMN” (W = Western)– Alternative for some music: tablature (guitar, lute, etc.)– CMN is among the most successful notations ever...– but enormously complex and subtle
![Page 14: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/14.jpg)
14
Notation Says Much about Representation
• CMN standard for Western music after c.1650
• Evolved for “classical” music, but heavily used for very wide range (pop, jazz, folk, etc.)
• Composers/arrangers/transcribers have pushed it hard => reveals things about music representation in general
• Will concentrate on notation (CMN)
![Page 15: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/15.jpg)
15
You Are Here
I. Motivation: Why is this Important and/or Interesting?
II. Representation and SemanticsIII. Music Notation, Representation,
and IntelligenceIV. Conclusions
![Page 16: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/16.jpg)
16
How to Read Music (CMN) Without Really Trying: The Basics
• Four basic parameters of a musical note1. Pitch: how high or low sound is
2. Duration: how long the note lasts
3. Loudness: perceptual analog of amplitude
4. Timbre or tone quality
• Above in decreasing order of importance for most Western music
• Principles of CMN (& e 1)
1. Pitch on vertical axis: clef gives offset (“zero”)
2a. Duration indicated by note/rest shapes
2b. Start times (sum of durations in the voice) on horizontal axis
3. Loudness indicated by signs like p , mf , etc.
4. Timbre indicated with words like “violin”, “horn”, “pizzicato”
![Page 17: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/17.jpg)
17
Why is Musical Information Hard to Handle?
1. Units of meaning: not clear anything in music is analogous to words (all representations)
2. Polyphony: “parallel” independent voices, something like characters in a play (all representations)
3. Recognizing notes (audio only)4. Other reasons
![Page 18: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/18.jpg)
18
Units of Meaning (Problem 1)
• Not clear anything in music is analogous to words – No explicit delimiters (like Chinese)– Experts don’t agree on “word” boundaries (unlike Chinese)
• Are notes like words?• No. Relative, not absolute, pitch is important• Are pitch intervals like words?• No. They’re too low level: more like characters• Are pitch-interval sequences like words?• In some ways, but
– Ignores note durations– Ignores relationships between voices (harmony)– Probably little correlation with semantics
![Page 19: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/19.jpg)
19
Independent Voices in Music (Problem 2) (& e 2)
J.S. Bach: “St. Anne” Fugue, beginning
![Page 20: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/20.jpg)
20
Independent Voices in Text
MARLENE. What I fancy is a rare steak. Gret?ISABELLA. I am of course a member of the / Church of England.*GRET. Potatoes.MARLENE. *I haven’t been to church for years. / I like Christmas
carols.ISABELLA. Good works matter more than church attendance.
--Caryl Churchill: “Top Girls” (1982), Act 1, Scene 1
M: What I fancy is a rare steak. Gret? I haven’t been...
I: I am of course a member of the Church of England.
G: Potatoes.
Performance (time goes from left to right):
![Page 21: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/21.jpg)
21
Complex Notation: Multiple Voices (&
e 3)
Multiple voices on a staff rapidly gets worse with more than 2 (Telemann “Liebe, Liebe”):• 2 voices in mm. 5-6: not bad: stem direction is enough• 3 voices in m. 7: notes must move sideways• 4 voices in m. 8: almost unreadable—without color!• Still acceptable because specific voice is rarely important
![Page 22: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/22.jpg)
22
Problems: Example 1 (superficial but interesting)
• Ravel work has slur with 7 inflection points• Impressive, but complexity is purely graphical• No big deal in terms of representation• …but influence of performance on notation is
revealing
![Page 23: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/23.jpg)
23
Duration and Higher-Level Concepts of Time
• Schubert Impromptu (& e 4)
• Measures: everything between barlines• Time signature: 3/4 = 3 quarter notes
per measure• Triplets: 3 notes in the time normally
used by 2– General concept is tuplets
![Page 24: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/24.jpg)
24
Problems: Example 2 (Deep)
• Chopin Nocturne has nasty situation (& e 5)
• One notehead is triplet in one voice, but normal duration in another
• “Semantics” (execution) well-defined, obvious– Note starts 1/16 before barline…– But also (2/3)*(1/16) before barline! How to
play?
• Reason: musical necessity• Solution for performer: “rubato”• Solution for music IR program: ?
![Page 25: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/25.jpg)
25
Problems: Example 3 (Medium)
• Bach: time signature change in middle of measure
• (& e 6)
• “Semantics” well-defined and obvious– Measure has duration of 18 16ths…– But not until the middle of the measure!
• How does this make sense?• Triplets express same relationship as
equivalent simple/compound meter• Invisible (unmarked) triplets• Cf. Bach Prelude: two time signatures at once
(& e 7)• Reason: avoid clutter
![Page 26: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/26.jpg)
26
Problem 4 (Medium)
• Brahms Capriccio (& e 8)
• Time signature 6/8 => measure lasts 12 16ths
• A dotted half note always lasts 12 16ths…
• but here it clearly lasts only 11 16ths!• Reason: avoid clutter
![Page 27: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/27.jpg)
27
Two Ways to Have Two Clefs at Once
• Clef gives vertical offset to determine pitch
• Debussy (& e 9)
– Bizarrely obvious something odd involving clefs
• Ravel (& e 10)
– Only comparing time signature (3/8) and note durations makes it clear both clefs affect whole measure
• Reason: save space (by avoiding a 3rd staff)
![Page 28: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/28.jpg)
28
Surprise: Music Notation has Meta-Principles! (1)
1. Maximize readability (intelligibility)– Avoid clutter = “Omit Needless Symbols”– Try to assume just the right things for
audience– Audience for CMN is (primarily) performers– General principle of any communication
• Applies to talks as well as music notation!
– Examples: Schubert, Bach, Brahms
![Page 29: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/29.jpg)
29
Surprise: Music Notation has Meta-Principles! (2)
2. Minimize space used– Save space => fewer page turns (helps
performer); also cheaper to print (helps publisher)
– Squeezing much music into little space is a major factor in complexity of CMN
– Especially important for music: real-time, hands full
– Examples: Telemann, Debussy, Ravel
![Page 30: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/30.jpg)
30
The “Rules” of Music Notation
• Tempting to assume that rules of such an elaborate & successful system as CMN work (self-consistent, reasonably unambiguous, etc.) in every case
• But (a) “rules” evolved, with no established authority; (b) many of the “rules” are very nebulous
• In common cases, there's no problem• If you try to make every rule as precise as possible,
result is certainly not self-consistent• Trying to save space makes rules interact; something
has to give!
![Page 31: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/31.jpg)
31
Music Notation Software and Intelligence
• Despite odd notation, really nothing strange going on in almost all of these examples– Ravel slur, Debussy & Ravel 2 simultaneous clefs, Bach &
Schubert invisible triplets, Brahms “short” dotted-half note, Telemann 4 voices/staff are all simple situations
– Chopin Nocturne is complex
• Programmers try to help users by having programs do things “automatically”
• A good idea if software knows enough to do the right thing “almost all” the time—but no program does!
• Notation programs convert CMN to performance (MIDI) and vice-versa => requires shallow “semantics”; makes things much harder
![Page 32: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/32.jpg)
32
You Are Here
I. Motivation: Why is this Important and/or Interesting?
II. Representation and SemanticsIII. Music Notation, Representation, and
IntelligenceIV. Conclusions
![Page 33: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/33.jpg)
33
Conclusions: Review (1)
• Representations express Semantics• Semantics of Music; Denotation &
Connotation• Principles of CMN• Meta-Principles of CMN
1. Maximize readability; Omit Needless Symbols
• Try to assume just the right things for audience• General principle of any communication
2. Minimize space used• Save space => fewer page turns, less paper
![Page 34: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/34.jpg)
34
Conclusions: Review (2)
• We need CMN or equivalent to solve spectrum of music-IR (and other music-IT) problems– But CMN can’t represent everything we want– Even when it can, may not, at least explicitly– Need high-level intelligence to interpret– Solution: unknown– Likely to require major funding :-)
![Page 35: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/35.jpg)
35
Conclusions: Why is this really Important and/or Interesting?
• Some problems directly related to other areas of informatics– Example: Approximate string matching in
bioinformatics
• Encourages progress on real semantics– Connotation is an important part of meaning
in everything– Can often ignore, but any semantics in arts
forces you to deal with connotation– Music is at least as quantifiable as any art,
so likely to be more tractable than others!
![Page 36: Mnri web page](https://reader033.fdocuments.in/reader033/viewer/2022042700/558e8d991a28abe52d8b4594/html5/thumbnails/36.jpg)
36
You Are Here
The End