Inferring the meaning of chord sequences via lyrics Tom O'Hara CS Adjunct at Texas State 2010-11...

Inferring the meaning of chord sequences via lyrics

Tom O'HaraCS Adjunct at Texas State 2010-11 ([email protected])[email protected]

Texas State University - Computer Science Talkbased on talk at ACM Workshop on Music Recommendation and Discovery (WOMRAD)

5 Dec 2011

O’Hara: chord meaning inference

Talk OverviewIntroduction: Lyric chord annotations for

unsupervised learning

Background: Supervised music categorization; parallel corpora in NLP

Process: Co-occurrence statistics via contingency tables

Analysis: Major vs. minor chord associations

Conclusion: Summary and future plans

Teas State U. 5 Dec 2011 2 / 26


Introduction OverviewTypical music recommendation approach

Parallel text corpora usage in NLP

New resource for music recommendation

Online sites for tabs and chords



IntroductionTypical music recommendation approach

Suggest songs based on common categories (e.g., mood)

Human annotations of song category Typically done at song level Tedious/subjective to do at segment level

Parallel text corpora in NLP Same documents in two or more languages Developed for human readers (e.g., UN delegates) Invaluable for automatic machine translation


In the beginning was the Word, En el principio era el Verbo, and the Word was with God, y el Verbo era con Dios, and the Word was God. y el Verbo era Dios.

“Bridge over Troubled Water” [Uplifting]

When you're weary. Feeling small. [Sad]When tears are in your eyes [Sad]I will dry them all. [Reassuring] I'm on your side [Reassuring]ohhhh when times get rough. [Sad/Reassuring?]


Introduction (continued)New resource for music recommendation

Guitar tablature (tabs) and chord annotations Kept up to date by musicians Augments human annotations Finer granularity (chord sequence vs. song)

Online sites for tabs and chords Usenet (e.g., alt.guitar.tab w/ 10K+ songs) Web sites (e.g., www.chordie.com w/ 200K+)


e----------------------------------------------------------b----------------------------------------------------------g----------------------------------------------------------d------------0------------------0------------------0--4-2-0a---------2------------------2------------------2----------e--0-0-4-------------0-0-4---------------0-0-4-------------

e-----------------------------------------b-----------------------------------------g-----------------------------------------d-----------0-4-2-0--------------0-4-2-0--a--------2-------------------2------------e-0-0-4---------------0-0-4---------------

A F#m Pretty Woman Walking Down The Street A F#mPretty Woman The Kind I Like To Meet D EPretty Woman I don't believe you Your're not the truth E7No One can Look as good as you

(mercy)


Background OverviewLearning meaning of music

Translation lexicon induction



BackgroundLearning meaning of music (MIR)

Generally combines audio and textual features Supervised classification

o Mood/meaning categories like Happy, Sad, Bizarreo User annotations (e.g., CAL500): Turnbull et al. (2008)

Unsupervised classificationo Online reviews: Whitman and Ellis (2004)

Lyric analysis and social tagso Affect filtering: Hu at al (2009)o Usage, readability, etc.: McKay el al. (2010)

Translation lexicon induction (NLP)o Co-occurrence analysis: Fung and Church (1994)o Linkage refinements: Melamed (2000)


Pitchfork album review:

Simon & Garfunkel's 1970 swan song, Bridge Over Troubled Water, was both their most effortless record and their most ambitious. The duo spent most of the 1960s as a highly regarded folk act distinguished by their intuitive harmonies and Paul Simon's articulate songwriting, yet compared to the Greenwich Village revivalists, whom they tried to emulate on songs like "A Simple Desultory Philippic" and "Bleecker Street", they were pretty square. By Bookends in 1968, they were settling into themselves, losing their folk revival pretensions and emphasizing quirky production techniques to match their soaring vocals. Two years later, Bridge did that album one better by revealing a voracious musical vocabulary that spanned gospel, rock, R&B, and even classical. As this thoughtful reissue attests, the album sounds unique even 40 years later, driven and defined entirely by their own personal musical and political obsessions.…


Process Steps1. Obtain song data with chords annotated2. Extract lyrics proper with chord annotations3. Optional: Map lyrics into meaning categories

a. Get tagged data on meaning categories for lyricsb. Preprocess lyrics and untagged chord annotationsc. Train to categorize over words and hypernymsd. Classify each lyric line from chord annotations

4. Fill contingency table5. Determine chord(s)/token associations


Mime-Version: 1.0Content-Type: TEXT/PLAIN; charset=US-ASCII

Bridge Over Trouble water [sic] Simon And GarfunkelCapo 1

Chords are :G1(355433) D7(xx0212) D9(xx0210)A7(x02020) B7(x21202) E9(020102) E7(020100) Gm(355333) Gmaj7(320002) B7sus4(xx2202) Fdim(xx0101) E9maj(020101)

Intro: D D9 G A7 Fdim D G A7 D G D G

D G D C G D G When you're weary. Feeling small. When tears are in your eyes D G D G D I will dry them all. A Bm A D D/C# D7 I'm on your side ohhhh when times get rough....

;; Chord Lyrics? When D you're weary. G Feeling D small. When C tears G are D in your G eyes <endl> I will D dry them all. G ??? D ??? G ??? D ??? <endl> I'm A on Bm your A side ohhhh when times get D rough. D/C# ??? D7 ??? <endl>

Chord Word CW C~W ~CW ~C~W B7 underground 1 71 0 17522 G two 2 2212 23 17521 C morning 1 1929 31 17522 C Feed 1 1929 8 17522 D7 so 1 265 142 17522 A things 4 1552 51 17519 D7 are 1 265 162 17522 Bm is 3 197 290 17520 B sitting 1 478 1 17522 C mean 1 1929 3 17522 Chord Word Dice Jaccard MI X2

B7 underground 0.027 0.014 7.933 243.375 G two 0.002 0.001 -0.486 0.259 C morning 0.001 0.001 -1.664 1.651 C Feed 0.001 0.001 0.164 0.014 D7 so 0.005 0.002 -1.085 0.607 A things 0.005 0.002 -0.162 0.055 D7 are 0.005 0.002 -1.272 0.850 Bm is 0.012 0.006 -0.117 0.020 B sitting 0.004 0.002 4.232 17.307 C mean 0.001 0.001 1.333 1.018


Obtain song data with chords annotated

Taken from Usenet alt.guitar.tab forum CRD in subject line

Sample[C] They're gonna put me in the [F] movies[C] They're gonna make a big star out of [G] meWe'll [C] make a film about a man that's sad and [F] lonelyAnd [G7] all I have to do is act [C] naturally

Lyrics are from “Act Naturally” by Johnny Russell and Voni Morrison, with chord annotations for song as recorded by Buck Owens.



Extract lyrics proper with annotations

Notes Removes e-mail headers and other extraneous text Two column table (one row per chord change)

o Chord, and words for that chordso Includes end of line and verse indicators

SampleC They're gonna put me in theF movies <endl>C They're gonna make a big star out ofG me <endl> We'llC make a film about a man that's sad andF lonely <endl> AndG7 all I have to do is actC naturally <endl> <endp>Teas State U. 5 Dec 2011 10 / 26


Fill contingency tableGeneral Format

Sample: G versus 'film'

X \ Y + -

+ X Y X ¬Y

- ¬X Y ¬X ¬Y

+ -

+ 1 2,213

- 0 17,522



Determine chord(s)/token associations

Compute co-occurrence statistics

Ex: Average mutual information

MI(G, film) = 3.156102AvgMI(G, film) = 0.000160

x y y) = (Y P x)= (X P

y) = Y x,= (X Plog y) = Y x,= (X P 2



Optional Process Step1. Obtain song data with chords annotated2. Extract lyrics proper with chord annotations3. Optional: Map lyrics into meaning categories




Oh Baby , don ' t you want to go Oh Baby , don ' t you want to go Back to the land of California To my sweet home Chicago Oh Baby , don ' t you want to go Oh Baby , don ' t you want to go Back to the land of California To my sweet home Chicago Now one and one is two Two and two is four I ' m heavy loaded baby I ' m booked , I gotta go Cryin ' , baby Honey , don ' t you want to go Back to the land of California To my sweet home Chicago...

4: American_state#1 2: state#2 province#1 3: administrative_district#1 1: child#2 kid#4 2: offspring#1 progeny#1 issue#6 3: relative#1 relation#3 4: person#11: girl#1 miss#1 missy#1 young_lady#1 young_woman#1 fille#1 2: woman#1 adult_female#1 3: female#2...

C They're gonna put me in theF movies <endl>

...G7 all I have to do is actC naturally <endl> <endp>

C Light-PlayfulF Light-Playful

...G7 Light-PlayfulC Light-Playful

Text Categorization Settings- Tokens for words and WordNet semantic classes- Default Rainbow settings (e.g., no stemming)- TF/IDF feature selection


Get tagged data on lyric meaning categories

CAL500 for training data Turnbull et al. (2008) 500 songs (but only 300 lyrics obtained) Annotated by at least 3 users 135 categories in broad groups

Emotion category frequency (with one category chosen)Label Freq Label FreqAngry-Aggressive 31 Laid-back-Mellow 7Arousing-Awakening 77 Light-Playful 1Bizarre-Weird 7 Loving-Romantic 1Calming-Soothing 91 Pleasant-Comfortable 3Carefree-Lighthearted 28 Positive-Optimistic 0Cheerful-Festive 9 Powerful-Strong 3Emotional-Passionate 23 Sad 3Exciting-Thrilling 2 Tender-Soft 2Happy 6Teas State U. 5 Dec 2011 14 / 26


Preprocess lyrics and chord annotationsIsolate punctuation

Add semantic classes for each word WordNet hypernyms Miller (1990)

WordNet Samplemovie#1, film#1, picture#6, moving picture#1, ... => show#3 => social event#1 => event#1 => ... => product#2, production#3 => creation#2 => artifact#1, artefact#1 => whole#2, unit#6 => ...Teas State U. 5 Dec 2011 15 / 26


Train to categorize w/ words & hypernyms

Rainbow text categorization McCallum (1996)

Song documents with meaning category labels Tokens for words and WordNet semantic classes Default Rainbow settings (e.g., no stemming) TF/IDF feature selection

FYI: Survey of WordNet text categorization work

Mansuy and Hilderman (2006)



Classify each lyric line from annotationsEach line classified as mini-document

Verse included for more context

Original annotationsC They're gonna put me in theF movies <endl>...G7 all I have to do is actC naturally <endl> <endp>

ResultC Light-PlayfulF Light-Playful...G7 Light-PlayfulC Light-PlayfulTeas State U. 5 Dec 2011 17 / 26


Process Summary1. Obtain song data with chords annotated2. Extract lyrics proper with chord annotations3. Optional: Map lyrics into meaning categories





Analysis OverviewIndividual chords with word tokens

Chord sequences with meaning category tokens



Individual chords with word tokensMajor vs. minor key differences

Avg. MI Chord Word XY X¬Y ¬XY.00034 C happy7 1,923 13.00005 G happy4 2,210 16.00030 Dm happy3 341 17.00008 Em happy2 548 18.00176 F bright 10 971 3.00018 Am bright 3 962 10.00071 Bm sad 3 197 4.00032 Bb sad 2 325 5.00039 Em sad 3 1,097 6.00542 Dm sorrow 2 342 5.00068 C sorrow 2 1,928 5



Chord sequences with meaning category tokens

Most frequent chord sequence associationsAvg. MI Chord Sequence Category XY X¬Y ¬XY.0027 D7, D7, D7, D7 Bizarre 30 36 1,358.0037 Em, G, G6, Em Carefree 18 6 594.0032 D, A, A, C#min Carefree 14 2 598.0032 C#min, D, A, A Carefree 14 2 598.0032 A, C#min, D, A Carefree 14 2 598.0032 A, A, C#min, D Carefree 14 2 598.0012 D7, G, C, G Bizarre 14 17 1,374.0018 C, D7, G, C Bizarre 14 19 1,374.0022 D, A, A, D Powerful 13 8 667.0014 C, D, C, D Happy 13 39 502



Conclusion OverviewSummary

Conclusion

Future work

References



SummaryIntroduction: Lyric chord annotations for

unsupervised learning

Background: Supervised music categorization; parallel corpora

Process: Co-occurrence statistics via contingency tables

Analysis: Major vs. minor associations; sequence samples



ConclusionCan indeed learn meaning of chord sequences

from annotated lyrics

Large untapped resource now exploitable for music recommendation



Much future workObjective measures for evaluation

Complication: subjectivity of chord sequence meaning

Additional aspects of music for modeling meaning Tempo and note sequences Better informed by music theory (Schmidt-Jones and Jones 2007)

Association over phrases, etc. Relational tuples (e.g., <guy, loses, girl>)

Songwriter aids Suggest chord sequences for lyrics And vice versa!



ReferencesP. Fung and K. W. Church. K-vec: A new approach for aligning parallel texts. In Proc.

COLING, 1994.

X. Hu, J. S. Downie, and A. F. Ehman. Lyric text mining in music mood classification. In Proc. ISMIR, pages 411-6, 2009.

T. Mansuy and R. Hilderman. Evaluating WordNet features in text classification models. In Proc. FLAIRS, 2006.

A. K. McCallum. Bow: A toolkit for statistical language modeling, text retrieval, classification and clustering. www.cs.cmu.edu/ mccallum/bow∼ , 1996.

C. McKay et al. Evaluating the genre classification performance of lyrical features relative to audio, symbolic and cultural features. In Proc. ISMIR, 2010.

I. D. Melamed. Models of translational equivalence among words. Computational Linguistics, 26(2):221-49, 2000.

G. Miller. Special issue on WordNet. International Journal of Lexicography, 3(4), 1990.

C. Schmidt-Jones and R. Jones, editors. Understanding Basic Music Theory. Connexions, 2007. http://cnx.org/content/col10363/latest.

D. Turnbull et al. Semantic annotation and retrieval of music and sound effects. IEEE TASLP, 16 (2), 2008.

B. Whitman and D. Ellis. Automatic record reviews. In Proc. ISMIR, 2004.Teas State U. 5 Dec 2011 26 / 26

Inferring the meaning of chord sequences via lyrics Tom O'Hara CS Adjunct at Texas State 2010-11...

Documents

Transcript of Inferring the meaning of chord sequences via lyrics Tom O'Hara CS Adjunct at Texas State 2010-11...