Inferring the meaning of chord sequences via lyrics

Tom O'HaraComputer Science DepartmentTexas State UniversitySan Marcos, TXto17@txstate.edu

WOMRAD-11: 2nd Workshop on Music Recommendation and Discovery

23 October 2011

Talk OverviewIntroduction: Lyric chord annotations for

unsupervised learningBackground: Supervised music categorization;

parallel corporaProcess: Cooccurrence statistics via contingency

tablesAnalysis: Major vs. minor chord associationsConclusion: Summary and future plans

Introduction OverviewTypical music recommendation approachParallel text corporaMore resources for music recommendationOnline tabs and chords

IntroductionTypical music recommendation approach

o Suggest songs based on common categories (e.g., mood)

o Human annotations of song categoryo Typically done at song levelo Tedious/subjective to do at segment level

Parallel text corporao Same documents in two or more languageso Developed for human consumption (e.g., UN)o Invaluable for automatic machine translation

Introduction (continued)New resource for music recommendation

o Guitar tablature (tabs)o Kept up to date by musicianso Augments human annotationso Finer granularity (chord sequence vs. song)

Online tabs and chordso Usenet (e.g., alt.guitar.tabs, 10K+)oWeb sites (e.g., Chordie, 200K+)

Background OverviewLearning meaning of musicTranslation lexicon induction

BackgroundLearning meaning of music

o Supervised classification• User annotations: Turnbull et al. (2008)

o Unsupervised classification• Online reviews: Whitman and Ellis (2004)

o Lyric analysis and social tags:• Affect filtering: Hu at al (2009)• Usage, readability, etc.: McKay el al. (2010)

Translation lexicon inductiono Co-occurrence analysis: Fung and Church (1994)o Linkage refinements: Melamed (2000)

Process Steps1. Obtain song data with chords annotated2. Extract lyrics proper (with annotations)3. Optional: Map lyrics into meaning categories

o a. Get tagged data on meaning categories for lyricso b. Preprocess lyrics and untagged chord

annotationso c. Train to categorize over words and hypernymso d. Classify each lyric line from chord annotations

4. Fill contingency table5. Determine chord(s)/token associations

Obtain song data with chords annotated

Taken from Usenet alt.guitar.tabs forumo CRD in subject line

Sample[C] They're gonna put me in the [F] movies[C] They're gonna make a big star out of [G] meWe'll [C] make a film about a man that's sad and [F] lonelyAnd [G7] all I have to do is act [C] naturally

Lyrics are from "Act Naturally" by Johnny Russell and Voni Morrison, with chord annotations for song as recorded by Buck Owens.

Extract lyrics proper (with annotations)o Removes e-mail headers and other extraneous texto Two column table (one row per chord change)• Chord; and words for that chords• Includes end of line and verse indicators

SampleC They're gonna put me in theF movies <endl>C They're gonna make a big star out ofG me <endl> We'llC make a film about a man that's sad andF lonely <endl> AndG7 all I have to do is actC naturally <endl> <endp>

Mapping lyrics via supervised classification

Get tagged data on meaning categories for lyricsPreprocess lyrics and untagged chord annotationsTrain to categorize over words and hypernymsClassify each lyric line from chord annotations

Get tagged data on meaning categories for lyrics

CAL500 for training data Turnbull et al. (2008)

o 500 songs (but only 300 lyrics obtained)o Annotated by 3 userso 135 categories in broad groups

Emotion Category FrequencyLabel freq Label freqAngry-Aggressive 31 Laid-back-Mellow 7Arousing-Awakening 77 Light-Playful 1Bizarre-Weird 7 Loving-Romantic 1Calming-Soothing 91 Pleasant-Comfortable 3Carefree-Lighthearted 28 Positive-Optimistic 0Cheerful-Festive 9 Powerful-Strong 3Emotional-Passionate 23 Sad 3Exciting-Thrilling 2 Tender-Soft 2Happy 6

Preprocess lyrics and chord annotations

Isolate punctuationAdd semantic classes for each word

oWordNet hypernyms Miller (1990)

WordNet Samplemovie#1, film#1, picture#6, moving picture#1, ... => show#3 => social event#1 => event#1 => ... => product#2, production#3 => creation#2 => artifact#1, artefact#1 => whole#2, unit#6 => ...

Train to categorize w/ words & hypernyms

Rainbow text categorization McCallum (1996)

o Song documents with meaning category labelso Tokens for Words and WordNet semantic classeso Default Rainbow settings (e.g., no stemming)o TF/IDF feature selection

Other WordNet text categorization workoMansuy and Hilderman (2006)

Classify each lyric line from annotations

Each line classified as mini-documento Verse included for more context

Original annotationsC They're gonna put me in theF movies <endl>...G7 all I have to do is actC naturally <endl> <endp>

ResultC Light-PlayfulF Light-Playful...G7 Light-PlayfulC Light-Playful

Back to Main Process1. Obtain song data with chords annotated2. Extract lyrics proper (with annotations)3. Optional: Map lyrics into meaning categories=>4. Fill contingency table5. Determine chord(s)/token associations

Fill contingency tableGeneral Format

X \ Y + -+ X Y X ¬Y- ¬X Y ¬X ¬Y

SampleG versus 'film' + -+ 1 2,213- 0 17,522

Determine chord(s)/token associations

Compute co-occurrence statisticsEx: Average Mutual Information

x y y) = (Y P x)= (X P

y) = Y x,= (X Plog y) = Y x,= (X P 2

Analysis OverviewIndividual chords with word tokensChord sequences with meaning category tokens

Individual chords with word tokensMajor vs. minor key differences

Avg. MI Chord Word XY X¬Y ¬XY.00034 C happy 7 1,923 13.00005 G happy 4 2,210 16.00030 Dm happy 3 341 17.00008 Em happy 2 548 18.00176 F bright 10 971 3.00018 Am bright 3 962 10.00071 Bm sad 3 197 4.00032 Bb sad 2 325 5.00039 Em sad 3 1,097 6.00542 Dm sorrow 2 342 5.00068 C sorrow 2 1,928 5

Chord sequences with meaning category tokens

Most frequent chord sequence associationsAvg. MI Chord Sequence Category XY X¬Y ¬XY.0027 D7, D7, D7, D7 Bizarre 30 36 1,358.0037 Em, G, G6, Em Carefree 18 6 594.0032 D, A, A, C#min Carefree 14 2 598.0032 C#min, D, A, A Carefree 14 2 598.0032 A, C#min, D, A Carefree 14 2 598.0032 A, A, C#min, D Carefree 14 2 598.0012 D7, G, C, G Bizarre 14 17 1,374.0018 C, D7, G, C Bizarre 14 19 1,374.0022 D, A, A, D Powerful 13 8 667.0014 C, D, C, D Happy 13 39 502

Conclusion OverviewSummaryFuture workReferences

SummaryIntroduction: Lyric chord annotations for

unsupervised learningBackground: Supervised music categorization;

parallel corporaProcess: Cooccurrence statistics via contingency

tablesAnalysis: Major vs. minor associations; sequence

samples

ConclusionCan learn meaning of chord sequences from

annotated lyricsLarge untapped resource now exploitable for

music recommendation

Future workObjective measures for evaluation

o Complication: subjectivity of chord sequence meaning

Additional aspects of music for modeling meaningo ex: Tempo and note sequences

ReferencesP. Fung and K. W. Church. K-vec: A new approach for aligning parallel texts. In Proc.

COLING, 1994.X. Hu, J. S. Downie, and A. F. Ehman. Lyric text mining in music mood classification. In

Proc. ISMIR, pages 411-6, 2009.T. Mansuy and R. Hilderman. Evaluating WordNet features in text classification models.

In Proc. FLAIRS, 2006.A. K. McCallum. Bow: A toolkit for statistical language modeling, text retrieval,

classification and clustering. www.cs.cmu.edu/ mccallum/bow, 1996.∼C. McKay et al. Evaluating the genre classification performance of lyrical features

relative to audio, symbolic and cultural features. In Proc. ISMIR, 2010.I. D. Melamed. Models of translational equivalence among words. Computational

Linguistics, 26(2):221-49, 2000.G. Miller. Special issue on WordNet. International Journal of Lexicography, 3(4), 1990.C. Schmidt-Jones and R. Jones, editors. Understanding Basic Music Theory. Connexions,

2007. http://cnx.org/content/col10363/latest.D. Turnbull et al. Semantic annotation and retrieval of music and sound effects. IEEE

TASLP, 16 (2), 2008.B. Whitman and D. Ellis. Automatic record reviews. In Proc. ISMIR, 2004.