Dig that Lick:Exploring Patterns in Jazz Solos

Simon Dixon1, Polina Proutskova1, Tillman Weyde2, Daniel Wolff2,Martin Pfleiderer3, Klaus Frieler3, Frank Höger3, Hélène-Camille

Crayencour4, Jordan Smith1,4, Geoffroy Peeters5, Doğaç Başaran6,Gabriel Solis7, Lucas Henry7, Krin Gabbard8, Andrew Vogel8

(1) Queen Mary University of London; (2) City, University of London; (3) University of MusicWeimar; (4) CNRS, IRCAM Lab, Sorbonne Université; (5) Telecom ParisTech; (6) Audible Magic;

(7) University of Illinois; (8) Columbia University

Mirage Symposium, June 8-9, 2021

1

Dixon et al. Dig that Lick 1 / 14

The Dig that Lick Project (2017-2019)

Full title: Dig that lick: Analysing large-scale data for melodicpatterns in jazz performances

Enhance existing infrastructures for the deployment of semanticaudio analyses over large collectionsFacilitate access to large audio and metadata collections viainterfaces for content selection, semantic analysis, and aggregationUse the developed infrastructure to analyse the use of melodicpatterns in a large jazz corpus of monophonic solosRelate analytic results to background knowledge to trace andinterpret musical influence across time, space, cultures andsocietiesConvince musicologists (!)

Dixon et al. Dig that Lick 2 / 14

Data: Audio and Metadata

Discographies

Up to 70 000 sessions

Audio Datasets

U.Columbia

~10 000

tracks

U.Illinois

~30 000

tracks

Jazz Encyclopedia

~10 000

tracks

Linked Open Data

LinkedJazz

WikipediaLoC

Smithsonian

VIAF

9 000 musicians

+ relationships

Data

Dixon et al. Dig that Lick 3 / 14

Metadata Ontology for Jazz

Dixon et al. Dig that Lick 4 / 14

(Automatic) Metadata Cleaning

Named Entity ResolutionCharlie Parker, Charley Parker, Чарли Паркер, Charlie “Bird” Parker,Charlie Parker Quartet, Charlie Parker Quintet, Charlie Parker All Starsb, el-b, synt-b, fretless-b, string-b, el-fretless-b, fretless-el-b,keyboard-b, amplified-b, bass

Reconciliation:Louis Armstrong (1901-1971) = Louis Armstrong (1900-1971)

DisambiguationBill Evans (p) ̸= Bill Evans (ss)Camden, on: Adam Birnbaum, Travels (Smalls Records SRCD-0036)̸=Camden, on: Rodney Green Quartet, Live At Smalls (SmallsLIVESL0036)

Dixon et al. Dig that Lick 5 / 14

Audio Processing: Automatic Melody Extraction

Task: estimate the notes of the main melody from the complexmixture of melody and accompanimentOur approach uses advanced AI and signal processing techniquesStage 1: Compute a pitch salience representation: using aconvolutional neural network (CNN) with source-filter non-negativematrix factorisation pretrainingStage 2: Exploit temporal information to track pitch over time:using a recurrent neural network (RNN)Results: generally successful, with some missed and extra notes,octave errors and semitone errorsExample: Original: Estimated: Both:

Dixon et al. Dig that Lick 6 / 14

Pattern Extraction

Importance of patterns to jazz is well evidencedPatterns in pitch (absolute or relative), time (absolute durations orrelative to metre), or bothWe focus on pitch, expressed as n-gramsSelection criteria: played multiple times, in multiple tracks, bymultiple peopleLevenshtein (edit) distance used for exact or inexact matching

Dixon et al. Dig that Lick 7 / 14

DTL1000 Dataset

1060 tracks selected randomly (100+ per decade from 1920-2019)Manual segmentation and labelling of solo instrument (player)Note tracks automatically extracted from monophonic solos

1700 solos, 6M pitch n-gram instances, 5.6M interval n-gramsMetadata (tune, band, musician, instrument, date, location, etc.)

Linked with our semantic modelCan be used to filter searchesDisplayed with results

Similarity search combining DTL1000 with other datasetsWeimar Jazz DatabaseCharlie Parker OmnibookEssen Folk Song Collection

Dixon et al. Dig that Lick 8 / 14

Pattern Search: List Results

Dixon et al. Dig that Lick 9 / 14

Pattern Similarity Search: Timeline Results

Dixon et al. Dig that Lick 10 / 14

Pattern Similarity Search: Graphical Results

Dixon et al. Dig that Lick 11 / 14

Conclusions

Data and interfaces for exploring melodic patterns in jazz solosMultiple data types (human and automatic transcriptions, collections)Audio and symbolic dataMetadata filters to constrain cultural context

Challenges: data coverage and reliabilityLimited availability of data, especially contextual metadataCurrent methods only address monophonic instrumentsAutomatic transcription and metadata processing are error-prone

Useful tools for case studiesTo discover and trace the history of patternsTo investigate how jazz musicians draw on each otherTo make inferences about influence of race, class, and gender

Dixon et al. Dig that Lick 12 / 14

Publications and PresentationsBaşaran, D., Essid, S., and Peeters, G. (2018).Main melody estimation with source-filter NMF and CRNN.In 19th International Society for Music Information Retrieval Conference, pages 82–89.

Frieler, K. (2019).Constructing jazz lines: Taxonomy, vocabulary, grammar.In M. Pfleiderer, W.-G. Z., editor, Jazzforschung heute: Themen, Methoden, Perspektiven, pages 103–132. Edition EMVAS,Berlin.

Frieler, K., Başaran, D., Höger, F., Crayencour, H.-C., Peeters, G., and Dixon, S. (2019a).Don’t hide in the frames: Note- and pattern-based evaluation of automated melody extraction algorithms.In 6th International Conference on Digital Libraries for Musicology, pages 25–32.

Frieler, K., Höger, F., and Pfleiderer, M. (2019b).Anatomy of a lick: Structure and variants, history and transmission.In Book of Abstracts of the Digital Humanities Conference.

Frieler, K., Höger, F., and Pfleiderer, M. (2019c).Towards a history of melodic patterns in jazz performance.In 6th Rhythm Changes Conference.

Frieler, K., Höger, F., Pfleiderer, M., and Dixon, S. (2018).Two web applications for exploring melodic patterns in jazz solos.In 19th International Society for Music Information Retrieval Conference, pages 777–783.

Gabbard, K. (2019).What we are digging out of the data?In 6th Rhythm Changes Conference.

Höger, F., Frieler, K., Pfleiderer, M., and Dixon, S. (2019).Dig that lick: Exploring melodic patterns in jazz improvisation.In 20th International Society for Music Information Retrieval Conference: Late Breaking Demo.

Solis, G. and Henry, L. (2019).Chasing the trane: Quantifying the social journey of a coltrane solo.In 6th Rhythm Changes Conference.

Weyde, T., Wolff, D., Dixon, S., Proutskova, P., Crayencour, H.-C., Smith, J., Peeters, G., and Başaran, D. (2019).Dig that lick: A technical primer for big data jazz studies.In 6th Rhythm Changes Conference.

Dixon et al. Dig that Lick 13 / 14

Acknowledgements

This research was funded under the Trans-Atlantic Program Digging intoData Challenge with the support of the UK Economic and SocialResearch Council (ES/R004005/1), the French National ResearchAgency (ANR-16-DATA-0005), the German Research Foundation (PF669/9-1), and the US National Endowment for the Humanities(NEH-HJ-253587-17).

Dixon et al. Dig that Lick 14 / 14