Click here to load reader

  • date post

  • Category


  • view

  • download


Embed Size (px)



    Dominikus Baur, Bartholomaus Steinmayr, Andreas ButzMedia Informatics Group

    University of Munich (LMU), Munich, Germany{dominikus.baur,andreas.butz}, [email protected]


    The lyrics of a song are an interesting, yet underused typeof symbolic music data. We present SongWords, an ap-plication for tabletop computers that allows browsing andexploring a music collection based on its lyrics. Song-Words can present the collection in a self-organizing mapor sorted along different dimensions. Songs can be orderedby lyrics, user-generated tags or alphabetically by name,which allows exploring simple correlations, e.g., betweengenres (such as gospel) and words (such as lord). In thispaper, we discuss the design rationale and implementationof SongWords as well as a user study with personal musiccollections. We found that lyrics indeed enable a differentaccess to music collections and identified some challengesfor future lyrics-based interfaces.


    Lyrics are an important aspect of contemporary popularmusic. They are often the most representative part of asong. They verbally encode the songs general message,thereby strongly contributing to its mood. For most people,singing along is one of the easiest ways to actively partici-pate in the music experience. Lyrics are also regularly usedfor identifying a song, since the first or most distinct lineof the chorus often also is the songs title. This importanceof lyrics makes purely instrumental pieces rather rare incontemporary popular music.

    Despite this central role of lyrics, computer interfacesmostly still ignore them. Media player software for per-sonal computers mostly only shows lyrics after installingadditional plug-ins, and although the ID3 metadata stan-dard for digital music contains a field for lyrics, it is rarelyused. More complex operations, such as browsing andsearching based on lyrics, are even further away and scarce-ly touched in research (e.g., [6]). We therefore think thatlooking at music from the perspective of lyrics can allowusers a fresh view on their collection, reveal unknown con-nections between otherwise different songs and allow themto discover new patterns between the lyrics and other as-pects of the music.

    Permission to make digital or hard copies of all or part of this work for

    personal or classroom use is granted without fee provided that copies are

    not made or distributed for profit or commercial advantage and that copies

    bear this notice and the full citation on the first page.c 2010 International Society for Music Information Retrieval.

    Figure 1. Browsing a music collection through its lyricson a tabletop

    In this paper, we give an overview of SongWords (seefigure 1 and video 1 ), an application for tabletop computerswhich supports navigating music collections and investi-gating correlations based on the lyrics of songs. We presentrelated research on browsing and tabletop interfaces, de-scribe and explain our interface and interaction design de-cisions, talk about the implementation of SongWords andpresent the results of a user study.


    Content-based MIR often uses not only the instrumentalbut also the vocal parts of a song. However, since ex-tracting the words of a song directly from the audio sig-nal has proven to be difficult, a common approach is togather lyrics from the internet based on available metadata(e.g., [14]). These lyrics then enable tasks that go beyondpure retrieval, such as semantic or morphologic analysis(topic detection [13], rhymes in hip hop lyrics [9], genreclassification from rhyme and style features [16]). Otherwork is approaching the problem of mapping textual lyricsto an audio signal ( [12], [7]). Combining an ontologywith lyrics enables even more sophisticated tasks: Bau-mann et al. used natural language processing and mappedtext to a vector space model to calculate a lyrical simi-larity value for pairs of songs [1]. Fujihara et al. pre-sented an approach for creating bi-directional hyperlinksbetween words in songs that could be applied not only totextual lyrics but also to the actual audio data [6]. They



    11th International Society for Music Information Retrieval Conference (ISMIR 2010)

  • Figure 2. Songs are organized on a map based on lyrics or tags (left), or sorted alphabetically by their artists name (right)

    also describe an application called LyricSynchronizer [8]that allows browsing collections by navigating through thealigned song lyrics. There is, however, no work on visual-izing a complete music collection based on lyrics.

    In order to make complex music collections accessible,a multitude of browsing interfaces are available. Beyondthe sorted lists commonly used in media player software,abstraction and filtering capabilities are useful, e.g., by ap-plying techniques from information visualization [23] orby providing views based on different facets [2]. Since mu-sic content provides a very high-dimensional data set, com-plexity also has to be reduced for visualization. PampalksIslands of Music [17] is the best known example for thisapproach. It has also been extended to incorporate multipleviews on different acoustic aspects [18]. Self-organizingmaps have also widely been used for visualizing text doc-uments (e.g., [5]). In a similar vein, several projects al-low browsing a music collection on tabletop displays usingself-organizing maps of different low- and high-level au-dio features (SongExplorer [11], MarGrid [10], DJen [3],MusicTable [22]). Lyrics, however, havent been used forbrowsing so far.


    When designing SongWords we started from two user tasks:First, users should be able to easily browse and searchthrough their personal collections based on lyrics. Song-Words should give them a new perspective on their ownsongs and let them browse through the collection from wordto word (similar to [7]). Second, we wanted to allow usersto corroborate or disprove hypotheses about connectionsbetween lyrics and genres. It should be easy to discovercorrelations between different genres and words, such asHip hop lyrics often use cuss words or Pop songs oftenrevolve around love and baby.

    Since such patterns are hard to discover by scrollingthrough a text-based list, we decided to map the high-di-mensional information space to a two-dimensional canvasusing Self-Organizing Maps [15]. Furthermore, as the re-sulting map at a reasonable level of detail largely exceededthe screen size, we also implemented a Zoomable User In-terface to navigate the large virtual canvas on a physical

    display. With a potentially very large number of items, wefinally chose to use an interactive tabletop display for itsadvantages regarding screen space [24] and its potential formulti-user interaction. In addition, zooming and panningwas found to work better using direct touch and bi-manualinteraction than using mouse input [4].

    3.1 Visualization and Interaction

    SongWords analyzes a given music collection and displaysit on a two-dimensional canvas. The visualization consistsof two self-organizing maps for lyrics and for tags, as wellas an alphabetical list by artists names for direct access tosongs (see figure 2). In addition, there is a view for theresults of text searches (see below). The user can switchbetween these different views by pressing one of a numberof buttons at the border of the screen.

    All songs of the collection are represented on the virtualcanvas by their cover art. To optimize the use of screenspace, each item is displayed as large as possible with-out overlapping with other songs. The underlying self-organizing map guarantees spatial proximity between sim-ilar items regarding the currently chosen aspect (lyrics ortags). The map contains black areas in the background thatconnect clusters of items and displays the most relevantwords or tags next to the song items to give overview andallow orientation. A common interaction that is possiblein each context is pan and zoom (see figure 3). Panningis triggered by putting the finger to the canvas outside of asong icon and dragging, with the canvas sticking to the fin-ger. Zooming and rotation are controlled by two or morefingers and the system calculates the geometric transfor-mation of the canvas from their movements.

    In addition to this geometric zoom for the virtual can-vas, SongWords also implements a semantic zoom for songicons (see figure 4): At the least detailed zoom level, songsare represented as colored squares to reduce screen clutterwith thousands of items. The items colors represent thehome collection of the song when several collections areavailable. When zooming in, the solid colors are replacedby the artwork of the corresponding record. By zoomingfurther in (or tapping once on the song icon) the artist, ti-tle and lyrics of the song become available. Here, the user


    11th International Society for Music Information Retrieval Conference (ISMIR 2010)

  • Figure 3. Uni- and bi-manual gestures for panning, rotation and zoom

    can scroll through the text if the screen space does not suf-fice, mark sections of it and search for these sections inthe collection. Despite SongWords focus on text, we de-cided against using an on-screen keyboard for text search.Not only would it have taken up screen space (or requiredan explicit mode switch) and suffered from typical prob-lems of virtual keyboards such as missing tactile feedback,it would also have allowed erroneous search terms. Searchresults in turn are displayed on a spiral based on their rele-vance (see below). If multiple song collections are visible(e.g., from different users), each song icon has a coloredborder that represents its home collect