Figure 1: Ensemble Metatone

performing with iPads and

percussion instruments.

MetaTravels and MetaLonsdale:

iPad Apps for Percussive

Improvisation

Charles MartinResearch School of Computer

Science, CECS

Australian National University,

Canberra, Australia

charles.martin@anu.edu.au

Henry GardnerResearch School of Computer

Science, CECS

Australian National University,

Canberra, Australia

henry.gardner@anu.edu.au

Ben SwiftResearch School of Computer

Science, CECS

Australian National University,

Canberra, Australia

ben.swift@anu.edu.au

Permission to make digital or hard copies of part or all of this work forpersonal or classroom use is granted without fee provided that copies are notmade or distributed for profit or commercial advantage and that copies bearthis notice and the full citation on the first page. Copyrights for third-partycomponents of this work must be honored. For all other uses, contact theowner/author(s). Copyright is held by the author/owner(s).CHI 2014 , April 26–May 1, 2014, Toronto, Ontario, Canada.ACM 978-1-4503-2474-8/14/04.http://dx.doi.org/10.1145/2559206.2574805

AbstractPercussionists are unique among instrumentalists in thattheir artistic practice is defined by an approach tointeraction rather than their instruments. Whilepercussionists are accustomed to exploring non-traditionalobjects to create music, these objects have yet toencompass touch-screen computing devices to any greatextent. The proliferation and popularity of these devicesnow presents an opportunity to explore their use incombining computer-generated sound together withpercussive interaction in a musical ensemble.

This interactivity demonstration presents twoiPad-instruments developed in collaboration withEnsemble Metatone, a group formed to explore the“infiltration” of iPad apps into a free-improvisationpercussion ensemble. The apps encourage the performers’exploration through percussive gestures and use networkfeatures to support cohesive improvisation.

Author Keywordsartistic research; music; expression; multitouch; gesture;percussion; user experience

ACM Classification KeywordsH.5.5 [Sound and Music Computing]: Systems

IntroductionPercussion is a musical practice more defined by themethods of interacting with instruments rather than theinstruments themselves. Percussionists perform by“striking, scraping, brushing, rubbing, whacking, orcrashing any... available object” [5]. These percussivegestures are used to coax wide varieties of timbres andmusical gestures from simple instruments. Forpercussionists, free improvisation is often a process ofgestural exploration, discovering new sounds fromtraditional and non-traditional instruments and respondingto other sounds in an ensemble. Like some of theinstruments in a modern percussion ensemble,touch-screen computing devices can be struck, scrapedand rubbed with fingers and hands. Their percussivea↵ordances, their power to generate computer music andtheir widespread popularity, motivate an exploration of theuse of touch-screen devices in a modern percussionensemble.

Figure 2: Rehearsing

MetaTravels on iPads alone.

Figure 3: Rehearsing

MetaTravels with iPads and

percussion setups.

New musical interface designs for touch screen have oftenfocussed on engaging novice or non-musical users [6, 4].In contrast, the iPad instruments described in this paperwere developed to harness the existing exploratory skills ofprofessional percussionists. The musical works that weredeveloped were targeted towards audiences ofexperimental and contemporary classical music.

The group’s rehearsal discussions and audience feedbackreveals a highly successful musical exploration with twoiPad-instruments, MetaTravels and MetaLonsdale. Theperformers’ percussive backgrounds allowed them tocreatively interact with the apps, inventing a vocabularyof gestures that expand the app’s capabilities. Thenetwork features and unique sounds enabled uniqueensemble and individual interactions.

Ensemble MetatoneEnsemble Metatone was brought together to explore the“infiltration” of iPad-based musical instruments into apercussion ensemble through free-improvisation, the onlysuch group in the world. The members of the group,Charles Martin (CM - one of the authors of this paper),Christina Hopgood (CH), Jonathan Gri�ths (JG), andYvonne Lam (YL), had previously studied and workedtogether as professional musicians. CM was the designerof the iPad instruments and facilitator of the group’srehearsals and performances. Over four studio sessionsand four live concerts, two di↵erent performanceparadigms were explored: in the first paradigm, only iPadswere used; in the second, the performers each played asetup which included a number of percussion instrumentsof their own choosing as well as an iPad.

Our approach for studying the emergence of “expressiveinteraction” [3] of the performers with the iPadinstruments followed common methods for collaborativelydeveloping a musical performance. The process forcreating the performance was collaborative amongst allfour musicians with free-improvisation leading to collectivedecisions about stucturing the work both in rehearsal andin performance [2]. The rehearsal process followed Cahn’sconcept of “Creative Music Making” [1] with a series offreely-improvised performances that were recorded, playedback and discussed.

Percussionist Centered DesignFor this research, iPads were chosen as the computerinstrument in the ensemble. While the field of computermusic abounds with creative examples oftouch-sensor-based instruments, the goals of this projectplaced constraints on what could practically be usedwithin the percussion group: the touch-screen devices

needed to be self-contained, durable, and easy toprovision with software updates. While the iPad touchscreens have drawbacks such as a lack of pressuresensitivity or physical feedback, their physical dimensionsmimic that of some simple percussion instruments such asthe woodblock or tambourine.

Figure 4: Screenshot of the

MetaLonsdale app. The green

circle denotes the user’s touch

point while red circles show

delayed notes. The delay and

auto play functions are controlled

by switches on the lower left and

the button on the lower right

changes the sounds that can be

played.

Two apps were produced for the group: MetaTravels andMetaLonsdale (shown in Figure 4), each becomingassociated with an improvised musical work of the samename. Both apps used the same percussion-inspiredinteraction scheme allowing access to pitched percussionsounds and field recordings. The majority of the iPadscreen was a performance surface and there were fewgraphical UI elements. Tapping the screen produced shortsounds at a pitch determined by the location of the tap.Swiping triggered continuous field recordings with thevelocity of the swipe directly mapped to the volume of thefield recording. Both apps featured simple delay functions,that repeat tapped notes, and switchable auto-playfeatures, that algorithmically produced backgroundsounds. These features were activated by switches in theuser interface. A button on both apps allowed theperformer to shu✏e the available sounds. The appsallowed logs of performances to be captured by sendingrecords of each touch event to a server.

Figure 5: JG’s iPad set on a

stick tray with a variety of

percussion instruments and

mallets

MetaTravelsMetaTravels was the first app developed for EnsembleMetatone and featured a sound palette of field recordingsfrom around the world, with chromatically-tuned pitches.The group undertook four iPad-only rehearsals (as shownin Figure 2) with the app and two more rehearsalscombining the iPads with percussion instruments (Figure3). Following this rehearsal series, MetaTravels was

premiered as part of the Canberra International MusicFestival in May 2013.

The performers in Ensemble Metatone were free to choosetheir own percussion instruments to complement the iPadapp. JG and YL settled on drumset-inspired setups andCM and CH used pitched percussion instruments. Whilethe performers commented that the app a↵orded lesscontrol over sounds than the acoustic instruments, theyacknowledged the unique sounds that were available: thefield recordings were “something we can’t replicate” (JG)and using the repeat function with a fast tapping gesturecreated, low pitched “organ”-like drones or distinct highpitched sounds.

The ability to shu✏e the available sounds randomlycontributed to the explorative nature of improvisedperformance. JG commented that “you have to find a wayof making something meaningful with it... that’s one ofthe interesting things about it, finding ways to make itwork.”

MetaLonsdaleMetaLonsdale was commissioned by David Sequiera for anexhibition opening at Everything Nothing Projects inCanberra. The concept for the work was to perform witheveryday sounds of the artistic precinct that surrounds thegallery as well as pitched percussion sounds. Fieldrecordings were gathered from the cafe and shopssurrounding the gallery. In contrast to the chromaticpitches of MetaTravels, diatonic pitches from one of threescales were available in MetaLonsdale. A UI buttomcycled through the scales, changing the pitches availableto the performers and incorporating a harmonicprogression into the instrument. The work was designedto be performed on iPads alone in the small gallery spaceand was premiered by CM and CH (see Figure 6).

The MetaLonsdale app included a network feature thatautomatically searches for other MetaLonsdale instanceson a wifi network. Over the course of a performance, theapp uses this feature to match the selected scale on eachiPad so that the performers’ harmonic position in thework is synchronised. This feature also randomly matchesthe performer’s switch settings. For example, if one playerswitches their delay function on, there’s a chance thateach other players delay function will switch on as well.

Audience members commented on the extra cohesion thatemerged with this feature, with one mentioning that thenetwork synchronisation “merge(d) improvisitory factorswith a harmonic structure that connected the audienceto” the performance. The feature also provoked uniqueensemble interactions where performers could be “bossy”(CH) - by changing the functionality of others’instruments.

Figure 6: An iPad only

performance in duo form at

Everything/Nothing Projects.

ConclusionsThe rehearsal and performance series successfullyleveraged the percussionists’ expert knowledge to createcoherent, improvised musical works with the two iPadapps. YL commented that “we’re really developing asense of... motion. There’s definitely parts of it where...we’re definitely now in a new section”. Even thoughcombining the iPads with percussion setups made them“just another instrument” (CH), the performers revelled inthe unique sounds that were available to them. The“textures” (YL) of the field recordings, the “coolharmonies” (JG) of the diatonic tunings, and the

surprising “organ” (JG) timbre of low note played withthe delay function were all noted by the performers. Thenetwork features were particularly e↵ective in producing“cohesive” (CH) improvisation, but also in provoking new“bossy” (CH) ensemble interactions.

Following the premiere performances, the two works wereperformed as a set by Ensemble Metatone at a publicrecital in Canberra in August 2013 and at theElectrofringe 2013 festival in Newcastle, Australia.

References[1] Cahn, W. L. Creative Music Making. Routledge, 2005.[2] Hayden, S., and Windsor, L. Collaboration and the

composer: Case studies from the end of the 20thcentury. Tempo 61, 240 (2007), 28–39.

[3] Hook, J., Green, D., McCarthy, J., Taylor, S., Wright,P., and Olivier, P. A vj centered exploration ofexpressive interaction. In Proc. CHI 2011, ACM Press(2011), 1265–1274.

[4] Ren, Z., Mehra, R., Coposky, J., and Lin, M.Designing virtual instruments with touch-enabledinterface. In Ext. Abstracts CHI 2012, ACM Press(2012), 433–436.

[5] Schick, S. The Percussionist’s Art: Same Bed,Di↵erent Dreams. University of Rochester Press, 2006.

[6] Wang, G., Oh, J., and Lieber, T. Designing for theiPad: Magic Fiddle. In Proc. New Interfaces forMusical Expression 2011, University of Oslo (2011),197–202.