music for interactivityInstrument

Structures of Music

Interactivity

Anders-Petter Andersson, Interactive Sound Design, Kristianstad University, 4 Sep 2012www.musicalfieldsforever.comwww.RHYME.nowww.interactivesound.org

what I do Composer in interactive art group MusicalFieldsForever.com

Teacher at the Interactive Sound Design BA Programme at Kristianstad University, interactivesound.org

PhD student in Musicology at Gothenburg University Researcher RHYME:

Improve health for persons with severe disabilities, through use of empowering “co-creative tangibles”RHYME.no

overview What do we listen to in music? What is the materials of music? What is it made of? What are the structures of music? How can we use music when designing interactivity? Conclusions Links

what do we listen to in music? Do I know the song? Do I like it? Why? Why not? What do I do when listening to the song?

what do we listen to in music?

what do we listen to in music?

∙ Mamma Mia∙ Original Abba, http://www.youtube.com/watch?v=unfzfe8f9NI&feature=fvsr

what do we listen to in music?

∙ Mamma Mia∙ A-teens version:∙ http://www.youtube.com/watch?v=rZhaIWBTAfA&feature=related∙ Punk: ∙ http://www.youtube.com/watch?v=9Y7m9bx7M14∙ Allan Braxe Remix2009: ∙ http://www.youtube.com/watch?v=TW8e2pjgY38∙ Electric-K hard tech edit:∙ http://www.youtube.com/watch?v=iGS7bCnUeME∙ Black Ingvars:∙ http://www.youtube.com/watch?v=gG7th1-1N4U∙ Sydney youth orchestra:∙ http://www.youtube.com/watch?v=isy7iDoI3rA

what do we listen to in music? What are the instruments playing? Has the song a rhythmic beat or not? Who is singing? What do they sing about?

what do we listen to in music? Can you expect what will happen next in the music? How is that possible?

what do we listen to in music? How is the song structured over time?

what do we listen to in music? What is a genre? Socially and culturally negotiated

instrument How is the sound produced? What instruments are playing?

instrument Idiophones

∙Xylophone, Mbira, produce sound by vibrating themselves ∙Sachs-Hornbostel musical instrument classification system, 1914

instrument Membranophones

∙Drums or kazoos, produce sound by a vibrating membrane

instrument Chordophones

∙Piano, hurdy gurdy, guitar, which produce sound by vibrating strings. Pluck, bow, struck, blow (Aeolian harp)

instrument Aerophones

∙Pipe organ or oboe, produce sound by vibrating columns of air. Wind, Brass, Free (bull roar).

instrument Electrophones

∙Electric actions, Electric amplification, Radio electric oscillation in synthesisers, Theremines, Computer programme circuts

filtreringer.dk

instrument Electrophones

∙Sequencer, drum machines, step sequencers, interactive environments Max/msp

instrument Musical gestures

iterative

impulsive

sustained

complex

instrument Ensemble

∙Rhythm section, Accompaniment, drums, base∙Solist∙Chord instrument (piano, guitar, horns, choir)

∙Register∙Soprano, alto, tenor, base

structures of music Fundamentals

∙Attack, sounding body, ending

∙Melody, melodic theme, figure

∙Scale, Mode/Modality

structures of music Fundamentals

∙Harmony, Chords, Modality

structures of music Fundamentals

∙Rhythm patterns and metrics/time Matrix

structures of music Fundamentals

∙Sound, tone color, timbre∙Effects, reverb, echo, distortion

structures of music Fundamentals

∙Genre, Socially constructed, cultural∙Style in music∙pop, folkmusik, techno, jazz +++

structures of music Songlines - singing the land

∙Dreamtime∙Language = singing∙singing = walking

structures of music Nomad foot bells, Namibia

∙Music instrument, dance, status/identity, mobile money

interactivity What can I use from music when designing interactivity? How can I re-design linear music for interactivity?

interactivity Interactive music composition model

∙ Sound Node∙ Composition Rule∙ Narrative structures∙ Interactions

interactivity Bop-IT

∙ InteractionsBop It, Flick It, Spin It, Pull It, Twist It

∙ Narrative structuresVerse, refrain (Pass It!), stick

Vox Bop, Beat Bop, Solo to a beat, Pass It, One-on-one

interactivity Bop-IT

∙ Composition Rule

Pull It melody up, if interaction then melody down

Bop It basedrum, if interaction cymbal

Flick It feather, if interaction feather + voice

Spin It spinning low, if interaction high

Pull It whistle up, if interaction down

Twist It rattle, if interaction rattle + base

Feedback voice over “you win”, “you loose”

Increasing tempo och Pulse with use

interactivity Bop-IT

∙ Sound NodesBasedrum, cymbal, feather, voice, spinning wheel low, high, whistle up,

down, rattle, rattle base, voice over

interactivity Electroplankton

∙Matrix, Graphics X/Y∙Punch cards∙Music Box, Pitch and time

interactivityElectroplanktonInteraction with hanenbow plant

∙Change angle of leaves, launch speed of planktons

Narrative structure∙Minimalist rytmic patterns, puls from “plopp” sound in water, plant blossom life cycle

Composition Rules∙Marimba sound, color, timbre change with activity Green/soft, yellow/harsh, red/base tone

Sound node∙Marimba, pentatonic scale, hard, soft, base, water “Plopp”, blossom sound not pentatonic

interactivityElectroplanktonInteraction with Volvoice filters

∙Record sound, play

Narrative structure∙2 bar loop, graphical form illustrates sound effects

Composition rule∙Blow up-record, release-erase, filters: pitch up change, down, backwards, cut up, flanger, distortion, chorus, ++

Sound Node∙User recorded sound

interactivity Electroplankton Interaction with Sun-animacule

∙ Place and move plankton eggs

Narrative structure∙ 2 bar loop, grow plankton, emit light, night-dawn-day-dusk

Composition rule∙ Sun’s low pitch if Left, high pitch if right

Sound node∙ Bell sounds pentatonic scale

interactivityTenori OnInteraction

∙Step sequencer, time: left-right∙”Free painting”vertical-amplitude, horisontal-pitch ∙Bouncing mode∙Push mode, pads∙Instruments: base, drum, melody∙Light on back for audience!

Narrative structure∙Sound and light animations

Composition rule∙Light and beat synchronised to beat

Sound node∙Midi, samples, synthesised

ORFI

39

bend sensor

lighting

speakers

dynamic graphics

microphones

ORFI

interactivity ORFI

∙ Inspired by traditional music ensemble improvisation

Interaction in the Jazz genre∙ Interaction with tactile mobile soft cushions in three groups

Narrative structures∙ 10 bar jazz blues

Composition Rules∙ Synch to beat, variation and modulation in horn riffs, if

overlapping interaction then growling, if breaks then soft nodes in Dorian mode, if synch to beat variation in horns

Sound Nodes∙ Saxophone Dorian mode, growl outside Dorian, pads, horns

riffs, walkingbase and drums

interactivity ORFI Interactions in Mini genre

∙ Interaction with tactile mobile soft cushions in three groups, sound play when activated

Narrative structures∙ Minimalist generative melodies

Composition rules∙ Melodies have identity (instrument, melodic, behaviour),

mutates and synchronise when played together with another node

Sound node∙ Marimba sounds, pentatonic scale

interactivity ORFI Interactions in Tati genre

∙ Interaction with tactile mobile soft cushions in three groups

Narrative structures∙ Film scenes, the loby, restaurant, beach

Composition rules∙ if overlapping interaction then high pitch, harsh timbre,

imperative spoken language, if breaks then soft nodes

Sound node∙ Spoken language, sound effects, environmental sounds,

music

43

the user can create many interpretations

44

open to many forms of interaction and activity levels

45

open to many forms of interaction

46

bend sensorslighting

speakers & transducer

dynamic graphics

microphones

Wave

accelerometers

touch

camera

projector

Wave

47

bend sensorslighting

speakers & transducer

dynamic graphics

microphones

Wave

accelerometers

touch

camera

projector

interactivity Wave

∙ Crossmedial (tactile, generative music, dynamic moving visuals, game) environment for social play in families with children with severe disabilities

Interaction (SuperCollider, atmel ”arduino” micro controller)

∙ Interaction with 7 soft arms with sensors (touch, microphone, camera, projector,

accelerometers, bend), lights, speakers and transducer/vibrator

Narrative structures∙ Dynamic beats (BPM), Delay loop, 3 slot Recording buffer/env (FIFO), Synch to

moving graphics projection

Composition Rules∙ Mic with ring modulation, Mic to buffer, big Accelerometer arm pitch down,

small arm pitch up, Bend speed up beat, Camera switch on grapichs, Touch lights in bubble field play melody low-high notes, audiovisual rules moving up-down for pitch up-down

Sound Nodes∙ Melodic marimba sound, drums, noise, user recorded sound, sound effects

(delay, ring modulation, reverb), sound translated to tactile transducer vibrations

Conclusion Music is structured in time! Music can mediate other interaction gestures in time. Music can increase experience of identity, co-creation,

collaboration Traditional linear music can be re-used in interactive

media

anders@interactivesound.orgwww.musicalfieldsforever.com

links Interactive art group www.musicalfieldsforever.com

RHYME research project:  improve life quality for persons with severe disabilities, through use of “co-creative tangibles” www.RHYME.no

Interactive sound design www.interactivesound.org(and at www.facebook.com/interactivesound)

Open source hardware platform for creating sensors, joysticks www.arduino.cc

Max/Msp music programming environment www.cycling74.com

Pure data (Pd) open source music programming environment www.puredata.info

Lady Ada DIY community and manual for programming arduino www.ladyada.net

Toshio Iwai’s Electroplankton www.youtube.com/watch?v=28UDxIQiaIY

Iwai’s Tenori on www.youtube.com/watch?v=_SGwDhKTrwU

Sergei Jorda’s ReacTable interactive music instrument (Björk) www.reacTable.com

anders@interactivesound.orgwww.musicalfieldsforever.com

www.RHYME.no