Human-Computer Interaction for Music

Workshop

COMP 250October 16, 2012

Collaborative Tabletop Performance

•Performers can swipe the tabletop to change the interface or instrument that he/she is using

•Supports any instrument that can be turned into a touch-screen interface: turntables, digital MPC, keyboard, reacTable, Audiopad, drum pad…

Dan Afergan

Collaborative Tabletop Performance

•Performers can send their instrument to another surface by pinching + “throwing” towards another performer

•Performers can share an instrument with a full hand swipe to someone else

•Ability to record/playback instruments or entire performances

Sonic Ball• A ball with built-in sensors that track velocity, height, spin, impact,

bounce etc. so that these values can be mapped on to musical controls.

• Players from one to any number wear Bluetooth head sets with microphones that allow them to hear the music and to communicate with each other (and to rap, sing, etc. along the with music). The music can be broadcast only to the headsets or can be sent to external speakers.

• The prototype will use a basketball (and thus build on all the physical skills already in place), but the technology will be adapted to soccer balls, tennis balls, footballs, bowling balls, etc. Can be made part of all traditional games using those balls or developed into new games that take advantage of the musical interface.

Prof. Auner

Sonic Ball• Any kind of music can be used, but the protoype will use dubstep. The ball

at rest produces a mellow ambient dubstep groove; when it is thrown the sounds get brighter as it speeds up (then duller as it slows down).

• When you throw it high, more layers are added (and then subtracted as it falls)When you catch it there is a cool percussive effect--break beat or drum fill; when it bounces there are single percussive effects. More motion, faster motion, harder impacts, higher trajectories, faster bounces, etc. produce more intense musical effects.

• The ball and headsets will charge on a wireless charging mat, which will also have all the controls (on/off switch), headsets or external speakers, etc.

• Rationale: The point is to add music to physical activities to increase skills and coordination, social bonding, to better understand physical phenomena through the musical mapping, integrate dance with sport, and to have fun.

Heart-rate Based Dynamic Tempo Modules

D.F.Brown – HCM Workshop Proposal

Rationale / Input• Rationale: An average resting human heart rate lies somewhere

within 60-100 BPM. Most musical ‘beats’ run within a spectrum of 40 – 200 BPM. Depending on how a heart rate is broken-down as a beat (heart BPM can be multiplied by two as one musical ‘beat’), the two lie in a remarkably similar realm. I find the beat of our hearts intrinsic in our experience and production of music. I do not think the history of human music could have developed far from the timing within the ever-present biological metronome of our heart. I’d like to find a method of bringing the less-controllable biological timing of our hearts into the process of music production and experience, in order to break away from strict timing structure towards a more abstract, ephemeral form.

• Inputs: Digital heart rate monitor, and one or several human controlled digitally alterable inputs (midi-controller, MPC, Kinect).

Interaction / Evaluation• Interaction: A digital musical interface with programmable functions

is loaded with a bank of drum beats, treble melodies, bass lines, and whatever else can be strategically managed by the specific digital device. The tempo of each sample, however, will increase or decrease in real time in correspondence with the heart rate of the performer, being digitally monitored and fed into the system. The result, in theory, would create a more biologically dynamic time-structure in the musical performance, directly correlated to and impacted by the physical experience of the performer in process.

• Evaluation: This system needs to be experimented with to see how the resulting music may sound with the available technology. There may be issues with lag-time that I am unaware of, perhaps not. I believe the system will have the desired effect if one can effectively hear changes in the music as the performer is evidently taxed or relaxed by his physical or psychological process.

Washboard app for iPad

Workshop Proposal by

Marci Cohen

Rationale: takes advantage of:• Interest in folk music• Ubiquity of iPads• Obscurity of real washboardsInteraction: • Screen has graphic of real washboard• User holds iPad vertically in lap, moves fingers

vertically along surface to trigger samples of real washboard

• Sound varies with speed of scratching

Human-Computer Interaction for Music

WorkshopShawn Fitzpatrick

Running Music Interface

Rationale: It is important for runners to keep a consistent pace when running long distances. This music interface will help runners maintain their desired pace by changing the tempo of their music.

Input: Runner’s desired pace, GPS coordinates used to calculate the runner’s speed

Running Music Interface

Interaction: The user changes the tempo of the song based on their running pace. If the user is running faster than their desired pace, the music slows down. If the user is running slower than their desired pace, the music speeds up.

Evaluation: Analyze how well the user adjusted their pace to the tempo of the music. If the tempo speeds up, the runner should speed up. If the tempo slows down, then the user should slow down.

Audience: Long-distance runners who would like to keep their pace through music

Suppose

I'm a software engineer.

Are my tests passing?How many rps are we getting?Are 5xx errors spiking?

Goal:

Know without distraction.

Eli Fox-Epstein

But isn't this justvanilla sonification?

"... without distraction"Can't distract me from my music.Can't make my music sound bad.

How:Figure out what user is hearing.Accompany that with data.Adjust to user's agitation.

Kathleen Kuo10/16/2012

Why create the Songboard? Can be placed in classrooms (like traditional

whiteboards) for educational use, or at home for recreational use

Supplement traditional music education by allowing teachers to show notation, rhythm and pitch in real time

Grant composers the ability to play any instrument, no longer limited to just piano, and have the notes instantly appear

Easy and fun sound- and touch-based input for musicians and composers of any age

How does the Songboard work? Frequency-sensitive sensors can be attached to or

placed near an instrument and capture pitches to display on the Songboard in real time

Touch-screen technology or projection + computer allow the teacher/musician/composer to create, alter, or delete notes on the board

Option to create and view music without having to write, click a mouse, or touch a screen

Potential to be used in multimedia and art installations on top of classroom, recreational use

CAR RADIO ADJUSTER FRANCINE LALOOSES

Rationale: To notify driver through music if s/he is driving slower/faster than normal. People tend to pay attention to music playing when driving.

Input(s): Driver’s gaze and eye movements if adjusting radio station, spoken words if controlling the radio through voice commands, button clicks of radio station presets, driver’s foot control as s/he presses the gas pedal.

Interaction: Car radio adjusts the music tempo being played. If driving faster than normal, music will be unrecognizable. <Confusion>If driving slower, music slows down to a crawl. <Frustration> Assumes driver will alter gas pedal behavior upon emotional states.

Evaluation: System will adapt to driver’s normal speed and provide feedback through music tempo. Success based on driver driving at consistent speeds on roads.

Audience: Drivers on the go who listen to the car radio.

Dylan PortelanceComp 250 - HCM

Using Genetic Algorithms for Music Composition

Start with population of audio samples.Use some kind of fitness function to select the

‘fittest’.Population reproduces and mutations occur.The fittest survive.Repeat until desired effect.

Beste Yuksel

Proposed Method

Ramdom sample of rhythms.Fitness function determined by mood of user

(fNIRS or other physiological measure)Those rhythms which trigger the desired

emotional response reproduce.Continue for many generations until achieve

desired musical output or hit limit.