Piano touch, timbre, ecological psychology, and cross-modal interference
description
Transcript of Piano touch, timbre, ecological psychology, and cross-modal interference
![Page 1: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/1.jpg)
Piano touch, timbre, ecological psychology, and cross-modal interference
Richard ParncuttCentre for Systematic Musicology
University of Graz, Austria
ISPS, 28-31 August 2013, ViennaInternational Symposium on Performance Science
![Page 2: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/2.jpg)
AbstractThe piano has a wide timbral range, and performance quality is often judged in timbral terms. Yet despite decades of research, there are still fundamental disagreements about the nature and origin of piano touch. Scientists (acousticians) maintain that the timbre of a single tone cannot be varied independently of its loudness. Performers, humanities scholars and concert audiences take the opposite for granted: timbre and loudness can be independently varied by gestural means. Both sides are right, but their implicit definitions of timbre differ, and both fail to clearly distinguish between physical measures and descriptions of subjective experience. Scientists assume that timbre depends only on physical sound parameters; but experiential parameters generally depend on concurrent input from other senses, the listener’s relevant knowledge and expectations, and immediately preceding and following events. The paradox of timbre disappears if we accept, based on empirical evidence, that timbre generally depends on input from more than one sensory modality (weak synesthesia). Embodied corporality and conceptual metaphors are the norm - not the exception. Gestural and ecological approaches to timbre perception pose existential challenges to disembodied cognitive orientations.
![Page 3: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/3.jpg)
Timbre in traditional psychoacoustics
American Standards Association (1960): “that attribute of sensation in terms of which a listener can judge that two sounds having the same loudness and pitch are dissimilar”
A negative definition!• Neither pitch nor loudness• The multidimensional remainder
![Page 4: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/4.jpg)
Timbre is experience!
• Traditional psychoacoustics: clear distinction between physical and experiential variables
• But many psychoacousticians talk about timbre as if it were physical
• I assume that timbre is always experiential
![Page 5: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/5.jpg)
Timbre depends on
Time domain– temporal envelope
Frequency domain– spectral envelope
![Page 6: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/6.jpg)
Vocal timbre
Time Domain:ArticulationTemporal envelope
Frequency DomainVocal Tract ShapeSpectral Envelope
![Page 7: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/7.jpg)
Timbre of musical instruments
• Mixture of time and frequency domain– Spectral flux: Different temporal envelopes of partials
• Absolute frequency– spectral & temporal envelopes depend on fo
– timbre learned for each absolute frequency
• Spectral interval structure– Clarinet, bowed violin: exactly harmonic– Bell: inharmonic– Piano, guitar: stretched harmonic
Spectral flux in a bass clarinet toneGrey & Moorer (1977)
McAdams (1993)
![Page 8: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/8.jpg)
Timbral spaceSpatial representation of timbral relationships
Method:• Choose some timbres• Rate similarity of all pairs• Multidimensional scaling• Result is usually 3-D, axes:
– onset, centroid, roughess…– depending on tested sounds
Interpretation:• Results difficult to generalise• Cognitive representation?• Distances in space depend on
exposure & familiarity McAdams et al. (1995), Psychological Research
![Page 9: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/9.jpg)
Ecological psychology of timbre• Environmental interaction• Experience as byproduct of interaction
(epiphenomenon)
Theory of James J. Gibson:• Affordances and information pick-up• Invariances of sound sources• Perceptual learning
Complementary to the cognitive approach Relationship to evolutionary psychology
![Page 10: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/10.jpg)
Distal versus proximal stimuliExternal versus internal focus of attention
In sport and music, performance may be better if attention directed to effects of movements - not movements themselves (Wulf & Prinz, 2001): • Golf: attention on goal, not movements• Pianists: attention to sound, not technique
Why?• Sophisticated unconscious motor control mechanisms • Perception of goal inseparable from proprioception
ImplicationPerception of timbre involves source perception
![Page 11: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/11.jpg)
Ecological-evolutionary foundationsof auditory perception
Assumption: Sounds are only interesting ( consciously perceived) if they carry information about environmental interaction that could affect survival or reproduction.
The affordances of a sound source • what it means to us • what we can do with it
Timbre reflects affordances!• Human voice• Lion footsteps, waterfall, car, gun…
![Page 12: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/12.jpg)
OPEN
BRIGHT
CLASSICALAIRY
VIBRATO
SPOKEN
SMOKEY
HARSH
PRESSED
CLEAR
NATURAL
BREATHY
LOOSE
NASAL
FORCEFUL
FIXED
pleasant
soft
dark
relaxed
yawning
held back
resonanteasy
round
black
swinging
support
forward
(Prem & Parncutt, 2007)
Words to describe the jazz voice
![Page 13: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/13.jpg)
“I fell in love with his voice. He is African American, and his voice has this depth, it is souly, in a wonderful way. He is like a gentleman, and has this warmth. He gives his voice space and that arouses a very balanced feeling in me. His sadness, his elegance, his joyfulness really come through, so that I can experience them.”
Affordances of vocal timbre
Data collected by Ella Prem
![Page 14: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/14.jpg)
Timbre is multimodal
• Performer can’t separate timbre (as experience!) from gestures used to achieve it
• Listeners share the performer’s experience (empathy)
How?• audiovisual mirror neurons (Kohler et al., 2002) • auditory-motor interactions (Zatorre et al., 2007)
![Page 15: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/15.jpg)
ProprioceptionPerception of position and movement of body
parts and associated muscular tension
Part of the somatic sensetouch, proprioception, temperature, pain
An aspect of corporalityAspects of music experience that depend on the bodies
of performers and listeners
![Page 16: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/16.jpg)
Weak synesthesiacross-modal interference
• All perceptual modalities: seeing, hearing tasting…• The norm - not the exception• Ecologically natural - cognitively strange
The limits of subjective analysisWe cannot completely separate…• input from different modalities: hearing, vision, touch• sensations within a modality (pitch, timbre).
![Page 17: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/17.jpg)
A non-musical example
When you taste yoghurt with a lighter (table-) spoon, it seems• thicker • sweeter• more expensive
Harrar & Spence (2013). The taste of cutlery. Flavour, 2: 21.
![Page 18: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/18.jpg)
Weak synaesthesia in music
• Musical pitch “rises” and “falls”• A piece “moves”• A voice is “warm”, “round”, “relaxed”
![Page 19: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/19.jpg)
Embodied cognitionin different disciplines
• Psychology: cognition depends on features of the physical body of an agent (Wilson & Foglia, 2011)
• Music psychology: dance, rhythm, aesthetics etc. (Leman, 2008).
• Humanities: conceptual metaphor, ideas in one domain help us understood in another (Lakoff & Johnson, 1980).
![Page 20: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/20.jpg)
Timbral range of the piano
One note: • Louder is brighter
Many notes:• higher is brighter• physical interactions among strings,
soundboard, internal resonances• dependence on timing, dynamics, pedaling
![Page 21: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/21.jpg)
The pianist’s “touch”A traditional psychoacoustic approach
Physics:• Hammer hits string in free flight• Only free parameter is hammer velocity (shank rotation?)• Fingertips hit the keys differently in legato and staccato touch
Perception:• Any effect beyond hammer-velocity effect is usually inaudible
due to masking(cf. “touch precursor”, “early noise”; Goebl et al., 2004)
![Page 22: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/22.jpg)
Piano touch: A synaesthetic approach
For the pianist:• All aspects of somatic sense: touch, position, movement,
temperature, pain…
For the audience:• Vision (perception of gesture)• Projected somatic sense (empathize with performer)
For both:Multisensory perception of the interaction between the instrument and the performer
![Page 23: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/23.jpg)
The feel of a piano
“Pianos and rooms are generally interdependent: anyone who has ever travelled with a piano knows that the same Steinway or Bösendorfer not only sounds different in different halls, but also seems to react differently in its mechanism. Indeed, the resistance of the key, over and above the measurable mechanical aspect, is a psychological factor”
Alfred Brendel (1976)
![Page 24: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/24.jpg)
The feel of a piano
For pianists, the physical resistance or bounciness of the piano mechanism depends• physically on temperature and humidity• psychologically on touch-sound relationship
(to which they are very sensitive)
![Page 25: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/25.jpg)
Multisensory piano timbreIn physics, spectral and temporal envelopes of an isolated piano tone cannot be changed independent of intensity
In psychology and experience, timbre involves interactions between:• performers’ proximal sensations (tactile, auditory, visual,
proprioceptive) • distal perception and cognition (performance space;
communication with the audience; cultural context)
These statements are not contradictory!
![Page 26: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/26.jpg)
Weak synasthesia in wind/string timbre
• independent control over pitch and timbre, but:
• intonation and timbre not psychologically separate (Ely, 1992; Geringer & Madsen, 1981; Platt & Racine, 1985)– musicians who play with good timbre are judged
by experts and amateurs to have good intonation and vice-versa
![Page 27: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/27.jpg)
Weak synesthesia in Renaissance choral timbre
Listeners, audiences want “authentic” timbre.Some confuse that with just intonation.
The sound is presumably optimal near 12-tone equal temperament (Devaney et al, 2011)
![Page 28: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/28.jpg)
Weak synaesthesia in acousmatic music
Acousmatic music is abstract, electronically synthesized sound from loudspeakers.Ecological approach to aesthetics: • Listeners constantly guess and imagine
sources or causes of musical sounds – as in everyday life
• Electronically generated sounds sound more “musical” if similar to familiar sounds
![Page 29: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/29.jpg)
Disembodiment of music theory and cognitive music psychology
• Music theory depends on the score• Cognitive music psychology depends on
abstract cognitive structures
The ecological solution:Study our experience of our physical interactions! Example: complex tones in speech and music (Terhardt, 1984).
![Page 30: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/30.jpg)
A new definition of timbre
1. Experiential2. Holistic and multimodal3. Quantitative and qualitative4. Proximal and distal
![Page 31: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/31.jpg)
1. Timbre is experiential
• Like pitch, loudness, and (in vision) color• Not a physical parameter!• Corresponds to physical states and events
![Page 32: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/32.jpg)
2. Timbre is holistic and multimodal
Timbre depends generally and intrinsically on• input from all senses: hearing, vision, touch, gesture• physical context: acoustics and appearance of listening space• temporal context – immediate, global• empathy: shared or projected knowledge, experience,
expectations• emotional reactions• associations
Our ability to separate these components is limited.
![Page 33: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/33.jpg)
3. Timbre is qualitative and quantitative
• complete description: both kinds of data• both are intrinsically vague and intangible • quantitative approach:
– multidimensional (axis labels are qualitative)• qualitative descriptors
– refer to environment, body, speech (Traube, 2004)
![Page 34: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/34.jpg)
4. Timbre is distal and proximal
• A complete description includes both• Traditional psychoacoustic approach is proximal• Combine with ecological source-based approach
Why? Timbre generally depends on imagined visual and tactile properties of sound sources, and past experience of those sourcese.g. different clarinet registers are all “clarinet”
![Page 35: Piano touch, timbre, ecological psychology, and cross-modal interference](https://reader035.fdocuments.in/reader035/viewer/2022062218/56816546550346895dd7be26/html5/thumbnails/35.jpg)
Solving the “touch” problem
• Redefine timbre: combine experiential psychophysics with ecological psychology
• Promote interdisciplinary interaction: humanities, sciences and performance
• Abandon philosophical materialism: (artistic) experience exists!