Two Types of Listeners?Marie Nilsenová (Tilburg University)

1. BackgroundWhen you and I listen to the same utterance, we may not perceive the linguistic (e.g., accenting, the asserting vs. questioning attitude) and paralinguistic (e.g., emotions) information in the signal in the same way. Specifically, we may differ in how much prominence we would assign to the complex pitch information included in the speech sound.

As experiments on the perception of complex tones with a missing fundamental (F0) show, listeners can be categorized into two groups, depending on how they asses the relative pitch of a sound:

• F0-listeners pay attention to the virtual pitch calculated by their auditory system. (For each component of a complex tone, the system calculates missing lower harmonics. It looks for a subharmonic match - a detected prominence.)

• Spectral listeners focus on the energy at certain (low) levels in the signal, I.e., on the pitch of a single spectral component.

With respect to the perception of music, F0-listeners seem to prefer short, sharp and impulsive tones (drums, guitar, piano, trumpet, flute); spectral listeners, on the other hand, tend to choose sustained tones and spectral information (lower-pitched strings, brass, organ or the signing voice).

Schneider et al (2005) used MRI and MEG to look for a neural basis behind these two types of pitch perception and found that F0-listeners demonstrated a leftward asymmetry in the lateral Heschl’s gyrus, while spectral pitch listeners possessed the opposite asymmetry.

So far, the bimodal distribution found among listeners has not been linked to the perception of information in speech stimuli, though a link to language learning has been established.

In speech, information about the emotions and attitudes of the speaker can be conveyed both by F0 modulation, as well as by changes in the timbre. For example, F0 changes are typically associated with the speech act status of an utterance (question, assertion) and with the epistemic state of the speaker (certainty about the truth of the utterance), viz. Safarova (2006), while timbre has been linked to speaker’s emotions, especially milder affective states (e.g., tense voice and anger, breathy and creaky voice with boredom or sadness), viz. Gobl & Ni Chasaide (2003).

We hypothesize that virtual pitch listeners would be more sensitive to the information contained in the F0 variation, while spectral pitch listeners would perceive changes in the spectrum as more salient.

• Research question:

Are there systematical differences among listeners in terms of how they perceive information encoded in pitch?

• Hypothesis:

F0-listeners judge speech stimuli more consistently on the basis of the perceived F0 changes than spectral listeners.

2. Methodology

Pilot study

A pilot study with 20 Dutch native speakers (10 musical professionals and 10 non-professionals, with an equal distribution of gender) and 12 complex-tone stimuli (each composed of two tones comprising two harmonics) confirmed the hypothesis. Participants who were identified as spectral pitch listeners were less consistent in their judgments on 16 speech-pitch pairs (with pitch extracted from the speech recording; 32 stimuli in total) for the presence of two emotions and two attitudes than participants who were virtual F0 listeners.

Primary study

An experimental study with two tasks:

1. Assessment of the listener type, based on a perception task with 36 synthesized complet two-tone stimuli (18 ambiguous and 18 control combinations).

2. Perception task with speech and sine-wave stimuli: 16 intonationally variable renderings of a complex sentence (‘Al draagt de aap een gouden ring het is en blijft een lelijk ding’) recorded by a female actor + 16 synthesized (hummed) pitch contours extracted from the speech stimuli.

Participants: Thirty-four Dutch native speakers (10 male, 24 female) without professional musical training.

Procedure: Participants first performed the listener-type categorical-judgment task during which they indicated whether they perceived a set of two-tone combinations to be falling or rising. Each combination was presented twice. Subsequently, they evaluated the speech and pitch-contour stimuli on a visual analogue scale for the presence/absence of the emotion ‘opgewekt’ (“happy”).

3. Conclusion & DiscussionThe listener type was calculated as

p = (f0 - fsp)/(f0 + fsp); where

f0 = listener selected the virtual F0 melody

fsp = listener selected the spectral melody.

There was no bimodal distribution of p;

instead, we found a non-normal distribution (M=.51, S.D.=.69; Shapiro-Wilk’s p=.001) of the score among our listeners with the listener type clearly skewed in the f0 direction.

Figure 1: The distribution of listener type, measured in the interval <-1; 1>, with absolute spectral listeners scoring -1 and absolute virtual F0 listeners scoring 1, was skewed in the direction of F0-type listener.

There was no (predicted negative) correlation between listener type and consistency score on the speech/pitch-contour VAS task (r=.036, p=.84).

An assessment of the methodology employed in the experiment (for both tasks) is currently under way, as well as an additional testing in other languages.

Contact InformationMarie Nilsenová

Faculty of ArtsPO BOX 90153

5000 LE TilburgThe Netherlands

Phone Number: +31(0)13 – 4662639E-mail: m.nilsenova@uvt.nl