Cognitive vs. affective listening modes and judgments of music â€“ An ERP study · 2012. 6....

C

Ea

b

c

d

e

a

ARAA

KEMLEJALM

1

edmpc(ciiaorg

ST

0d

Biological Psychology 85 (2010) 393–409

Contents lists available at ScienceDirect

Biological Psychology

journa l homepage: www.e lsev ier .com/ locate /b iopsycho

ognitive vs. affective listening modes and judgments of music – An ERP study

lvira Bratticoa,b,∗, Thomas Jacobsenc,d, Wouter De Baenee, Enrico Glereana,b, Mari Tervaniemia,b

Cognitive Brain Research Unit, Institute of Behavioral Sciences, University of Helsinki, FinlandFinnish Center of Excellence in Interdisciplinary Music Research, Department of Music, University of Jyväskylä, FinlandInstitute of Psychology I, University of Leipzig, GermanyExperimental Psychology Unit, Helmut Schmidt University/University of the Federal Armed Forces, Hamburg, GermanyDepartment of Experimental Psychology, Ghent University, Belgium

r t i c l e i n f o

rticle history:eceived 23 January 2009ccepted 31 August 2010vailable online 17 September 2010

eywords:vent-related potential (ERP)usic perception

ate positive potential (LPP)arly right anterior negativity (ERAN)

a b s t r a c t

The neural correlates of processing deviations from Western music rules are relatively well known. Lessis known of the neural dynamics of top-down listening modes and affective liking judgments in relationwith judgments of tonal correctness. In this study, subjects determined if tonal chord sequences soundedcorrect or incorrect, or if they liked them or not, while their electroencephalogram (EEG) was measured.The last chord of the sequences could be congruous with the previous context, ambiguous (unusual butstill enjoyable) or harmonically inappropriate. The cognitive vs. affective listening modes were differen-tiated in the event-related potential (ERP) responses already before the ending chord, indicating differentpreparation for the judgment tasks. Furthermore, three neural events tagged the decision process pre-ceding the behavioral responses. First, an early negativity, peaking at about 280 ms, was elicited by chord

udgment processesestheticsikingusical preference

incorrectness and by disliking judgments only over the right hemisphere. Second, at about 500 ms fromthe end of the sequence a positive brain response was elicited by the negative answers of both tasks. Third,at about 1200 ms, a late positive potential (LPP) was elicited by the liking judgment task whereas a largenegative brain response was elicited by the correctness judgment task, indexing that only at that latelatency preceding the button press subjects decided how to judge the cadences. This is the first study toreveal the dissociation between neural processes occurring during affective vs. cognitive listening modes
and judgments of music.
. Introduction

Listening to music involves several processes of cognitive,valuative and affective natures. As a consequence of every-ay exposure to music even individuals without any formalusic training are able to implicitly encode the structures and

roperties of tonality and harmony which underlie most musi-al genres of Western culture, like popular and classical musicTillmann et al., 2000). The implicit encoding of those musi-al properties leads to the formation of expectations for thencoming sounds. If those expectations are broken, i.e., if thencoming sounds are incongruous with the previous context,
ffective sensations of tension are generated. Tension is releasednce expectations are fulfilled, resulting in affective sensations ofelaxation (Meyer, 1956). The cognitive reactions to musical incon-ruity have been extensively measured by asking subjects to rate
∗ Corresponding author at: Cognitive Brain Research Unit, Institute of Behavioralciences, P.O. Box 9, 00014 University of Helsinki, Finland.el.: +358 44 373 3297; fax: +358 9 191 29450.

E-mail address: [email protected] (E. Brattico).

301-0511/$ – see front matter © 2010 Elsevier B.V. All rights reserved.oi:10.1016/j.biopsycho.2010.08.014

© 2010 Elsevier B.V. All rights reserved.

the congruity of sounds with the previous context (Krumhansl,1990).

Various brain-imaging techniques have also been used recentlyto record the online responses of the brain of healthy subjects to vio-lations of musical properties (Besson and Schön, 2003; Brattico andTervaniemi, 2006; Brattico, 2006; Koelsch et al., 2000; Patel et al.,1998). Several studies using the event-related potential (ERP) tech-nique converged in identifying a family of early neural responseselicited at around 200 ms after a musical rule violation, includingthe early right anterior negativity (ERAN), and the mismatch nega-tivity (MMN). In electrical recordings, these brain responses consistof voltage changes over the scalp and appear as negative deflectionsof the ERP commonly peaking at frontal scalp regions. Typically theMMN is elicited by local violations of scale properties (e.g., uncon-ventional intervals between tones), and is generated mainly in theauditory regions of the right temporal lobe (Brattico, 2006; Leinoet al., 2007). In contrast, the ERAN is elicited by unusual chord suc-
cessions violating hierarchical rules of Western tonal harmony andit is generated mainly in the bilateral prefrontal lobes (Koelsch andSiebel, 2005; Koelsch et al., 2000; Leino et al., 2007; Steinbeis et al.,2006; for a discussion about differences and similarities betweenthe ERAN and MMN, see Koelsch, 2009). ERP responses occurring at
dx.doi.org/10.1016/j.biopsycho.2010.08.014

http://www.sciencedirect.com/science/journal/03010511

http://www.elsevier.com/locate/biopsycho

mailto:[email protected]

dx.doi.org/10.1016/j.biopsycho.2010.08.014

3 l Psych

eseca

cpaPTMcmaaotirsier

wrpsoiaaaiahaifaaesopmatDoamt

titSs(cgfvt

94 E. Brattico et al. / Biologica

arlier latencies, such as the N100 peaking at around 100 ms fromound onset, may also be modulated by musical properties; how-ver, the related brain evidence is comparably scarce and not fullyonvergent (Krohn et al., 2006; Kuriki et al., 2005; Tervaniemi etl., 2003).

Another important group of brain responses identified in asso-iation with musical rule violation consists of late long-lastingositive potentials of large amplitude peaking at around 300 ms,nd thus named P300, or at around 600 ms, and thus termed600, from stimulus onset (Besson and Schön, 2003; Brattico andervaniemi, 2006; Brattico, 2006). Differently from the ERAN andMN elicited under semi-active or totally passive experimental

onditions, the P300 and P600 are obtained only under experi-ental conditions in which subjects are attending to the sounds

nd performing a task related to them. These late brain responsesre hence supposed to reflect the conscious cognitive processingf a deviation from musical expectations formed on the basis ofhe previous context, and the context updating for the subsequentncoming sound events. Nevertheless, the issue whether brainesponses to harmonic or melodic incongruity are stable acrossubjects or variable across individuals has been largely neglectedn the MMN, ERAN or P600 literature (apart from effects of musicalxpertise) since those ERP responses have been typically computedegardless of the subjects’ behavioral ratings of tonal correctness.

Listening to music not only involves the cognitive processes,hich most frequently attracted the attention of music neu-

oscience researchers (see above), but evaluative and affectiverocesses as well (see Peretz and Zatorre, 2003). Listeners uncon-ciously assess the quality of the music heard according to a positiver negative value, on the basis of properties related to the musictself, such as formal structure, beauty, harmonic congruousness,nd also based on individual characteristics of the listener, such asttitudes, personality, motivation and prior knowledge (e.g., Northnd Hargreaves, 1997a,b). One simple type of evaluative judgments represented by liking: along with processing the perceptual,ffective and formal contents of a musical piece, listeners assessow much they like or dislike the piece. Liking judgment is typicallyccompanied by a hedonic sensation, which in aesthetic contextss mainly conscious to the subject (Berridge and Robinson, 2003;or discussions of liking within the musical context, see Bratticond Jacobsen, 2009, forthcoming). Dissociations between affectivend cognitive ratings of music have been hinted at. In a behavioralxperiment using the dichotic listening procedure, one group ofubjects judged whether melodies sounded correct or not and a sec-nd group of subjects judged whether the same melodies soundedleasant or not. The results showed that pleasantness ratings ofelodies predominantly involved the right cerebral hemisphere,

s indicated by the left-ear preference, whereas no brain lateraliza-ion occurred during correctness ratings (Gagnon and Peretz, 2000).istinct neural mechanisms for affective vs. cognitive processingf music were also suggested by neuropsychological evidence ofpatient with bilateral temporal lobe lesions having preservedusic emotion and appreciation combined with disrupted percep-

ual musical abilities (Peretz et al., 1998).In other sensory modalities, researchers have explored the rela-

ion between liking judgments and cognitive descriptive ones. Fornstance, the liking judgments of words were delayed as comparedo the lexical decision ratings of the same words (Mandler andhebo, 1983). Similarly, the evaluative ratings of paintings werelower than judgments of prior encounters of the same stimuliMandler and Shebo, 1983). Thus, liking seems to require more
omplex and resource-demanding processes than knowing, sug-esting that those two different processes may be independentrom each other. A subsequent study involving behavioral ratings ofisual geometric abstract black and white pictures as well as elec-rophysiological recordings confirmed that evaluative processes in
ology 85 (2010) 393–409

the visual domain are relatively slow, and, in part, occur later thandescriptive processes (Jacobsen and Höfel, 2003). Fast and slowroutes to affective evaluative processes have been hypothesized(Scherer and Zentner, 2001). The fast primary route to the positive(or negative) appraisal or evaluation of an event relies on the quickand automatic brain responses. Those responses are below the levelof consciousness and likely originate in the brainstem and primarysensory cortices. The slow responses, instead, occur mainly in theevolutionarily more recent structures of the brain, such as the pre-frontal cortex, and lead to the conscious appraisal and liking of apiece of art or an everyday object (Jacobsen et al., 2006). In sum,the results obtained with visual or linguistic stimuli call for furtherinvestigation of the interrelation between cognitive and evaluativeprocesses also in the musical domain.

The ERP method allows one to determine the dynamics of infor-mation processing in the cerebral cortex by analyzing the mainvoltage changes time-locked to the stimulus or task of interest(Luck, 2005). In our experiment, as a first main aim, we wereinterested in determining whether a top-down affective vs. cog-nitive listening mode adopted during two judgment tasks wouldby itself modulate the sensory processing of sounds even beforethe judgment decision. As a second main aim, we wished to inves-tigate whether different sub-processes involved in assessing theliking vs. the tonal correctness of the same musical material wouldbe reflected in the electric brain responses. Finally, as a thirdexploratory aim, we wished to assess the amount of variability andsubjectivity in affective and cognitive judgments of music and intheir underlying neural mechanisms.

According to the valence lateralization model, positive emo-tions are controlled by the left-hemispheric frontal cortex, whereasnegative emotions rely on the intact functioning of the right-hemispheric frontal cortex (Davidson, 1993; Fox, 1991; forsupporting results with musical stimuli, see Altenmüller et al.,2002; Flores-Gutierrez et al., 2007; Gagnon and Peretz, 2000;Schmidt and Trainor, 2001). We hence hypothesized that the holis-tic mental strategies of affective vs. cognitive modes of listening tomusic would modulate the amplitude and lateralization of the earlyauditory-cortex responses to each chord already before the time ofjudgment decision. We further predicted that the affective compo-nent of the liking judgments, i.e., the associated hedonic enjoymentof the listened music, would be reflected in the lateralization of thebrain responses. Moreover, we expected to obtain enhanced latepositive brain responses to liking judgments similarly to what wasfound in other sensory modalities and cognitive domains. Specif-ically, a late positive response termed the late positive potential(LPP) was elicited during the performance of evaluative classifica-tion of faces, visual art, objects, words, and similar (e.g., Cacioppoet al., 1994; Crites et al., 1995; Jacobsen and Höfel, 2003; Schupp etal., 2000).

The brain responses associated with liking vs. correctness rat-ings may also provide evidence for the hypothesis according towhich affective judgment processes precede cognitive ones andrequire only primitive stimulus analysis, as suggested by Zajonc(1980; see also related findings by Peretz et al. (1998) on recogni-tion of basic emotions in music). In our study, if the first brain eventselicited by the ending chords are most prominent in associationwith the liking judgment task, then the Zajonc (1980)’s hypothe-sis would be confirmed. In the opposite case, if the incorrectnessof the final chords is neurally detected mainly at early latencies(e.g., at 150 ms as expected from the ERAN findings) and onlysubsequent potentials characterize the liking judgments, contrary
hypotheses suggesting the primacy of cognitive processing on lik-ing would be supported. A third alternative hypothesis derives fromthe most recent theories of aesthetic processing, which includeseveral stages from initial emotional reactions, to cognitive pro-cessing, and finally late conscious appraisal (Jacobsen and Höfel,

l Psych

2acse

wcnarmtsott(tfvrttcoi

2

2

trytdpt

2

eca3

imaecccoecagcs1l

wiacicmT

E. Brattico et al. / Biologica

003; Leder et al., 2004; Scherer, 2004). According to this thirdlternative hypothesis, we would expect early electrophysiologi-al markers reflecting emotional and initial feature analysis of theounds and late markers of conscious discrimination of tonal prop-rties and of affective decisions.

In the current study, several factors affecting music processingere carefully controlled for. The musical stimuli were purposively

omposed for the experiment and they were hence completelyovel to the subjects in order to minimize effects of familiaritynd episodic memory on the behavioral and electrophysiologicalesponses. Following the recent line of research on early ERPs tousical stimuli (see, e.g., Koelsch et al., 2000; Patel et al., 1998),

he stimuli were chosen to consist of isochronous (i.e., with theame rhythm) sequences or cadences of chords following the rulesf Western tonal music but otherwise without a direct associa-ion with a specific musical genre (like classical or popular music)o avoid retrieval of personal and social attitudes or associationse.g., Jacobsen and Höfel, 2003; North and Hargreaves, 2007). Fur-hermore, we preferred cadences rather than excerpts of musicrom commercial CDs because of the difficulty with rhythmicallyarying sounds to time-locking them with the electrophysiologicalesponses. The level of individual variability in judgment outcomeshat could be tested with the current experimental paradigm washus compellingly restricted to the mere manipulation of levels ofhord congruity with formal harmony rules. Hence, our third aimf studying the subjectivity of the liking and correctness judgmentsn music is intended to be exploratory in nature.

. Methods

.1. Subjects

12 native German speaking volunteers (mean age 22.5 ± 7.3 SD; 2 males) par-icipated in the study. They had minimal musical education (with 4 subjects havingeceived some instrumental lessons during childhood; mean musical training: 2.4ears). At the time of the experiment the subjects were not active in choir or orches-ra and were not receiving any formal training in music. All the subjects, however,eclared to listen to music daily with 5 subjects listening to records more than 2 her day. Three additional subjects were discarded from data analysis because ofechnical reasons (too many artifact contaminated trials in the EEG recording).

.2. Stimulation paradigm

180 short musical excerpts were composed for the experiment by an experi-nced musician (EB; for examples of the excerpts, see Fig. 1a). The excepts wereadences consisting of five musical chords where the first four chords lasted 500 ms,nd the ending chord lasted 1000 ms. The total duration of each cadence was hence000 ms, with the manipulated chord occurring at 2000 ms.

In the chord cadences, the ending chord varied in three different ways, accord-ng to its congruity with the preceding musical context, as defined by Western tonal

usic theory of harmony (Piston, 1962): it could either fully fit with the contextnd conform to the rules of music theory that recommend the tonic triad at thend of a musical piece (‘congruous’), or it could only mildly fit with the precedingontext, being it a less used chord at the final position of a musical cadence. Theseadences were termed ‘ambiguous’ as they meant to generate more than one per-eptual interpretation, similarly to ambiguous stimuli used, for instance, in studiesf visual awareness (cf. Rees, 2007). Hence judgments of ambiguous cadences werexpected to alternate between each individual interpretation. Finally, the last chordould be strongly incongruous with the preceding context, by introducing suddenlydistant modulation or an unusual dissonant chord at the final position (‘incon-

ruous’). For examples of the three types of sequences, see Fig. 1a. The ambiguoushord was chosen because it could also be liked by listeners, as it created suspendedensation and introduced an aesthetically pleasurable degree of novelty (Berlyne,971). Accordingly, the strict correlation between music-theoretical congruity and

iking was meant to be avoided.The succession of harmonic functions preceding the ending chord in the cadence

as of two types: IV, II, VI, VII or I, IV, II, V. Each of the three ending chords (congruous,ncongruous, ambiguous) could be of six types, varying in their pitch class content
nd harmony function. From these two cadence types and 18 ending chords, 10adences were composed, varying the chord inversions and registers of each chordn the cadence. The 10 cadence types were then combined with the 3 different endinghords, resulting in 30 chord cadences. Each of them was transposed over 6 majorusical scales from B flat major to E flat major, resulting in a total of 180 cadences.
he classification of the chords as ambiguous, congruous or incongruous was based

ology 85 (2010) 393–409 395

on music theory principles determining which chords are legal in the ending posi-tion of a cadence and which are completely unacceptable. Furthermore, the usabilityof the cadences for the present experiment was piloted with 4 expert judges withexperience in music psychology experiments and minimal musical background,who rated the chords according to their correctness or incorrectness. Two cadencesused in the pilot experiment were discarded as being inappropriate for the experi-ment. Three other expert judges, with professional musical background (one flutist,one violinist and a composer) and experience with music psychology experiments,then listened to the resulting stimulus set and agreed with 100% consistency in theclassification of chords as being congruous, ambiguous or incongruous.

In the experiment, each cadence was presented twice (360 trials): once for thecorrectness judgment task and once for the liking one. The cadences were presentedinto 4 blocks each including 90 trials, and each lasting 12 min. The total durationof the recording phase of the experiment was hence less than 1 h (48 min plusshort breaks between the blocks). The chord cadences were generated with the Rea-son software (a sequencing program that contains a virtual sampler; PropellerheadSoftware, Stockholm, Sweden). The sounds (mono, 44100 Hz sampling rate) weresampled from a grand piano (volumes were kept natural and thus not fully cali-brated) and they were edited with cutoff frequency and EQ. The grand piano wasmultisampled: every third key was sampled (an octave consists of four samples).

2.3. Experimental procedure

Two tasks were adopted: a correctness judgment task and a liking judgment task.For both tasks, subjects were asked to give binary forced-choice responses by but-ton press: ‘correct’/’incorrect’ for the correctness judgment task and ‘like’/’dislike’for the liking judgment one. The buttons for the negative and positive answers to thetwo tasks were assigned to two button pads, one held in the right hand and the otherin the left hand. The button assignments were counterbalanced across subjects. Thesubjects were offered 18 practice trials (9 for the liking judgment task and 9 for thecorrectness judgment one, and each of them having 3 congruous, 3 incongruous and3 ambiguous cadences) before the preparation of the electrophysiological measure-ments with the same trial structure as in the main experiment. The practice trialsalso served to select subjects on the basis of their task performance: one subjectwho could not give correctness and liking judgments of the musical stimuli inde-pendently of each other as determined by self-report was excluded from furtherrecordings.

In the actual experiment, the trials were pseudo-randomized with the constraintof a maximum of 3 trials of any kind (correctness, liking, right answer, left answer) ina row. Ambiguous stimuli were randomly separated into ‘correct’ and ‘incorrect’ (and‘like’ – ‘dislike’) and handled like this for the randomization. The stimuli occurredonly once during the first half of the experiment, with a repetition buffer of about10 stimuli in the part preceding the second half of the experiment. The combinationof the first half and second half presentation and the contribution of a stimulus to atask were counterbalanced across participants.

A visual cue appeared on the computer screen in front of the subject 1500 msbefore the presentation of the first chord and lasted for 1200 ms. This cue was of twokinds and served to guide the subjects on which task they had to perform on theincoming sounds. Specifically, prompted by a large ‘K’ appearing on the computerscreen, participants judged the correctness or incorrectness of the excerpt (correct-ness judgment task; ‘K’ stands for ‘Korrekt’ in German), and similarly, prompted bya large ‘M’, they judged whether they liked the excerpt or not (liking judgment task;‘M’ stands for ‘Magst’ in German). Subjects were asked to keep watching the loca-tion of the screen in which the cue was given and not to blink during the sequenceand before they answered. They could blink after they had answered and beforethe start of the next trial. The silent interstimulus interval (ISI) during which thesubjects were asked to give the behavioral response lasted 5000 ms (see Fig. 1b). Nofeedback was given. The subjects were instructed to answer faithfully and speedily,but without time pressure.

2.4. Data analysis

2.4.1. Behavioral dataThe reaction times (RT) and the frequencies of answers according to task and

stimulus categories collected during the EEG measurements were also studied. Toassess the effects of the two different tasks, the tonal properties of the chords and theanswers by the subjects, we carried out two- and three-way ANOVAs with Task (lik-ing, correctness), Stimulus (congruous, ambiguous, incongruous) when appropriate,and Answer (positive, negative) as within-subject factors.

To assess the independence between liking and correctness ratings of stimuliacross all 12 subjects, we computed inter-task answer concordance scores per eachstimulus item and per each stimulus category. These computations were possiblesince the 180 chord cadences used in the study were presented twice, once forthe correctness and once for the liking judgment task. The inter-task answer con-
cordance score corresponded to 1 when the same stimulus was judged as ‘like’ and‘correct’ or as ‘dislike’ and ‘incorrect’ (concordant answers) and to −1 if it was judgedas ‘like’ and ‘incorrect’ or as ‘dislike’ and ‘correct’ (discordant answers). An averagevalue of 0 for an item would then show that the judgments were independent. Inaddition, we determined the within-task stability of the subjects’ answers by assign-ing a score of 1 if a participant rated an item positively (‘like’ for the liking judgment

396 E. Brattico et al. / Biological Psychology 85 (2010) 393–409

F ment,i f the e

tpfijcifiatw

2

fitFOtoas

ig. 1. (a) Examples in musical notation of the chord cadences used in the experincongruous according to Western tonal music theory). (b) Schematic illustration o

ask and ‘correct’ for the correctness judgment task) and a score of −1 if a partici-ant rated an item negatively (‘dislike’ for the liking judgment task and ‘incorrect’or the correctness judgment task). Mean values close to 0 for an item would hencendicate that there was low stability or high variability in the answers across sub-ects, whereas average values close to 1 or −1 would show that the item was ratedonsistently across subjects in the negative or positive direction. Mean values of thenter-task concordance scores and the within-task stability scores, Kendall’s W coef-ciency score measuring agreement among raters (0 corresponding to no agreementnd 1 to complete agreement), and similarity matrices showing Pearson’s correla-ions between answers of pairs of subjects for each stimulus category and each taskere computed.

.4.2. Electrophysiological dataThe EEG (500 Hz sampling rate; online low-pass filter 100 Hz; Synamp ampli-

er, Neuroscan Ltd., El Paso, Texas) was recorded with a 25-electrode cap havinghe following electrodes according to the extended 10–20 international system:
P1, FPz, FP2, F7, F3, Fz, F4, F8, FC5, FC6, T7, C3, Cz, C4, T8, CP5, CP6, P7, P3, Pz, P4, P8,1, Oz, O2. In addition, the following non-scalp electrodes were applied: left mas-
oid, right mastoid, VEOG at the supra and infraorbital right eye and HEOG at theuter canthi of both eyes. The reference electrode was placed at the tip of the nosend FC2 worked as ground electrode. The preprocessing of the EEG signal and theubsequent ERP extraction were performed with our custom Matlab functions inte-

with the three different categories of ending chords (congruous, ambiguous, andxperimental procedure.

grated with functions from EEGLab (Delorme and Makeig, 2004) and Fieldtrip (bythe Centre for Cognitive Neuroimaging, Donders Institute, Nijmegen, Netherlands)toolboxes.

Continuous EEG signals were divided into two different epochs: the first epochincluding the ERP responses to the first four chords of the cadence, hence starting at100 ms before and ending 2000 ms after the onset of the chord cadence (baseline setat the pre-stimulus interval) and the second epoch including the ERP responses tothe last manipulated chord, hence starting at 1900 ms and ending at 3800 ms (base-line set at the interval from 1900 ms to 2000 ms, i.e., 100 ms before the onset of thelast chord). The first epoch was aimed at studying the effects of the listening modeand preparation for the subsequent judgment whereas the second epoch was usedto investigate in detail the effects of the different answers and judgment processesassociated with them. The duration of the second epoch was chosen because mostof the subjects had behaviorally responded within 4000 ms and then they startedmaking eye movements. Trials exceeding a standard deviation of 30 �V at the elec-trodes HEOG, FP1, FPz, FP2, Cz, and left and right mastoid and of 40 �V at VEOG
within a sliding window of 200 ms were rejected automatically. ERPs were offlinefiltered digitally (low pass 20 Hz; transition band of 10 Hz).
We hence conducted four types of ERP averaging. The first one (epoch:−100–2000 ms) was performed according to the experimental task for each subjectand each electrode, and included only the brain responses to the first four chords ofthe cadence. The second ERP extraction (epoch: 1900–3800 ms) was performed by

E. Brattico et al. / Biological Psychology 85 (2010) 393–409 397

RTs for liking and correctness judgment tasks

0

100

200

300

400

500

600

liking correctness

a)

0

10

20

30

40

50

60

liking correctness

Frequencies for liking and correctness judgment tasks

b)

0

20

40

60

80

100

congruous ambiguous incongruous

Frequencies for the liking judgment task

0

20

40

60

80

100


Frequencies for the correctness judgment task

frequ

enci

es o

f ans

wer

s (%

)

frequ

enci

es o

f ans

wer

s (%

)

frequ

enci

es o

f ans

wer

s (%

)

‘dislike ’ ‘incorrect ’

negative answerpositive answer

reac

tio

n t

imes

(ms)

F action( rs rep

ateittccwttEeit

67(ws12a‘sfc(

‘like’

ig. 2. Histogram plots representing the mean frequencies of answers and mean rea) and mean frequencies of answers according to the stimulus category (b). The ba

veraging EEG epochs for each behavioral answer, subject and electrode. The thirdype of ERP averaging (epoch: 1900–3800 ms) was conducted in order to excludeffects of sensory processing on the judgment processes associated with the behav-oral answers: this averaging included only the epochs locked to the stimulus itemshat each subject had rated in a discordant way, i.e., negatively and positively inhe two experimental tasks. Here we took advantage of the fact that each stimulusadence was presented twice, once for the liking judgment task and once for theorrectness judgment task. For example, the EEG trial locked to a stimulus item thatas liked in the liking judgment task and rated ‘incorrect’ in the correct judgment

ask was included in the ERP averaging, whereas a trial locked to a stimulus itemhat was liked and also judged as ‘correct’ was instead rejected. The fourth type ofRP averaging was conducted disregarding the subjects’ behavioral responses, i.e.,poching the EEG according to the stimulus categories of congruous, ambiguous andncongruous chord cadences. This analysis aimed to allow a direct comparison withhe previous electrophysiology literature on music perception.

For the first and second ERP analysis, the number of artifact-free trials was7.5 ± 14.2 (SD) for the ‘like’ answers, 77 ± 16.8 (SD) for the ‘dislike’ answers,5.4 ± 18.8 (SD) for the ‘correct’ answers, and 66 ± 9.5 (SD) for the ‘incorrect’ answersfor the first ERP analysis the trials for the two answers of each task were summedith each other). For the third ERP analysis, including only trials associated with

timuli discordantly rated in the two tasks, the number of accepted trials was7.9 ± 12.3 (SD) for the ‘like’ answers, 26.4 ± 11.9 (SD) for the ‘dislike’ answers,7.9 ± 12.6 (SD) for the ‘correct’ answers, and 15.7 ± 10.1 (SD) for the ‘incorrect’
nswers. As one would expect, the slightly higher number of accepted trials for the
dislike’ and ‘correct’ answers stems from the fact that subjects dislikes the correctounding cadences more often than they liked the incorrect sounding ones. For theourth ERP analysis the number of artifact-free trials was 103.9 ± 20.3 (SD) for theongruous stimuli, 106.4 ± 22.9 (SD) for the ambiguous stimuli, and 107.5 ± 24.3SD) for the incongruous stimuli.

‘correct’

times (RTs) of the behavioral ratings for the liking and correctness judgment tasksresent the standard error of the mean (SE).

In order to test any differences evidenced by visual inspection in the neuralresponses between the different tasks and answers and between the different stim-ulus categories, we first extracted the individual peak latencies of the ERP responsesto the different stimuli, tasks and answers from the electrodes showing the largestERP effect from wide time intervals where ERP effects were likely to occur (alltime intervals are listed in Table 2). Second, we quantified the mean amplitudesof the ERP responses to the different stimuli, tasks and answers from the elec-trodes F3, Fz, F4, C3, Cz, C4, P3, Pz, P4, O1, Oz, O2 as the average values withinthe ±30 ms window around the peak latency of the individual difference wave-forms measured from the electrode with the largest effect. The mean amplitudes ofthe ERP responses were then compared with a general linear model with repeated-measures (analysis of variance or ANOVA) including, when appropriate, Task (liking,correctness), Answer (positive, negative), Stimulus (congruous, ambiguous, incon-gruous), Anterior–posterior (frontal, central, parietal, occipital) and Laterality (left,middle, right) as factors.

In all statistical analyses and when Mauchly’s sphericity tests gave significantresults, type I errors were controlled for by decreasing the degrees of freedom withthe Greenhouse–Geisser epsilon (the original degrees of freedom for all analyses arereported throughout the paper and the ε values are reported only when appropriate).

3. Results

3.1. Behavioral data

As illustrated in Fig. 2a (left panel), the task significantly affectedthe speed of the behavioral response, with slower RTs to the likingjudgment task as compared with the correctness judgment one, F(1,

398 E. Brattico et al. / Biological Psych

Table 1Descriptive statistics for reaction times and frequencies of behavioral responses(standard error of the mean, SE, in parenthesis).

Stimulus RTs in ms Frequencies in %

‘Like’Congruous 72 (5)Ambiguous 485 (64) 40 (4)Incongruous 24 (5)

‘Dislike’Congruous 25 (4)Ambiguous 421 (80) 58 (4)Incongruous 78 (4)

Liking judgment 453 (70) 49 (1)‘Correct’

Congruous 82 (4)Ambiguous 432 (68) 47 (3)Incongruous 23 (4)

‘Incorrect’Congruous 18 (4)Ambiguous 383 (73) 50 (3)

1fMao1

trlfs‘F

svOtwbwscacMptiSa

sFvtm

ada

Incongruous 73 (5)

Correctness judgment 407 (69) 49 (1)

1) = 6.9, p = .02, M = 453 ms (from the end of cadence, i.e., 1453 msrom the onset of the last chord) for the liking judgment task, and

= 407 ms for the correctness judgment task. Furthermore, neg-tive answers (dislike, incorrect) were given earlier than positivenes, as demonstrated by the significant main effect of Answer, F(1,1) = 5.5, p = .04.

A significant interaction Task × Answer was obtained in thewo-way repeated-measures ANOVA for the frequencies of theesponses, F(1, 11) = 5, p = .045, with the highest occurrence of ‘dis-ike’ answers in the liking judgment task (M = 54%) and the lowestor the ‘like’ answers (M = 45%, LSD post hoc test: p < .05) and noignificant difference between the frequencies for the ‘correct’ andincorrect’ ratings (for further details on behavioral results, seeig. 2a right and Table 2).

To further study the effects of the different types of musicaltructures on each behavioral parameter, we replicated the pre-ious ANOVA for answer frequencies1 adding Stimulus as a factor.f particular interest was to test whether the link between music-

heoretical congruity and liking was successfully counterbalancedith the introduction of the ambiguous cadences. The interaction

etween Stimulus and Answer for the frequency of the responsesas significant, F(2, 22) = 61.9, p < .0001, ε = 0.6 (see Fig. 2b). The

timulus cadences that followed the rules of harmony (termedongruous) resulted in predominantly positive answers (‘like’nd ‘correct’), with M = 76.7%, SD = 15.5, whereas the ambiguousadences were similarly rated either positively or negatively (with

= 54%, SD = 12.4 for negative answers and M = 43.4%, SD = 11.4 forositive answers, with a non-significant difference in LSD post hocest, p = .1), and the cadences that violated the harmony rules (thencongruous ones) were mainly judged negatively with M = 75.4%,D = 15.1 (for further details on the behavioral results, see Fig. 2bnd Table 1).

The means of inter-task answer concordance scores across allubjects per each stimulus and stimulus category are shown in
ig. 3. The stimuli belonging to the ambiguous category obtainedalues most closely approaching 0 (M = 0.3), signifying low inter-ask concordance. This behavioral result justifies the ambiguous
usic-theory-based classification. However, also some of the stim-

1 We did not perform the same repeated-measures ANOVA including Stimuluss factor for the RTs because their means were distorted as a consequence of theiscrepancies between answer frequencies according to stimulus categories. Suchnalysis would have resulted in the Simpson’s paradox (Simpson, 1951).

ology 85 (2010) 393–409

uli from the congruous and incongruous categories were ratedinconsistently resulting in mean concordance scores of 0.4 and 0.5,respectively. Furthermore, as shown in Fig. 3, ambiguous stimuliobtained a mean within-task stability score in both tasks which wasvery close to 0 (M = −0.18 for the liking judgment task and M = −0.02for the correctness judgment task), contrasted with the more stablepositive ratings given in both tasks to congruous stimuli (M = 0.49for the liking judgment task and M = 0.65 for the correctness judg-ment task) and negative ratings given in both tasks to incongruousstimuli (M = −0.57 for the liking judgment task and M = −0.52 for thecorrectness judgment task). As visible in Fig. 4, similarity matrices,comparing answers by pairs of subjects, converged in showing thatthe subjects’ answers were the least correlated for the liking judg-ments of the ambiguous stimuli and the most correlated for thecorrectness judgments of incongruous stimuli. Finally, the overallKendall’s W obtained with the concordance scores across stimu-lus categories was equal to .053, indicating low agreement in theratings between subjects.

3.2. Electrophysiological data

3.2.1. Listening mode effects on ERPs to the first four chordsThe first chord of the cadences used as a stimulus for both

the liking and correctness judgment tasks elicited a P1, N1 andP2 complex, followed by a sustained negativity (see Fig. 5). Thenext chords elicited also fronto-central N1–P2 complexes althoughpartially overlapped with the sustained negativities to the pre-vious chords. The ERP waveforms to the correctness and likingjudgment tasks coincided almost completely. Some differencesbetween the waveforms were visible on average at 305 ms (i.e.,already in response to the first chord), particularly when the wave-forms were re-referenced to the mastoids (see Fig. 5), with largernegativities for the correctness judgment task (M = −3.6 �V) com-pared to the liking one (M = −2.6 �V), their difference amounting to−1 �V with a significant main effect of Task, F(1, 11) = 11, p = .007for mastoid-referenced waveforms (and F(1, 11) = 3.8, p = .08 fornose-referenced waveforms with peak latency at 323 ms). For moredetails on the mean latency and amplitude of this ERP effect seeTable 2. As illustrated by the voltage map in Fig. 5, the early nega-tive wave to the correctness task was found to be fronto-centrallydistributed with a significant interaction Task × Anterior–posterior,F(3, 33) = 4, p = .04, ε = 0.6.

3.2.2. Answer effects on ERPs to the ending chordEarly negativity: As illustrated by Fig. 6 and Table 2, the ERP

waveforms averaged according to subjects’ behavioral answersto the liking and correctness judgment tasks differed right afterthe onset of the manipulated ending chord (after 2000 ms). TheERPs to the liking judgment task showed a small peak, on aver-age, at 246 ms after the onset of the ending chord, whereas theERPs to the correctness task peaked, on average, at 307 ms. Atthis early latency, the ERP deflections were modulated by the dif-ferent tasks and answers given by the subjects, as indicated bythe significant main effect of Answer, F(1, 11) = 24.1, p = .0001,and the significant interactions Answer × Anterior–posterior,F(3, 33) = 8.5, p = .004, ε = 0.5, Answer × Laterality, F(2, 22) = 9.4,p = .001, Task × Answer × Anterior–posterior, F(3, 33) = 4.8, p = .04,and Task × Answer × Anterior–posterior × Laterality, F(6, 66) = 2.7,p = .02. Overall, the ERPs were maximal over the fronto-centralscalp area, with a significant main effect of Anterior–posterior, F(3,33) = 4.9, p = .006, ε = 0.5.

We analyzed the 3- and 4-way interactions in separaterepeated-measures ANOVAs for the factor Task. For the lik-ing judgment task we obtained only a significant interactionAnswer × Laterality, F(2, 22) = 3.6, p = .04, due to the negative dif-ference between ERP responses to the disliked vs. liked stimuli


Concordance scores across tasks

Stability scores for liking judgment task

-1.0

-0.5

0.0

0.5

1.0


-1.0

-0.5

0.0

0.5

1.0


Stability scores for correctness judgment task

0.0

0.2

0.4

0.6

0.8

1.0

congruous ambiguous incongruous-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

20 40 60 80 100 120 140 160 180

for each chord means across chords

means across chords

for each chord means across chords

for each chord

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

20 40 60 80 100 120 140 160 180

20 40 60 80 100 120 140 160 180-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Fig. 3. Histogram plots representing the mean concordance score and mean stability scores for the ratings of each of the 180 chord cadences repeated twice across the twoexperimental tasks (right) and averaged across chord cadences for each of the 3 stimulus categories (left). Congruous chord cadences are depicted in blue, ambiguous chordcadences in red and incongruous chord cadences in black. The bars in the plots at the left represent the standard error of the mean (SE).


twee

oaA1t

TD

Fig. 4. Similarity matrices showing Pearson’s correlations be

nly over the right hemisphere, amounting, on average, to −1.2 �V,
s reflected by the significant main effect of Answer in a separateNOVA for the amplitudes measured at the right electrodes, F(1,1) = 5.9, p = .03. For the correctness judgment task, the negativity tohe ‘incorrect’ answers was more pronounced, as showed by the sig-
able 2escriptive statistics for mean latencies and amplitudes of ERP responses (standard error

ERP responses Mean late

Task-based averaging100–400 ms at Cz

Correctness judgment minus liking judgment 305 (18)Answer-based averaging

140–400 ms at P4‘Dislike’ minus ‘like’ 246 (26)‘Incorrect’ minus ‘correct’ 307 (16)


1000–1400 ms at Oz‘Dislike’ minus ‘like’ 1156 (33)‘Incorrect’ minus ‘correct’ 1247 (33)

Discordant-answer averaging140–400 ms at Cz

‘Dislike’ minus ‘like’ 274 (29)‘Incorrect’ minus ‘correct’ 234 (26)


1100–1400 ms at Oz‘Dislike’ minus ‘like’ 1246 (24)‘Incorrect’ minus ‘correct’ 1241 (25)

Stimulus-locked averaging140–400 ms at P4

Ambiguous minus congruous 301 (22)Incongruous minus congruous 284 (18)

n pairs of subjects for each stimulus category and each task.

nificant main effect of Answer in a separate ANOVA, F(1, 11) = 30.6,
p < .0001, with ERP mean amplitude to the ‘incorrect’ judgmentsamounting to 0.4 �V and that to the ‘correct’ judgments equalto 2.7 �V (difference equal to −2.35 �V). Moreover, the ANOVAfor the correctness judgment task resulted in the following sig-
of the mean, SE, in parenthesis).

ncy in ms Mean amplitude in �V

−1.3 (0.4)

−1.4 (0.4)−3.4 (0.4)

2 (0.7)1.8 (0.8)

2.2 (1.1)−3.2 (1.2)

−1.8 (1)−3.6 (1.5)

4.2 (1.7)3.4 (1.9)

4.8 (1.3)−4.6 (1.9)

−1.6 (0.6)−2.9 (0.6)


Fig. 5. (a) Grand-average ERP waveforms, referenced to the nose electrode, elicited at 12 electrode sites in response to the chord cadences during the liking and correctnessjudgment tasks. (b) Grand-average ERP waveforms, referenced to the algebraic mean of the mastoid electrodes, elicited at 12 electrode sites in response to the chord cadencesd e ERPv ncemg ERPs

npAesfAaprrsawpweEcpro2t

Tr

uring the liking and correctness judgment tasks. (c) Enlarged is the grand-averagoltage map is indicated by the gray shaded area. (d) Voltage map of the N1 enharand-average difference waveforms (ERPs to the correctness judgment task minus

ificant interactions: Answer × Anterior–posterior, F(3, 33) = 9.4,= .005, ε = 0.5, Answer × Laterality, F(2, 22) = 6.5, p = .006, andnswer × Anterior–posterior × Laterality, F(6, 66) = 4.3, p = .001. Asvidenced by the scalp maps in Fig. 6c, these interactions revealedcalp topographic differences in the distribution of the effector the ‘incorrect’ answers. Separate ANOVAs for the factornterior–posterior resulted in a significant main effect of Answert the central, F(1, 11) = 20, p < .0001, parietal, F(1, 11) = 45.7,< .0001, and occipital regions, F(1, 11) = 40.1, p < .0001, with a

ight-hemispheric lateralization for the negative ERPs to the ‘incor-ect’ answers only at the central electrodes, as evidence by aignificant interaction Answer × Laterality, F(2, 22) = 11, p < .0001,nd p < .0001 in LSD post hoc tests, and at the parietal electrodes,ith the significant interaction Answer × Laterality F(2, 22) = 5.8,= .01, and p = .03 in LSD post hoc tests. Consequently, at the latencyindow in which an ERAN to chord harmony violations would be

xpected, a significant difference in the negative direction betweenRPs to chords judged as ‘incorrect’ and those judged ‘correct’ in theorrectness judgment task only was elicited, with a right central-osterior distribution over the scalp (see Fig. 6c). A less pronouncedight negative difference between the ‘dislike’ and ‘like’ answersf the liking judgment task was also found at latencies around
50–300 ms but it was smaller and more distributed over the scalphan the negativity to the correctness judgment task (see Fig. 6c).
Early negativity to discordant answers: As shown by Fig. 7 andable 2, the results of the traditional ERP analysis were partiallyeplicated in the repeated-measures ANOVA for the discordant-

waveform elicited at the frontal (Fz) electrode. Time interval used for plotting theent to the correctness task plotted at the amplitude peaks of the nose-referencedto the liking one).

answer ERP averaging, from which the trials locked to stimuliobtaining either only negative or only positive answers in the twoexperimental tasks were excluded. The mean amplitudes measuredon average at 275 ms for the liking judgment task and on averageat 234 ms for the correctness judgment task (no significant differ-ence between the latencies according to a paired samples t-test)were modulated by the positive or negative answers given by sub-jects in the two tasks, as revealed by the significant interactionAnswer × Anterior–posterior, F(3, 33) = 4.8, p = .045, ε = 0.4. Sepa-rate ANOVAs for the factor Anterior–posterior showed a significantmain effect of Answer at the frontal region, F(1, 11) = 5.5, p = .04,with a negative difference between negative and positive judgmentequaling to −2 �V, and at the central region, F(1, 11) = 8.3, p = .02,with an ERP negativity amounting to −2.1 �V.

Late positivity: Large posterior positive brain deflections wereelicited on average at 469 ms after the onset of the last chord forthe liking judgment task and at 516 ms for the correctness judg-ment task with a significant difference between the two latencies,t(11) = −2.3, p = .04. These positive brain deflections were modu-lated by the behavioral answers, as illustrated by the significantmain effect of Answer, F(1, 11) = 8.5, p = .01: for both the liking andthe correctness judgment tasks the negative answers elicited larger
ERP positivities (M = 6.9 �V) than the positive answers (M = 5.3 �V;see Fig. 6b and Table 2). Furthermore, the behavioral tasks mod-ulated the scalp topographies of the ERP responses, as shown bythe interactions Task × Anterior–posterior, F(3, 33) = 5.3, p = .004,ε = 0.4, resulting from slightly larger positivity at occipital regions


Fig. 6. (a) Grand-average ERP waveforms, referenced to the nose electrode, elicited at 12 electrode sites in response to the chord cadences during the ‘like’/’dislike’ answersof the liking judgment task and the ‘correct’/’incorrect’ answers of the correctness judgment task. (b) Enlarged grand-average ERP waveform elicited at the right parietal (P4)electrode. Time intervals used for plotting the voltage maps are indicated by gray shaded areas with darker gray for the liking correctness task and with lighter gray for thecorrectness judgment task. (c) Voltage map representing the posterior early negativity elicited by the incorrect ratings. (d) Voltage maps of the late positivities to negativea RP resm nd-av‘ e int

fie

to(hct(mAh

amtiE((Flmtanp

nswers of the correctness and liking judgment tasks. (e) Voltage maps of the later Eaps were calculated by integrating ±30 ms around the amplitude peaks of the gra

like’ or ‘correct’ answers, respectively). The numbers in parentheses refer to the tim

or the correctness judgment task compared to the liking one, asndicated by a marginally significant main effect of Task at occipitallectrodes, F(1, 11) = 3.7, p = .08.

At this latency the ERP responses were overall lateralized tohe right hemisphere, as indicated by the significant main effectf Laterality in the 4-way ANOVA, F(2, 22) = 13.4, p = .004, ε = 0.7M = 6.2 �V over the right hemisphere and 5.6 �V over the leftemisphere), and this lateralization was most evident at frontal,entral and parietal regions, as shown by the significant interac-ion Laterality × Anterior–posterior, F(6, 66) = 5.5, p = .002, ε = 0.6p < .02–.001 in LSD post hoc tests). The ERPs were overall maxi-

al at parietal regions, as revealed by the significant main effect ofnterior–posterior, F(3, 33) = 18.1, p = .001, ε = 0.5 (p < .002 in postoc LSD tests).

Late positivity to discordant answers: As illustrated by Fig. 7nd Table 2, these results were partly confirmed by the repeated-easures ANOVA for the ERP amplitudes obtained when only

rials associated with stimuli rated discordantly by subjects werencluded in the ERP averaging. At around 480 ms, larger positiveRPs were evoked by negative ‘dislike’ and ‘incorrect’ answersM = 5.8 �V) compared with positive ‘like’ and ‘correct’ answersM = 2.5 �V), as confirmed by the significant main effect of Answer,(1, 11) = 7.4, p = .02. Also in this ANOVA, all ERP responses wereateralized to the right hemisphere, as evident from the significant

ain effect of Laterality, F(2, 22) = 9.2, p = .006, ε = 0.6 (M = 3.7 �V athe right hemisphere and 4.2 �V at the left hemisphere), especiallyt central, parietal, and occipital regions, as shown by the sig-ificant interaction Laterality × Anterior–posterior, F(6, 66) = 4.3,= .001 (p < .01–.001 in LSD post hoc tests), and were maximal

ponses to negative answers of the correctness and liking judgment tasks. All voltageerage difference waveforms (ERPs to ‘dislike’ or ‘incorrect’ answers minus ERPs to

ervals from which the voltage maps were plotted, as shown in panel b.

at parietal regions, as revealed by the significant main effect ofAnterior–posterior, F(3, 33) = 16.8, p < .001, ε = 0.6 (p < .003 in posthoc LSD tests).

Later ERP response: At the very late latencies of 1156 ms, onaverage, for the liking judgment task, and 1247 ms, on average, forthe correctness judgment task (no significant difference in paired-samples t tests), i.e., at latencies shortly preceding the buttonpress, the ERP responses during the two judgment tasks diverged(see Fig. 6 and Table 2), as indicated by the significant interac-tions Task × Answer × Anterior–posterior, F(3, 33) = 12.3, p = .003,ε = 0.4 and Task × Answer × Laterality, F(2, 22) = 3.5, p < .05. Thosebrain responses lasted for several hundreds of ms, i.e., from about1000–1800 ms after the ending chord (see Fig. 6). As illustrated bythe voltage scalp maps of Fig. 6e, the brain deflections to the ‘incor-rect’ answers of the correctness judgment task (M = 4.7 �V) wereless positive than those to the ‘correct’ answers (M = 6.8 �V) withthe resulting negative difference mean value amounting to−2.1 �V.This negative difference wave to ‘incorrect’ answers was predom-inant over the right hemisphere, as evidenced by the significantinteraction Answer × Laterality in a separate ANOVA for the factorTask, F(2, 22) = 5.4, p = .03, ε = 0.7. Moreover, the negative differencebetween the ‘incorrect’ and ‘correct’ judgments was particularlyevident at central, parietal and occipital regions, as indicated by thesignificant interaction Answer × Anterior–posterior, F(3, 33) = 4.9,
p = .04, ε = 0.4 and LSD post hoc tests, p < .0001. In contrast, as illus-trated by Fig. 6, the brain deflections for the ‘dislike’ answers ofthe liking judgment task were more positive than those for the‘like’ answers at parietal and occipital regions and vice versa morenegative at the frontal regions, as indicated by the significant inter-


Fig. 7. (a) Grand-average ERP waveforms, referenced to the nose electrode, elicited at 12 electrode sites in response to the chord cadences during the ‘like’/’dislike’ answersof the liking judgment task and the ‘correct’/’incorrect’ answers of the correctness judgment task. In these ERP waveforms the trials locked to stimuli obtaining only positiveor only negative answers in the two experimental tasks were excluded. (b) Enlarged grand-average ERP waveform elicited at the right parietal (P4) electrode. Time intervalsused for plotting the voltage maps are indicated by gray shaded areas with darker gray for the liking correctness task and with lighter gray for the correctness judgment task).(c) Voltage map representing the posterior early negativity elicited by the incorrect ratings. (d) Voltage maps of the late positivities to negative answers of the correctnessa answb islike’n ere plo

afhrrti

rwcEiacttwwLabTSEpet

nd liking judgment tasks. (e) Voltage maps of the later ERP responses to negativey integrating ±30 ms around the grand-average difference waveforms (ERPs to ‘dumbers in parentheses refer to the time intervals from which the voltage maps w

ction Answer × Anterior–posterior in a separate ANOVA for theactor Task, F(3, 33) = 6.3, p = .02, ε = 0.4 and by subsequent LSD postoc tests, p < .05–.01 (see Fig. 5). Overall, the very late positive ERPesponse to ‘dislike’ ratings vs. ‘like’ ratings were lateralized to theight frontal scalp region, as evidenced by the significant interac-ion Anterior–posterior × Laterality, F(3, 33) = 6, p = .002. No othernteractions for the factor Laterality were obtained (F ≤ 1).

Later ERP response to discordant answers: These findings wereeplicated and even strengthened with the ERPs computed onlyith discordant answer, i.e., excluding the stimuli associated with

onsistent answers across tasks (see Fig. 7 and Table 2). The meanRP amplitudes measured at 1246 ms, on average, for the lik-ng judgment task differed from those measured at 1241 ms, onverage, for the correctness judgment task, as shown by the signifi-ant interaction Task × Answer, F(1, 11) = 11.5, p = .006. Specifically,he brain deflections to the ‘dislike’ answers were more positivehan those to the ‘like’ answers (p = .045 in LSD post hoc test),hereas, oppositely, the ERP responses to the ‘incorrect’ answersere more negative than those to the ‘correct’ answers (p = .02 in

SD post hoc tests). The mean ERP amplitudes differed in polaritynd scalp distribution according to the tasks and answers giveny the subjects, as demonstrated by the significant interactionask × Answer × Anterior–posterior, F(3, 33) = 7.3, p = .01, ε = 0.5.
eparate ANOVAs for the factor Task further evidenced that theRP responses to the ‘dislike’ answers were more positive as com-ared to those to the ‘like’ answers of the liking judgment taskspecially at parieto-occipital regions, with the significant interac-ion Answer × Anterior–posterior, F(3, 33) = 4.3, p = .04, ε = 0.5 and
ers of the correctness and liking judgment tasks. All voltage maps were calculatedor ‘incorrect’ answers minus ERPs to ‘like’ or ‘correct’ answers, respectively). Thetted, as shown in panel b.

p < .0001 in post hoc LSD tests. In contrast, as visible from thevoltage scalp maps of Fig. 7e, the negative difference between‘incorrect’ and ‘correct’ answers of the correctness judgment taskwas broadly distributed over the scalp (no significant interactionAnswer × Anterior–posterior). The Laterality factor did not yield asignificant effect or interactions in the 4-way ANOVA.

3.2.3. Stimulus effects on ERPs to the ending chordEarly negativity: When the ERPs were averaged according to the

congruous, ambiguous, and incongruous stimulus categories, anearly negativity resembling the ERAN wave was elicited at 284 ms,on average, after the onset of the ending chord for the differ-ence wave between ERPs to incongruous and congruous chordsand at 301 ms, on average, for the difference wave between ERPsto ambiguous and congruous chords (no significant differencebetween these latencies in a paired samples t test). As Fig. 8and Table 2 evidence, a large early negativity was elicited bythe incongruous chord when compared to the congruous chord(M = −2.2 �V) and a small early negativity was elicited by theambiguous chord compared with the congruous one (M = −1.3 �V),confirmed by a significant main effect of Stimulus, F(2, 22) = 7.3,p = .004. The ERP negative deflection in response to the incon-gruous chord was significantly larger than that to the congruous
chord (p = .001 in post hoc LSD tests), and to a smaller extent, thenegativity to the ambiguous chord also differed from that to thecongruous chord (p < .05 in post hoc LSD tests; no significant dif-ference between ERP amplitudes to incongruous and ambiguouschords). The ERPs to the three chord conditions further varied in


Fig. 8. (a) Grand-average ERP waveforms, referenced to the nose electrode, elicited at 12 electrode sites in response to the three stimulus categories of congruous, ambiguous,incongruous chords. (b) Enlarged grand-average ERP waveform elicited at the right parietal (P4) electrode. Time intervals used for plotting the voltage maps are indicatedby gray shaded areas with darker gray for the ambiguous stimulus category and with lighter gray for the incongruous stimulus category. (c) Voltage maps representing theearly negativity elicited by the incongruous and ambiguous chords. The maps were calculated by integrating ±30 ms from the grand-average difference waveforms (ERPsto ambiguous or incongruous chords minus ERPs to congruous chords). The numbers in parentheses refer to the time intervals from which the voltage maps were plotted,as shown in panel b. (d) Enlarged grand-average ERP waveform to the incongruous and congruous stimulus categories and to the ‘incorrect’ and correct’ answers by thes

sleotA2fwpiafsf

ats

vp6so

ubjects, elicited at the right parietal (P4) electrode.

calp distribution, according to the significant interaction Stimu-us × Anterior–posterior, F(6, 66) = 5.4, p < .0001, with a significantarly negativity to incongruous stimuli (compared to congruousnes) at central, parietal, and occipital regions, as testified byhe main effects of Stimulus in separate ANOVAs for the factornterior–posterior, F(2, 22) = 5.8, p = .01, F(2, 22) = 9.3, p = .001, F(2,2) = 122.3, p < .0001, respectively (p < .0001 in LSD post hoc testsor the comparison between incongruous and congruous ERPs) andith a small significant negativity to ambiguous stimuli (com-ared to congruous ones) at parietal and occipital regions (p < .05

n LSD post hoc tests for the comparison between ambiguousnd congruous ERPs). No differences in the ERP amplitudes wereound between the two cerebral hemispheres, as indicated by non-ignificant main effects and interactions containing Laterality asactor.

We further analyzed the ERP deflections to stimulus categories
t subsequent time windows, corresponding to those used for theask-related analysis (see Table 2). The results did not show anyignificant ERP effect.2
2 Repeated measures ANOVA on the brain responses obtained from the time inter-al at 400–600 ms did not yield a significant main effect of Stimulus, F(2, 22) = 2,= .1; although they evidenced a significant interaction Stimulus × Frontality, F(6,6) = 4.4, p = .008, ε = 0.5. This interaction, however, derived only from a non-ignificant main effect of Stimulus in a separate ANOVA for the parietal electrodesnly, F(2, 22) = 3.2, p = .06.

3.2.4. Comparison between early negativities to ‘incorrect’,ambiguous and incongruous chords

The apparent similarity between the early negativities to stimu-lus categories and the ones to the correctness task was investigatedby two repeated-measures ANOVAs with Condition (2 levels:stimulus, task), and Ending type (2 levels: ambiguous/incorrect,congruous/correct for the first ANOVA and incongruous/incorrect,congruous/correct for the second ANOVA). As illustrated byFig. 8d, no significant differences between the early negativi-ties obtained with the two different averaging methods wereobserved, as reflected by the non-significant interactions Condi-tion × Ending type in both ANOVAs (F < 1 for both). In contrast,as expected, the main effects of Ending type was significant bothin the ANOVA comparing negative ERPs to ambiguous stimu-lus category and ‘incorrect’ answers, F(1, 11) = 5.2, p < .05, andin the ANOVA comparing incongruous stimulus category and‘incorrect’ answers, F(1, 11) = 8.7, p = .01 (here the interaction End-ing type × Anterior–posterior was also significant, F(3, 33) = 4.6,p = .04, ε = 0.5). Additionally, we found a significant interactionCondition × Ending type × Anterior–posterior, F(3, 33) = 6.5, p = .02,ε = 0.4, when comparing ERPs to incongruous and congruousstimuli vs. ERPs to ‘incorrect’ and ‘correct’ judgments; however,separate ANOVAs did not evidence any significant difference in the
scalp distribution of the two ERP responses. We may hence con-clude that the two early negativities were not clearly discerned inthe ANOVAs (see also Figs. 7 and 8). Finally, no significant latencydifferences were observed between early negativities elicited bystimulus categories and by behavioral answers.

l Psych

4

wjwcacotaWemrwoesosmlwdTawcmEtwtbcoscleiijaaoTaitm

4

otedoumai


. Discussion

In this electrophysiological and behavioral study, participantsith minimal musical education performed correctness and liking

udgments of the same musical material. The musical sequencesere of different types, varying in the structural music-theoretical

ontent, and were transposed over several keys, thus introducinglso a variation on the acoustic pitch dimension. Further, the lasthord was manipulated with the intent to induce different nuancesf liking and correctness judgments by subjects. We expected holis-ic effects of the listening mode during cadence presentation as wells specific effects of the cognitive or affective judgments of music.ith regard to the electrophysiological results, first, we obtained

nhanced negative waves in preparation for the correctness judg-ents (i.e. before the ending manipulated chord). These effects

esulted from the performance of two tasks on the same stimuli,hich were presented twice (once for the liking judgment task and

nce for the correctness judgment task). Hence, the holistic ERPffects of the listening mode cannot be explained by differentialensory processing of the music stimuli. Second, after the onsetf the last decisive chord, the early ERP responses were mostlyensitive to the subjects’ judgments of tonal incorrectness in theusical passage, and restricted to the right hemisphere for the dis-

iking judgments. At later latencies, larger positive brain responsesere elicited by the negative answers of both tasks, although with aistinct scalp topography for the correctness and liking judgments.hird, at around 1200 ms from the onset of the last chord, the likingnd correctness judgments clearly differed in the brain responses,ith a posterior slow positivity to the disliked vs. the liked musi-

al passages and a posterior negativity to the incorrectly soundingusical passages vs. the rightly sounding ones. Importantly, all the

RP effects were reproduced and even strengthened in an addi-ional analysis where only discordant answers to the two tasksere included, suggesting that the ERP waves are indeed linked

o the neural processes preceding distinct cognitive and affectiveehavioral judgments. However, note that those two neural pro-esses did not result in being completely independent from eachther but rather partially overlapping, at least during the initialensory processing of the musical features. In turn, the neural disso-iation between the two judgment tasks seem to occur at very lateatencies, just preceding the behavioral outcome of the cognitive orvaluative judgments. Finally, we further tentatively explored thendependence between affective and cognitive judgment processesn music and their individual variation. We found that subjects’udgments of chord correctness deviated from the ‘objective’ evalu-tion method based on music theory in a significant number of casesnd that liking ratings were often given independently from thosef chord correctness, particularly for the ambiguous chord category.he variability in judgments across subjects was also consider-ble. These findings suggest that the current dual task paradigmncluding ambiguous stimuli could be adopted in the future to fur-her investigate the neural correlates of subjective processes during

usic listening.

.1. Liking ratings slower than correctness ones

The present behavioral results showed that liking judgmentsf a music passage were slower than correctness judgments ofhe same musical material. These findings confirm the results onvaluative vs. cognitive judgments already obtained in the visualomain, with paintings (Mandler and Shebo, 1983) or abstract
bjects (Jacobsen and Höfel, 2003) as stimuli. Consequently, eval-ative processes leading to liking judgments of a musical passageay be as well based on representational and perceptual processes
nd follow the latter ones in the information processing dynam-cs. According to the fast vs. slow route theory of emotion, hence,

ology 85 (2010) 393–409 405

liking judgment would require interpretive appraisal of the percep-tual and emotional connotations of music, possibly involving theprefrontal cortex (as the current data would suggest), rather thanbeing an automatic response of the fast subcortical route, directlylinking peripheral auditory processes to the limbic system (Powerand Dalgleish, 1999; Scherer and Zentner, 2001). Our findings arevalid with subjects having little or no music education. A differentrelation between the processing speed of evaluative and cognitiveprocesses might instead be found with subjects, such as profes-sional musicians, possessing a fast and efficient access to formalknowledge of the structures and rules of tonal harmony (Bessonand Faita, 1995; Koelsch et al., 2002). These musically educated sub-jects might also more immediately link their evaluative reactionswith the cognitive computation of musical properties. According tosome theorists, emotion–cognition sequences can become modu-larized during development based on frequent learned behaviors(Power and Dalgleish, 1999). Neuroimaging studies are in progressin our laboratory to further test these study hypotheses.

4.2. ERP effects of the listening mode

We also obtained enhanced negative responses in correspon-dence of the second chord of the cadence (thus before the decisivemanipulated chord) during the correctness judgment task. A well-known phenomenon in the electrophysiological literature consistsin the enhanced negative difference, termed early Nd, occurringin the latency window of the N1 wave of the ERP. The early Nd iselicited by auditory stimuli, like tone pips, syllables and environ-mental sounds, which are presented at the attended ear contrastedwith similar auditory stimuli presented at the unattended oppositeear in a dichotic listening paradigm (Coch et al., 2005; Hillyard et al.,1973; Luo and Wei, 1999). It has been suggested that the early Ndreflects the initial rapid feature processing of auditory stimuli whenthey are relevant for the task. In some studies, the early Nd is fol-lowed by a late frontocentral Nd, occurring at around 300–600 msafter stimulus presentation. Since its first observation, the late Ndhas been associated with higher cognitive processes. A recent studyby Singhal and Fowler (2005) convincingly demonstrated that thelate Nd is related to some aspects (most likely intramodal rehearsal)of the working memory system, since its amplitude is reducedby memory set size in a Sternberg task. The enhanced negativebrain waves to sounds during the correctness judgment task maytherefore indicate working memory operations of the auditory sys-tem while listening to the chords. The association of the late Ndwith modal working memory is coherent with the ERP waveformsbeing re-referenced to the mastoid electrodes, as the contributionfrom the temporo-mastoidal region points at the involvement ofauditory-cortex processes.

Alternatively, the early negative enhancement observed duringthe correctness judgment task may be interpreted as a contingentnegative variation (CNV; Birbaumer et al., 1990). The CNV com-plex, and especially its late centroparietal component, is a slowwaveform preceding task-related stimuli. Typically it is associatedwith forewarned reaction time tasks and is elicited in the anticipa-tory interval between a pre-cue stimulus giving information aboutthe forthcoming stimulus and the imperative stimulus to whichthe subject has to respond. The CNV correlates with motor prepa-ration, increasing in amplitude with the amount of known detailsabout the forthcoming response (Leuthold et al., 2004), with antici-patory sensory and cognitive processes related to an incoming task(van Boxtel and Böcker, 2004), with reproduction of time intervals
between visual stimuli (Elbert et al., 1991), and with the degree ofpreparatory effort subjects put into a task (Falkenstein et al., 2003).More generally, it has been suggested that the CNV reflects corticalarousal related to anticipatory attention, motor activity, motiva-tion and information processing (Nagai et al., 2004; Tecce, 1972).

4 l Psych

Cnertjttstammmatsd

4

fttcp‘owmiaefmrl2tps(batsTlrttpKiihEEosv

oat


onsidering the present results, the negative wave to the correct-ess task judgment may hence index the recruitment of additionalncoding resources devoted to establish the tonal context in prepa-ation of correctness judgments. Our subjects had minimal musicalraining so it is reasonable to suppose that a task asking explicitudgments of musical correctness may have been less natural thanhe liking judgment task (and sometimes appropriately designedasks are needed to tackle implicit musical skills of nonmusicians;ee Smith et al., 1994). Recently, Kreutz et al. (2008) demonstratedhe existence of top-down strategies during listening to music usingquestionnaire survey. In particular, listeners may be classified asusic-empathizers when they focus on the affective aspects of theusic and try to get tuned to the emotions of the composer or asusic-systemizers who are rather interested in finding structures

nd organization behind the music. Future studies might addresshe neural substrates of individual differences in music listeningtyles by adopting the current dual-task mixed paradigm, able toiscern between affective vs. cognitive listening processes.

.3. ERP effects of the judgment processes

The neural responses associated with the different judgmentsor the liking and correctness tasks differed substantially afterhe onset of the last chord of the cadence, i.e., after the chordhat varied in the degree of congruity with the previous musi-al context and in the associated aesthetic value. The largerosterior negative response to the chords which were rated as

incorrect’ as compared with those rated as ‘correct’ was obtainednly over the right hemisphere for the liking judgment task andithout a clear posterior distribution as for the correctness judg-ent task (Fig. 6). These results were partially replicated when

ncluding in the analysis only the brain responses to stimulittributed with discordant judgments in the two tasks (Fig. 7). How-ver, these unconventionally-obtained early negative waves wereronto-centrally distributed for both liking and correctness judg-

ent tasks. The early negative responses found in our study areeminiscent of the fronto-central ERAN response to chord stimu-us categories defined according to music theory (Koelsch et al.,000), having neural generators localized in the prefrontal cor-ex (Maess et al., 2001). For a more direct comparison with therevious literature, we averaged the ERP responses locked to thetimulus categories disregarding the subjects’ behavioral answersFig. 8). The results evidenced significant ERAN-like responses tooth ambiguous and incongruous chords (although smaller for thembiguous chords) that did not substantially differ in mean ampli-ude, latency, and scalp distribution from the ERAN-like responsesorted according to the subjects’ answers in the behavioral tasks.his suggests that the neural processes underlying the stimulus-ocked ERAN brain response to chord incongruity are indeed theeflection of individual processes associated with harmony viola-ion detection, as also the modulation of the ERAN amplitude byhe degree of musical expertise possessed by subjects or by genderreviously testified (Koelsch et al., 2002, 2003; for a review, seeoelsch, 2009). It should be, however, kept in mind that some of the

ncongruous chords utilized here were highly dissonant hence mix-ng chord inharmonicity with context-built unexpectedness, andence contaminating the cognitive ERAN response with sensoryRP processes. Nevertheless, since the direct comparison betweenRAN-like responses derived from different ERP averaging meth-ds was only secondary to our aims, we would encourage to furthertudy the variability of the ERAN by top-down processes and indi-
idual factors.
In addition, at around 600 ms from the onset of the last chordf the cadence, larger late positivities were elicited by the negativenswers of both tasks (‘dislike’, ‘incorrect’), without any differen-iation between the tasks. This was supported by ERPs to stimuli

ology 85 (2010) 393–409

obtaining discordant answers. In the music electrophysiology lit-erature, a late positivity termed P600 or late positive component(LPC) (Besson et al., 1994; Besson and Faita, 1995; Regnault et al.,2001; Schön and Besson, 2005; Steinbeis et al., 2006; Brattico et al.,2006) has been associated with the conscious detection of a musicalincongruity and the related upgrading of the musical context. Inter-estingly, for the correctness judgment task the late positivity wasobtained by averaging the EEG epochs time locked to the subjects’ratings of incorrectness, rather than by selecting the epochs lockedto the sounds that were defined as incongruous a priori according tomusic theory (as indicated by the traditional procedure in the ERPliterature). To our knowledge, a similar procedure has been adoptedpreviously only in a study of music-theory-based vs. individuallyperceived sound unpleasantness (Schön et al., 2005).

The most marked difference in the brain responses between theliking and correctness judgments was obtained at a very late latencyafter the end of the musical cadence. At around 1200 ms (200 msafter the end of the chord stimulus) the chords rated as ‘incorrect’elicited a negative brain deflection whereas the disliked stimulielicited an enhanced positive response compared to the liked stim-uli. These brain processes occurred about 200 ms before the buttonpress and their very late latency indicates underlying controlledprocesses employed during the liking and correctness judgments.We could also speculate that the use of a dual-task mixed paradigmand the processing efforts related to it may have triggered metacog-nitive reflective processes in which the content of perceptionbecomes the object of thought at a higher level (Zelazo, 2004).The content of these controlled processes may, however, differ, assuggested in our study by the different morphology and scalp distri-bution of the ERP deflections. The neural response to the ‘incorrect’stimuli closely resembles the late posterior negative slow wave(LPN), occurring at around 1000 ms after stimulus presentation,which is often encountered in studies involving recognition tasksand episodic memory (Johannson and Mecklinger, 2003). The cur-rent study cannot, however, disentangle whether the largest rolein the elicitation of this response was played by action monitor-ing aimed at solving conflict between alternative button pressesor by memory integration mechanisms combining the previouslyheard sounds with implicit tonal knowledge and with the specificprocessing goals. The scalp distribution of the current LPN, simi-larly to what reported in the literature, hints at neural generatorsin posterior parietal cortex.

In contrast, the liking judgment task elicited a slow late pos-terior positivity occurring between 1000 and 1800 ms after theonset of the ending chord and associated with the ‘dislike’ answersas opposed to the ‘like’ ones. This brain response resembles inlatency and parietal scalp distribution the late positive potential(LPP), reflecting operations of evaluative classification in cogni-tive domains such as visual art or language (e.g., Cacioppo et al.,1994; Crites et al., 1995; Schupp et al., 2000) or generally cor-related with valenced affective judgments (e.g., Cunningham etal., 2005). Particularly, larger LPP has been observed in responseto affectively intense pictures (both in the positive and negativedirection) as compared to neutral pictures (Schupp et al., 2000).These slow LPP waves may extend over several milliseconds oreven seconds as in the recent study by Pastor et al. (2008), witha paradigm where affective pictures were presented in blocks ofsimilarly valenced content or in intermixed presentation mode. Inour study, the LPP to the liking judgment task may denote the affec-tive strategy employed by subjects to decide whether they liked amusical passage or not. A hedonic color hence characterized the
valenced evaluative processes tagged by this study.
Our results, in sum, show that a relatively short succession ofchords is sufficient to elicit distinct neural correlates of liking andcorrectness judgments. Moreover, these neural correlates were notentirely due to processing of the acoustic features of the incongru-

l Psych

oaotomaeEcasacbjlnljtaaettbt

4

plwctme2

eomclhr2brsat(aaernot

o‘r


us chords, since similar ERP responses were observed in the ERPnalysis locked only to those stimuli (mainly belonging to ambigu-us, but also to the incongruous and congruous stimulus categories)hat produced opposite behavioral answers when presented twice,nce for the liking judgment task and once for the correctness judg-ent task. In other words, the brain responses to the behavioral

nswers obtained with this discordant-answer ERP averaging werelicited by the same set of stimuli in both tasks; hence, the observedRP differences could not be ascribed merely to the sensory pro-essing of the manipulated chords. To note the stimuli in our studyre susceptible of improvement in future endeavors focusing onubjectivity of judgment processes in music: the last remains ofpossible confound represented by acoustic processes could be

ancelled by avoiding the most obvious harmonic violations, i.e.,y selecting as stimulus material only those chords that would be

udged as incorrect according to the preceding context. Neverthe-ess, our ERP results, showing differences in affective and cognitiveeural processes triggered by the ending chord of a cadence, are in

ine with the findings by Steinbeis et al. (2006). In that study, sub-ects were asked during a behavioral session to rate the perceivedension, the intensity of subjective emotion felt during the musicnd the overall emotionality of the music. One single chord, placedt the end of a tonal cadence and varying in the degree of expect-dness, was able to affect the behavioral ratings of local perceivedension, overall emotionality of the music, the chord-induced elec-rodermal activity (a measure of autonomic arousal), and the ERANrain responses (although those were computed without regard ofhe subjects’ behavioral responses).

.4. Scalp topographies of the ERPs obtained

The role of the right-hemisphere for the generation of the lateotentials elicited by musical sounds was predominant, regard-

ess of the listening strategies adopted. This observation is in lineith the music-neuroscientific literature showing the stronger

ontribution of the right hemisphere for cognitive processing ofhe pitch changes intrinsic in melodies (Zatorre et al., 1994), har-

ony rules of chord cadences (e.g., Koelsch et al., 2005; Tillmannt al., 2003), and passive listening to melodies (Patterson et al.,002).

Conversely, while the ERAN to in congruous chords objectivelyvaluated on the basis of music-theory was bilaterally distributedver the scalp, the early negativity obtained for the liking judg-ent task in the main ERP averaging was lateralized to the right

erebral hemisphere. In a speculative vein, this finding could beinked to emotion theories postulating a larger role of the rightemisphere for affective processing, substantiated also by neu-oimaging findings on emotional speech (or prosody; Adolphs et al.,002; Schirmer et al., 2001). The ERAN generators are supposed toe located in the bilateral inferofrontal operculum, a structure alsoesponsible for processing emotional prosody in the right hemi-phere (Adolphs et al., 2002). It is reasonable to suppose that theffective processes leading to a liking judgment would modulatehe lateralization of the related ERAN response. Besides genderKoelsch et al., 2003), affective processes might hence represent andditional candidate factor accounting for the variation in the later-lization of the ERAN response observed in the literature (e.g., Louit al., 2005; Leino et al., 2007). We should, however, note that suchight-lateralization of the ERAN to the liking judgment task wasot replicated in the ERP waveforms locked to discordant answersnly, most likely due to their low signal to noise ratio compared to
he main ERP analysis.
Furthermore, at the very late latencies subsequent to the endf the stimulus, the right lateralization was mostly visible for the

incorrect’ answers of the correctness judgment task. The LPP brainesponse to the liking judgment task was, instead, lateralized to

ology 85 (2010) 393–409 407

the right frontal lobe for both answers (‘like’ and ‘dislike’), indicat-ing a larger role for this brain structure in an affective evaluativeprocess. The differential lateralization for the negative and posi-tive answers of the correctness judgment task may be linked tomodels of emotion processing (Davidson, 1993; Heilman, 1997),according to which the two cerebral hemispheres play a distinctrole in the regulation of the activity of the limbic system andhence in the subjective experience of positive and negative emo-tions (the valence lateralization hypothesis). Particularly, the rightfrontal lobe is supposed to control the perception and inductionof negative emotions (such as sadness, depression, unpleasant-ness) whereas positive emotional experiences (such as happiness,joy, pleasantness) would rely on the intact contribution of the leftfrontal lobe. The present results may therefore give a hint for astrong right-hemispheric lateralization for the negative outcomein cognitive judgment processes vs. a larger role of the right hemi-sphere, and specifically its frontal region, for affective judgmentprocesses in general. Nevertheless, in the musical domain very littleelectrophysiological evidence supports the valence lateralizationhypothesis (Altenmüller et al., 2002; Schmidt and Trainor, 2001).The largest group of studies, adopting brain metabolic measures(Blood and Zatorre, 2001; Blood et al., 1999; Brown et al., 2004;Khalfa et al., 2005) shows a comparable participation of limbic andparalimbic or supratemporal structures of the left and right hemi-sphere to the affective content or valence of music. Similarly, theliterature on processing of visual pictures evoking strong emotionsshows bilateral LPP, especially when the valence of the emotions istaken into account (cf. Moser et al., 2006; Schupp et al., 2000; butfor an exception see Cunningham et al., 2005).

To conclude, the present study showed early negative and latepositive ERP responses to negative judgments, possibly reflectingan initial close connection between sensory, cognitive and eval-uative neural processes. At very late latencies, following the endof the musical stimulus, the neural processes underlying likingand correctness judgments diverged, hinting at affective strategiesemployed for liking decisions and memory integration mechanismsleading to correctness decisions. Behavioral ratings further sup-ported the adoption of different strategies employed by subjectsduring the experimental tasks. This pattern of findings conse-quently confirms the hypothesis that affective liking processesoccur according to several stages, from initial feature extractionand incongruity detection, to conscious processing and finally lateconscious appraisal and decision making (Jacobsen and Höfel, 2003;Leder et al., 2004; Scherer, 2004). Our electrophysiological inves-tigation indicates for the first time the existence of distinct neuralstructures underlying cognitive vs. affective modes and judgmentsduring music listening. Further studies with higher spatial resolu-tion techniques are called for to disentangle the role of specific brainstructures and possibly of the two cerebral hemispheres in differ-ent listening modes and judgment processes in music as well asthe amount of subjectivity and individual strategies adopted duringsuch processes.

Acknowledgements

The authors wish to thank Noa Nakai for help in preparing thestimuli, and Mira Müller for suggestions concerning the analysis ofthe behavioral data. The study was supported by the project FP6-2004-NEST-PATH-028570-BrainTuning.

References

Adolphs, R., Damasio, H., Tranel, D., 2002. Neural systems for recognition of emo-tional prosody: a 3-D lesion study. Emotion 2, 23–51.

Altenmüller, E., Schurmann, K., Lim, V.K., Parlitz, D., 2002. Hits to the left, flops tothe right: different emotions during listening to music are reflected in corticallateralisation patterns. Neuropsychologia 40, 2242–2256.

4 l Psych

B

B

B

B

B

B

B

B

B

B

B

B

B

B

C

C

C

C

D

D

E

F

F

F

G

H

H

J

J

J

K

K


erlyne, D.E., 1971. Aesthetics and Psychobiology. Appleton-Century-Crofts, NewYork.

erridge, K.C., Robinson, T.E., 2003. Parsing reward. Trends in Neurosciences 26,507–513.

esson, M., Faita, F., 1995. An event-related potential (ERP) study of musicalexpectancy: comparison of musicians with nonmusicians. Journal of Experimen-tal Psychology: Human Perception and Performance 21, 1278–1296.

esson, M., Faita, F., Requin, J., 1994. Brain waves associated with musical incon-gruities differ for musicians and non-musicians. Neuroscience Letters 168,101–105.

esson, M., Schön, D., 2003. Comparison between language and music. In: Peretz, I.,Zatorre, R. (Eds.), The Cognitive Neuroscience of Music. Oxford University Press,New York, NY, pp. 269–293.

irbaumer, N., Elbert, T., Canavan, A.G., Rockstroh, B., 1990. Slow potentials of thecerebral cortex and behavior. Physiological Reviews 70, 1–41.

lood, A.J., Zatorre, R.J., 2001. Intensely pleasurable responses to music correlatewith activity in brain regions implicated in reward and emotion. Proceedings ofthe National Academy of Sciences 98, 11818–11823.

lood, A.J., Zatorre, R.J., Bermudez, P., Evans, A.C., 1999. Emotional responses to pleas-ant and unpleasant music correlate with activity in paralimbic brain regions.Nature Neuroscience 2, 382–387.

rattico, E., 2006. Cortical Processing of Musical Pitch as Reflected by Behaviouraland Electrophysiological Evidence. Helsinki University Press, Helsinki.

rattico, E., Jacobsen, T., 2009. Subjective appraisal of music: Neuroimaging evi-dence. Annals of the New York Academy of Sciences 1169, 308–317.

rattico, E., Jacobsen, T., forthcoming. Toward a model of the neural and mentalchronometry of the aesthetic experience of music. Cortex.

rattico, E., Tervaniemi, M., 2006. Musical creativity and the human brain. In:Deliege, I., Wiggins, G.A. (Eds.), Musical Creativity: Current Research in Theoryand Practice. Psychology Press, London.

rattico, E., Tervaniemi, M., Näätänen, R., Peretz, I., 2006. Musical scale propertiesare automatically processed in the human auditory cortex. Brain Research 1117,162–174.

rown, S., Martinez, M.J., Parsons, L.M., 2004. Passive music listening spontaneouslyengages limbic and paralimbic systems. NeuroReport 15, 2033–2037.

acioppo, J.T., Crites Jr., S.L., Gardner, W.L., Bernston, G.G., 1994. Bioelectrical echoesfrom evaluative categorizations. I. A late positive brain potential that varies asa function of trait negativity and extremity. Journal of Personality and SocialPsychology 67, 115–125.

och, D., Sanders, L.D., Neville, H.J., 2005. An event-related potential study of selec-tive auditory attention in children and adults. Journal of Cognitive Neuroscience17, 605–622.

rites Jr., S.L., Cacioppo, J.T., Gardner, W.L., Berntson, G.G., 1995. Bioelectrical echoesfrom evaluative categorization. II. A late positive brain potential that varies as afunction of attitude registration rather than attitude report. Journal of Person-ality and Social Psychology 68, 997–1013.

unningham, W.A., Espinet, S.D., DeYoung, C.G., Zelazo, P.D., 2005. Attitudes to theright- and left: frontal ERP asymmetries associated with stimulus valence andprocessing goals. NeuroImage 28, 827–834.

avidson, R.J., 1993. The neuropsychology of emotion and affective style. In: Lewis,M., Haviland, J.M. (Eds.), Handbook of Emotion. Guilford Press, New York, pp.143–154.

elorme, A., Makeig, S., 2004. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics. Journal of Neuroscience Methods 134, 9–21.

lbert, T., Ulrich, R., Rockstroh, B., Lutzenberger, W., 1991. The processing of tem-poral intervals reflected by CNV-like brain potentials. Psychophysiology 28,648–655.

alkenstein, M., Hoormann, J., Hohnsbein, J., Kleinsorge, T., 2003. Short-term mobi-lization of processing resources is revealed in the event-related potential.Psychophysiology 40, 914–923.

lores-Gutierrez, E.O., Diaz, J.L., Barrios, F.A., Favila-Humara, R., Guevara, M.A., delRio-Portilla, Y., Corsi-Cabrera, M., 2007. Metabolic and electric brain patternsduring pleasant and unpleasant emotions induced by music masterpieces. Inter-national Journal of Psychophysiology 65, 69–84.

ox, N.A., 1991. If it’s not left, it’s right: electroencephalograph asymmetry and thedevelopment of emotion. American Psychologist 46, 863–872.

agnon, L., Peretz, I., 2000. Laterality effects in processing tonal and atonal melodieswith affective and nonaffective task instructions. Brain and Cognition 43,206–210.

eilman, K.M., 1997. The neurobiology of emotional experience. Journal of Neu-ropsychiatry and Clinical Neuroscience 9, 439–448.

illyard, S.A., Hink, R.F., Schwent, V.L., Picton, T.W., 1973. Electrical signs of selectiveattention in the human brain. Science 182, 177–180.

acobsen, T., Höfel, L., 2003. Descriptive and evaluative judgment processes: behav-ioral and electrophysiological indices of processing symmetry and aesthetics.Cognitive, Affective, & Behavioral Neuroscience 3, 289–299.

acobsen, T., Schubotz, R.I., Hofel, L., Cramon, D.Y., 2006. Brain correlates of aestheticjudgment of beauty. Neuroimage 29 (1), 276–285.

ohannson, M., Mecklinger, A., 2003. The late posterior negativity in ERP studiesof episodic memory: action monitoring and retrieval of attribute conjunctions.
Biological Psychology 64, 91–117.
halfa, S., Schön, D., Anton, J.L., Liegeois-Chauvel, C., 2005. Brain regions involvedin the recognition of happiness and sadness in music. Neuroreport 16, 1981–1984.

oelsch, S., 2009. Music-syntactic processing and auditory memory: similarities anddifferences between ERAN and MMN. Psychophysiology 46, 179–190.

ology 85 (2010) 393–409

Koelsch, S., Fritz, T., Schulze, K., Alsop, D., Schlaug, G., 2005. Adults and childrenprocessing music: an fMRI study. Neuroimage 25, 1068–1076.

Koelsch, S., Gunter, T.C., Friederici, A.D., Schröger, E., 2000. Brain indices of musicprocessing: “non-musicians” are musical. Journal of Cognitive Neuroscience 12,520–541.

Koelsch, S., Maess, B., Grossmann, T., Friederici, A.D., 2003. Electric brain responsesreveal gender differences in music processing. Neuroreport 14, 709–713.

Koelsch, S., Siebel, W.A., 2005. Towards a neural basis of music perception. Trendsin Cognitive Sciences 9, 578–584.

Koelsch, S., Schmidt, B.-H., Kansok, J., 2002. Effects of musical expertise on the earlyright anterior negativity: an event-related brain potential study. Psychophysi-ology 39, 657–663.

Kreutz, G., Schubert, E., Mitchell, L.A., 2008. Cognitive styles of music listening. MusicPerception 26, 57–73.

Krohn, K., Brattico, E., Valimäki, V., Tervaniemi, M., 2006. Neural representations ofhierarchical scale pitch structure. Music Perception 24, 281–296.

Krumhansl, C.L., 1990. Cognitive Foundations of Musical Pitch. Oxford UniversityPress, New York.

Kuriki, S., Isahai, N., Ohtsuka, A., 2005. Spatiotemporal characteristics of the neuralactivities processing consonant/dissonant tones in melody. Experimental BrainResearch 162, 46–55.

Leder, H., Belke, B., Oeberst, A., Augustin, D., 2004. A model of aesthetic appreciationand aesthetic judgements. British Journal of Psychology 95, 489–508.

Leino, S., Brattico, E., Tervaniemi, M., Vuust, P., 2007. Representation of harmonyrules in the human brain: further evidence from event-related potentials. BrainResearch 1142, 169–177.

Leuthold, H., Sommer, W., Ulrich, R., 2004. Preparing for action: inferences from CNVand LRP. Journal of Psychophysiology 18, 77–88.

Loui, P., Grent-’t-Jong, T., Torpey, D., Woldorff, M., 2005. Effects of attention onthe neural processing of harmonic syntax in Western music. Cognitive BrainResearch 25, 678–687.

Luck, S.J., 2005. Event-Related Potentials: A Methods Handbook. MIT Press, Cam-bridge, Mass.

Luo, Y.-J., Wei, J.-H., 1999. Cross-modal selective attention to visual and auditorystimuli modulates endogenous ERP components. Brain Research 842, 30–38.

Maess, B., Koelsch, S., Gunter, T.C., Friederici, A.D., 2001. Musical syntax is processedin Broca’s area: an MEG study. Nature Neuroscience 4, 540–545.

Mandler, G., Shebo, B.J., 1983. Knowing and liking. Motivation and Emotion 7,125–144.

Meyer, L.B., 1956. Emotion and Meaning in Music. The University of Chicago Press,Chicago, London.

Moser, J.S., Hajcak, G., Bukay, E., Simons, R.F., 2006. Intentional modulation of emo-tional responding to unpleasant pictures: an ERP study. Psychophysiology 43,292–296.

Nagai, Y., Critchley, H.D., Featherstone, E., Fenwick, P.B., Trimble, M.R., Dolan, R.J.,2004. Brain activity relating to the contingent negative variation: an fMRI inves-tigation. Neuroimage 21, 1232–1241.

North, A.C., Hargreaves, D.J., 1997a. Experimental aesthetics and everyday musiclistening. In: Hargreaves, D.J., North, A.C. (Eds.), The Social Psychology of Music.Oxford University Press, New York, Oxford, pp. 84–103.

North, A.C., Hargreaves, D.J., 1997b. Liking, arousal potential, and the emotionsexpressed by music. Scandinavian Journal of Psychology 38, 45–53.

North, A.C., Hargreaves, D.J., 2007. Lifestyle correlates of musical preference: 3.Travel, money, education, employment and health. Psychology of Music 35,473–497.

Pastor, M.C., Bradley, M.M., Löw, A., Versace, F., Moltó, J., Lang, P.J., 2008. Affec-tive picture perception: emotion, context, and the late positive potential. BrainResearch 1189, 145–151.

Patel, A.D., Gibson, E., Ratner, J., Besson, M., Holcomb, P.J., 1998. Processing syntacticrelations in language and music: an event-related potential study. Journal ofCognitive Neuroscience 10, 717–733.

Patterson, R.D., Uppenkamp, S., Johnsrude, I.S., Griffiths, T.D., 2002. The processingof temporal pitch and melody information in auditory cortex. Neuron 36, 767–776.

Peretz, I., Gagnon, L., Bouchard, B., 1998. Music and emotion: perceptual determi-nants, immediacy, and isolation after brain damage. Cognition 68, 111–141.

Peretz, I., Zatorre, R. (Eds.), 2003. The Cognitive Neuroscience of Music. Oxford Uni-versity Press, Oxford.

Piston, W., 1962. Harmony, 3rd ed. W.W. Norton and Company.Power, M., Dalgleish, T., 1999. Two routes to emotion: some implications of multi-

level theories of emotion for therapeutic practice. Behavioural and CognitivePsychotherapy 27, 129–141.

Rees, G., 2007. Neural correlates of the contents of visual awareness in humans.Philosophical Transactions of the Royal Society B Biological Sciences 362,877–886.

Regnault, P., Bigand, E., Besson, M., 2001. Different brain mechanisms mediate sen-sitivity to sensory consonance and harmonic context: evidence from auditoryevent-related brain potentials. Journal of Cognitive Neuroscience 13, 241–255.

Scherer, K.R., 2004. Which emotions can be induced by music? What are the under-lying mechanisms? And how can we measure them? Journal of New Music
Research 33, 239–251.
Scherer, K.R., Zentner, M.R., 2001. Emotional effects of music: production rules. In:Juslin, P.N., Sloboda, J.A. (Eds.), Music and Emotion: Theory and Research. OxfordUniversity Press, Oxford, pp. 361–392.

Schirmer, A., Alter, K., Kotz, S.A., Friederici, A.D., 2001. Lateralization of prosodyduring language production: A lesion study. Brain and Language 76, 1–17.

l Psych

S

S

S

S

S

S

S

S

Psychologist 35, 151–175.Zatorre, R., Evans, A.C., Meyer, E., 1994. Neural mechanisms underlying melodic


chmidt, L.A., Trainor, L.J., 2001. Frontal electrical activity (EEG) distinguishesvalence and intensity of musical emotions. Cognition and Emotion 15, 487–500.

chupp, H.T., Cuthbert, B.N., Bradley, M.M., Cacioppo, J.T., Ito, T., Lang, P.J., 2000.Affective picture processing: the late positive potential is modulated by moti-vational relevance. Psychophysiology 37, 257–261.

chön, D., Besson, M., 2005. Visually induced auditory expectancy in music reading:a behavioral and electrophysiological study. Journal of Cognitive Neuroscience17, 694–705.

chön, D., Regnault, P., Ystad, S., Besson, M., 2005. Sensory consonance: an ERP study.Music Perception 23, 105–117.

impson, E.H., 1951. The interpretation of interaction in contingency tables. Journalof the Royal Statistical Society Series B – Statistical Methodology 13, 238–241.

inghal, A., Fowler, B., 2005. The effects of memory scanning on the late Nd andP300: an interference study. Psychophysiology 42, 142–150.

mith, J.D., Kemler Nelson, D.G., Grohskoff, L.A., Appleton, T., 1994. What child isthis? What internal was that? Familiar tunes and music perception in novicelisteners. Cognition 52, 23–54.

teinbeis, N., Koelsch, S., Sloboda, J.A., 2006. The role of harmonic expectancy viola-tions in musical emotions: evidence from subjective, physiological, and neuralresponses. Journal of Cognitive Neuroscience 18, 1380–1393.

ology 85 (2010) 393–409 409

Tecce, J.J, 1972. Contingent negative variation (CNV) and psychological processes inman. Psychological Bulletin 77, 73–108.

Tervaniemi, M., Huotilainen, M., Brattico, E., Reinikainen, K., Ilmoniemi, R.J., Alho,K., 2003. Event-related potentials to expectancy violation in musical context.Musicae Scientiae 7, 241–261.

Tillmann, B., Bharucha, J.J., Bigand, E., 2000. Implicit learning of tonality: a self-organizing approach. Psychological Review 107, 885–913.

Tillmann, B., Janata, P., Bharucha, J.J., 2003. Activation of the inferior frontal cortexin musical priming. Cognitive Brain Research 16, 145–161.

van Boxtel, G.J.M., Böcker, K.B.E., 2004. Cortical measures of anticipation. Journal ofPsychophysiology 18, 61–76.

Zajonc, R.B., 1980. Feeling and thinking: preferences need no inferences. American

perception and memory for pitch. Journal of Neuroscience 14, 1908–1919.

Zelazo, P.D., 2004. The development of conscious control in childhood. Trends inCognitive Sciences 8, 12–17.

Cognitive vs. affective listening modes and judgments of music â€“ An ERP study · 2012. 6....

Documents

Transcript of Cognitive vs. affective listening modes and judgments of music â€“ An ERP study · 2012. 6....