1

Title of PhD Thesis: Mirror-touch synaesthesia: The role of shared

representations in social cognition

Michael Joseph Banissy

Institute of Cognitive Neuroscience

University College London

PhD in Cognitive Neuroscience

I, Michael Joseph Banissy, confirm that the work presented in this thesis is my own.

Where information has been derived from other sources, I confirm that this has been

indicated in the thesis.

2

TABLE OF CONTENTS

Abstract 5

Publications Arising from Thesis 6

Acknowledgements 7

Chapter 1: Introduction 8

Chapter 2: Prevalence and Characteristics of Mirror-Touch Synaesthesia 36

Chapter 3: Sensory Processing in Synaesthesia 63

Chapter 4: Mirror-Touch Synaesthesia and Empathy 78

Chapter 5: Facial Expression Recognition in Mirror-Touch Synaesthesia 95

Chapter 6: A Methodological Introduction to TMS 112

Chapter 7: The Role of Sensorimotor Simulation in Auditory Emotion 125

Discrimination

Chapter 8: The Role of Sensorimotor Simulation in Facial Expression 140

Recognition

Chapter 9: Conclusions 155

References 175

Appendix 212

3

LIST OF FIGURES AND TABLES

Figure 1.1 fMRI data from a control and grapheme-colour synaesthetic subject when presented with synaesthesia inducing graphemes. 16 Figure 1.2 Summary of Muggleton et al. (2007). 19 Figure 1.3 Activations resulting from the interaction between mirror-touch synaesthete ‘C’ and non-synaesthetic control participants. 30 Figure 1.4 Sepcular and anatomical mappings reported by mirror-touch synaesthetes. 31 Figure 1.5 Summary of Banissy and Ward (2007). 32 Figure 2.1 Summary of task used to confirm potential cases of synaesthesia in experiment 1. 43 Figure 2.2 Mean reaction time performance and percentage of error tyes on human trials for mirror-touch synaesthetes recruited within the prevalence study compared to synaesthetes recruited via self referral. 49 Figure 2.3 Summary of task used for somatotopic cueing experiment. 51 Figure 2.4 Mean reaction time performance and percentage of error types for mirror-touch synaesthetes and non-synaesthetic control subjects observing a light flash on another person’s face or light flash only. 52 Figure 2.5 The ‘Who, What, Where Model of Mirror-Touch Synaesthesia’. 56 Figure 2.6 The influence of perspective on synaesthetic experience. 58 Figure 3.1 Synaesthetes who experience colour outperform individuals who do not experience synaesthetic colour on a measure of colour perception. 70 Figure 3.2 Synaesthetes who experience touch outperform individuals who do not experience synaesthetic touch on a measure of tactile perception. 72 Figure 4.1 Mirror-touch synaesthetes show significantly higher scores than controls on the emotional reactivity componenet, but not other components, of the Empathy Quotient. 83 Figure 4.2. In experiment 2, mirror-touch synaesthetes show significantly higher scores than controls on the emotional reactivity componenet, but not other components, of the Empathy Quotient. 88 Figure 4.3 Mirror-touch synaesthetes show significantly higher scores than controls on the fantasizing componenet, but not other components, of the Interpersonal Reactivity Index. 89 Figure 4.4 Mirror-touch synaesthetes show significantly higher scores than controls on the Openess subscale, but not other components, of the BFI. 89 Figure 5.1 Summary of tasks used. 102 Figure 5.2 Mean accuracy and reaction time performances of synaesthetes and controls on the films facial expression task. 104 Figure 5.3 Mean accuracy performances on the CFMT and CFMT+. 105 Figure 5.4 Mean error score for synaesthetes and controls on upright and inverted trials of the CFPT. 107 Figure 5.5 Mean accuracy performances on the expression matching and identity matching task for synaesthetes and controls. 108 Figure 6.1 The sequence of events for a transcranial magnetic stimulation pulse. 116 Figure 6.2 The effects of fibre orientation and electric field orientation for the application of a TMS pulse. 117 Figure 6.3 TMS-induced electrical fields produced by circurlar and figure-of-eight shaped coils. 117

4

Figure 6.4 The spatial and temporal resolution of TMS compared with other experimental techniques 118Figure 6.5 Functional dissociation method that can be employed using TMS. 121 Figure 7.1 Summary of cTBS and task protocol. 131 Figure 7.2 Summary of TBS sites stimulated. 133 Figure 7.3 Magnitude of disruption or facilitation in milliseconds following cTBS targeted at rSI, rPM and the vertex across task groups. 135 Figure 8.1 Summary of TMS sites stimulated. 146 Figure 8.2 Magnitude of disruption or facilitation across expression types following cTBS targeted at rSI, rIFG, and right V5 / MT. 149 Figure 9.1 The ‘Who, What, Where Model of Mirror-Touch Synaesthesia’. 164 Table 2.1 Reaction time performance and percentage of mirror-touch errors and other error types for potential mirror-touch synaesthetes when observing a human or corresponding object being touched. 48

5

ABSTRACT

Synaesthesia is a condition in which one property of a stimulus results in conscious

experiences of an additional attribute. In mirror-touch synaesthesia, the synaesthete

experiences a tactile sensation on their own body simply when observing touch to

another person. This thesis investigates the prevalence, neurocognitive mechanisms,

and consequences of mirror-touch synaesthesia. Firstly, the prevalence and

neurocognitive mechanisms of synaesthesia were assessed. This revealed that mirror-

touch synaesthesia has a prevalence rate of 1.6%, a finding which places mirror-touch

synaesthesia as one of the most common variants of synaesthesia. It also indicated a

number of characteristics of the condition, which led to the generation of a

neurocognitive model of mirror-touch synaesthesia. An investigation into the

perceptual consequences of synaesthesia revealed that the presence of synaesthesia is

linked with heightened sensory perception - mirror-touch synaesthetes showed

heightened tactile perception and grapheme-colour synaesthetes showed heightened

colour perception. Given that mirror-touch synaesthesia has been shown to be linked

to heightened sensorimotor simulation mechanisms, the impact of facilitated

sensorimotor activity on social cognition was then examined. This revealed that

mirror-touch synaesthetes show heightened emotional sensitivity compared with

control participants. To compliment this, two transcranial magnetic stimulation

(TMS) studies were then conducted to assess the impact of suppressing sensorimotor

activity on the expression recognition abilities of healthy adults. Consistent with the

findings of superior emotion sensitivity in mirror-touch synaesthesia (where there is

facilitated sensorimotor activity), suppressing sensorimotor resources resulted in

impaired expression recognition across modalities. The findings of the thesis are

discussed in relation to neurocognitive models of synaesthesia and of social cognition.

6

PUBLICATIONS ARISING FROM THESIS

Research Articles:

Banissy, M. J., and Ward, J. (2007). Mirror-touch synaesthesia is linked with

empathy. Nature Neuroscience, 10, 815-816.

Banissy, M. J., Walsh, V., and Ward, J (2009). Enhanced sensory perception in

synaesthesia. Experimental Brain Research, 196, 565-571.

Banissy, M. J., Cohen Kadosh, R., Maus, G. W., Walsh, V., & Ward, J. (2009).

Prevalence, characteristics, and a neurocognitive model of mirror-touch

synaesthesia. Experimental Brain Research, 198, 261-272.

Banissy, M. J., Sauter, D. A., Ward, J, Warren, J. E., Walsh, V., and Scott, S.

(Submitted). Suppressing sensorimotor activity modulates the discrimination

of auditory emotions but not speaker identity. Journal of Neuroscience.

Banissy, M. J., Garrido, L., Kusnir, F., Duchaine, B., Walsh, V, and Ward, J.

(Submitted). Superior facial expression but not idenitity recognition in mirror-

touch synaesthesia. Journal of Neuroscience.

Invited Book Chapters and Reviews:

Banissy, M. J., and Ward, J. (2008). On being moved: From mirror neurons to

empathy. Child and Adolescent Mental Health, 13, 50-51.

Ward, J., Banissy, M. J., and Jonas, C. (2008). Haptic perception in synaesthesia. In

Human Haptic Perception: Basics and Applications, Edited by

M.Grunwald. Birkhäuser, Basel.

7

ACKNOWLEDGEMENTS

Firstly, I would like Vincent Walsh for his unparalleled support and guidance. I

would also like to thank Jamie Ward for providing me with the opportunity to

complete this research, for his support, and for his advice. The two of them have

shaped my intellectual development and I will be forever grateful.

I am also thankful to a large number of my friends and colleagues at University

College London and the University of Sussex for their support. Of note, Roi Cohen

Kadosh, Brad Duchaine, Lúcia Garrido, Amir Javadi, Clare Jonas, Ryota Kanai, Neil

Muggleton, David Pitcher, Sophie Scott, and Lilli Tcheang. There are also a number

of friends and family who have provided me with encouragement and support over

the years; Lucy Annett, Edith Cole, Margaret Cole, Stanley Cole, Steven Cole, Claire

Doyle, Martin Family, Stephen Ford, Davina Reynolds, and Mark Taylor.

My sincerest thanks and love goes to Sian Fitzpatrick. Her encouragement, support

and faith in me have been incredible. I am thankful to you for so many reasons and

more grateful than I could ever say.

Finally, deepest thanks go to my parents, for their unconditional support and love

throughout the years. To my sister Jasmine and to my brother Jamie – you are both an

inspiration and I only wish I had the words to explain how much. Thank you for

everything. This is for you all, with love always.

Chapter 1

8

CHAPTER 1: INTRODUCTION

This chapter provides a summary of the motivation to investigate mirror-touch

synaesthesia. It proposes that synaesthesia relies upon similar mechanisms of

multisensory interactions that are present in non-synaesthetic individuals and can be

used to inform normal models of cognitive processing. The condition of synaesthesia

is introduced and the prevalence and characteristics of the condition are discussed.

An overview of current psychological and neurobiological studies is provided which

reveals insights into the neurocognitive mechanisms of synaesthesia and demonstrates

how the condition makes use of neural pathways involved in normal sensory

perception. Recent research demonstrates that developmental mirror-touch

synaesthesia appears to rely upon activation of the same somatosensory

representations within the human mirror-touch system that are activated when non-

synaesthetic individuals observe another person being touched The aims of this thesis

are described. These include i) investigations into the neurocognitive mechanisms of

mirror-touch synaesthesia and ii) investigations into the role of sensorimotor

simulation processes in emotion processing and empathy.

1.1 Origins of synaesthesia

Derived from the Greek roots syn (meaning together) and aisthesis (meaning

sensation), synaesthesia is a condition in which one property of a stimulus (the

inducer) gives rise to a conscious experience of a different attribute (the concurrent;

Grossenbacher and Lovelance, 2001). For example, in tone-colour synaesthesia,

sounds may elicit the experience of colour (Ward, Huckstep, and Tsakanikos, 2006);

in grapheme-colour synaesthesia, the visual presentation of achromatic letters or

numbers results in subjective experiences of colour (Simner et al., 2006); and in

lexical-gustatory synaesthesia, words trigger synaesthetic experiences of taste (Ward,

Simner and Auyeung, 2005).

Accounts of the condition can be traced to the 19th century (c.f. Jewanski, Day,

and Ward, 2009). For example, Sachs (1812) describes synaesthesia involving

colours for music and simple sequences. Later, Galton (1880) described cases of

individuals in whom numbers were consciously visualized in space (spatial number

forms) and of synaesthesia involving colour. While early accounts of the condition

Chapter 1

9

aroused some interest, failure to develop an objective approach to confirm the

phenomenon resulted in a decline in research. It was not until the advent of the

development of new behavioural and neurophysiological measures which could be

used to corroborate self reports that the topic of synaesthesia returned as a topic of

legitimate scientific investigation (Baron-Cohen et al., 1987; Cytowic and Wood,

1982; Marks, 1975; see Ramachandran and Hubbbard, 2001; Rich and Mattingely,

2002; Sagiv, 2004 for reviews).

Since this time, research into the topic of synaesthesia has grown rapidly with

a focus moving beyond exploring the reality of the condition to consider how

synaesthesia can be used to inform models of typical cognition in domains such as

numerical cognition (Cohen Kadosh and Henik, 2007), language (Simner, 2007),

multisensory processing (Sagiv and Ward, 2006), imagery (Barnett and Newell, 2008;

Spiller and Jansari, 2008), and attention (Treisman, 2004). In this introductory

chapter, I will review this literature by describing studies on the prevalence,

authenticity and aetiology of synaesthesia. The focus of this thesis is to investigate a

newly documented type of synaesthesia, mirror-touch synaesthesia (in which

individuals’ experience tactile sensations on their own body simply when observing

touch to others) and to use this condition to examine more general neurocognitive

processes in social cognition. In this chapter, I will introduce research investigating

synaesthesia involving touch and consider the evidence that mirror-touch synaesthesia

relies upon similar mechanisms to multisensory interactions which are shown in non-

synaesthetic individuals. Finally, I will discuss how synaesthesia may be used to

inform typical models of cognition and discuss the role of sensorimotor simulation in

social cognition.

Chapter 1

10

1.2 Prevalence and characteristics of synaesthesia

Synaesthesia is typically considered as having three defining features; 1)

experiences are conscious perceptual or percept-like experiences; 2) experiences are

induced by an attribute not typically associated with the conscious experience; 3)

these experiences occur automatically (Ward and Mattingley, 2007). Further, the

synaesthetic percept tends to co-exist with the percept of the inducing stimulus rather

than over-riding it – for example in lexical-gustatory synaesthesia written or heard

words are recognised but also result in a simultaneous subjective sensation of taste in

the mouth and tongue area (Ward and Simner, 2003). Note that throughout the thesis

the terminology of referring to different types of synaesthesia in terms of inducer-

concurrent pairs separated with a hyphen is used. As such, touch-colour synaesthesia

refers to tactile inducers eliciting a concurrent experience of colour, and vision-touch

synaesthesia refers to a visual inducer eliciting a tactile experience.

Cases of synaesthesia can be either developmental or acquired, with

developmental cases thought to be dependent upon genetic and environmental factors

and acquired cases reflecting synaesthesia following specific environmental

influences (e.g. following brain injury or drug ingestion). Developmental forms of

synaesthesia have been shown to run in families and previous research suggested that

the condition may be more common in women than men, which may reflect an X-

linked dominant mode of inheritance (Baron-Cohen, Burt, Smith-Laittan, Harrison,

and Bolton, 1996). More recent research indicates that synaesthesia may be equally

common within males and females and that previous methodologies may have led to

an over inflated male-female ratio (Ward and Simner, 2005; but see Barnett et al.,

2008). Similarly, reports of twins discordant for synaesthesia (Smilek, Mofatt,

Pasternak, White, Dixon, and Merilke, 2002), as well as evidence that synaesthesia

Chapter 1

11

can skip generations (Hubbard and Ramachandran, 2003), and that the proportion of

sons or daughters born to synaesthete mothers does not significantly differ (Barnett et

al., 2008; Ward and Simner, 2005), suggest that an X-linked dominant mode of

inheritance may be an over simplified account of the genetic mechanisms which

underlie developmental forms of the condition (Asher et al., 2009).

Current estimates on the prevalence of developmental synaesthesia indicate

that the condition has a prevalence rate of at least 4% and a female to male ratio of 1:1

(Simner et al., 2006; Ward and Simner, 2005). Although, depending on whether one

includes cases of ordinal linguistic personification (in which individuals attribute

genders or personalities to letters or numbers; Simner and Holenstein, 2007) or spatial

number forms (Sagiv, Simner, Collins, Butterworth, and Ward, 2006), the prevalence

rate of 4% is likely to be much higher (Simner et al., 2006).

A trend of all studies of the prevalence of synaesthesia is to report a higher

proportion of synaesthetes who experience colour evoked by letters or other linguistic

stimuli (e.g. grapheme-colour / day-colour synaesthesia; Baron-Cohen, Burt, Smith-

Laittan, Harrison, and Bolton, 1996; Rich, Bradshaw, and Mattingley, 2005; Simner

et al., 2006). It is perhaps not surprising that this variant of synaesthesia has been the

topic of much research amongst synaesthesia researchers. Research into this variant

of the condition has highlighted a number of interesting individual differences

between synaesthetes. For example, distinctions have been made between projector

and associator synaesthetes; which distinguishes between synaesthetes whose locus of

experienced colour is projected to a specific spatial location (projector synaesthetes)

and synaesthetes whose concurrent is perceived internally or in the ‘minds eye’

(associator synaesthetes) (Dixon, Smilek, and Merilke, 2004; see also Ward, Li, Salih,

and Sagiv, 2007). Similarly, Ramachandran and Hubbard (2001) have categorised

Chapter 1

12

synaesthetes based upon the level of induction of the synaesthesia, distinguishing

between higher synaesthetes, in whom conceptual properties of a grapheme trigger

colours (e.g. a number name or dice pattern for a particular number), and lower

synaesthetes in whom the physical properties of the grapheme (e.g. shape / form)

trigger synaesthetic experience. Distinctions such as these may also be valid for other

variants of synaesthesia (see below).

1.3 Authenticity and perceptual nature of synaesthesia

1.3.1 Authenticity of synaesthesia

Typically, the authenticity of synaesthesia has been confirmed behaviourally

using tests of consistency of synaesthetic associations over time. Synaesthetes tend to

show greater consistency in inducer-concurrent pairings (synaesthetes are typically

around 90% consistent) compared with non-synaesthetic subjects asked to freely

associate or use a particular strategy (i.e. memory or imagery), even when tested over

longer time periods (Baron-Cohen, Harrison, Goldstein and Wyke, 1993). This

pattern has been shown to be the case in a number of variants of synaesthesia,

including grapheme-colour (Baron-Cohen et al., 1996), emotion-colour (Ward, 2004)

and lexical-gustatory synaesthesia (Ward and Simner, 2003).

A further paradigm used to confirm the automaticity of the synaesthetic

experience has been the modified ‘synaesthetic stroop’ task in which synaesthetic

inducers are paired with either a congruent or incongruent concurrent. For example,

if a grapheme-colour synaesthete perceives the letter A as red, then this grapheme

would be presented in a colour which is either congruent with synaesthetic experience

(e.g. A) or a colour which is incongruent with synaesthetic experience (e.g. A). The

subject is asked to ignore the synaesthetic colour and name the true colour of the

grapheme. Grapheme-colour synaesthetes tend to be faster in the congruent relative

Chapter 1

13

to incongruent condition (Mills, Boteler, and Oliver, 1999), while non-synaesthetes do

not show this pattern. As with tests of consistency, this pattern of performance has

been found for different subtypes of synaesthesia, including not only grapheme-colour

synaesthesia, but also music-colour (Ward et al., 2006), music-taste (Beeli, Esslen,

and Jäncke, 2005), and mirror-touch (Banissy and Ward, 2007; summarised later in

Figure 1.5) variants.

Notably, individuals who have over-learned colour associations may also

behave similar to synaesthetes on the synaesthetic stroop task. For example, Elias and

colleagues report a single case study in which a non-synaesthetic individual with

reliable digit-colour associations, as a result of years of training using coloured

numerical codes in cross-stitching, performed comparably to synaesthetic subjects on

tests of consistency and stroop interference (synaesthetes differed from the control on

functional magnetic resonance imaging [fMRI] measures of synaesthesia in colour

selective regions but not on behavioural measures; Elias, Saucier, Hardie, and Sart,

2003). This is consistent with the findings of MacLeod and Dunbar (1988) who

trained non-synaesthetic subjects to associate black and white geometric shapes with

colour names over thousands of trials. When participants were later presented with a

stroop task, involving the geometric shapes presented in a congruent or incongruent

colour, the typical stroop interference pattern was observed (MacLeod and Dunbar,

1988). In neither study were subjects experiencing synaesthetic colour interactions,

implying that associative (rather than perceptual) components may be able to account

for some of the patterns of performance shown by colour synaesthetes on synaesthetic

stroop and consistency measures. However, more recent findings suggest that, in

colour synaesthetes, the synaesthetic stroop effect may be a consequence of both

perceptual and associative components (Nikolić, Lichti and Singer, 2007). Using

Chapter 1

14

principles of colour-opponency (Hering, 1868/1964), Nikolic and colleagues varied

incongruent stimuli within the synaesthetic stroop by using colour-opponent (i.e. red

changed to green) or non-opponent colours (i.e. red changed to blue). If synaesthetic

stroop relies upon perceptual processes as well as associative components then one

would expect the colour-opponent condition to produce the most interference – this

pattern was observed (Nikolić et al., 2007).

1.3.2 Psychophysical studies

In addition to measures of stroop and consistency, other psychophysical

measures have been used to investigate how closely synaesthetic perception resembles

veridical sensory perception. Again, much of this work has focussed on the

perceptual reality of synaesthetic colours in grapheme-colour synaesthesia. These

findings indicate that synaesthetic and real colours interact under conditions of

binocular rivalry (Kim, Blake, and Palmeri, 2006); that synaesthetic colours can

induce orientation contingent colour adapting after-effects such as a synaesthetic

‘McCollough Effect’ (Blake, Palmeri, Marois, and Kim, 2004; but see Hong and

Blake, 2008); that synaesthetic and real colours can combine to produce apparent

motion (Kim et al., 2006); and that, in projector synaesthetes, synaesthetic experience

can be modulated by background contrast, implying that synaesthesia relies upon

early contrast-dependent visual mechanisms (Hubbard, Manohar, and Ramachandran,

2006; Witthoft and Winawer, 2006).

1.3.3 Neuroimaging studies

Aside from behavioural and psychophysical tests, functional brain imaging

methods have been used to distinguish between synaesthetic and non-synaesthetic

subjects. These have included positron emission tomography (PET) studies of

Chapter 1

15

word/grapheme-colour synaesthesia triggered by speech (Paulesu et al., 1995); fMRI

studies of grapheme-colour (Aleman, Rutten, Sitskoorn, Dautzenberg, and Ramsey,

2001; Hubbard, Arman, Ramachandran, and Boynton, 2005; Weiss, Zilles, and Fink,

2005; Sperling, Prvulovic, Linden, Singer, and Stirn, 2006; Rich et al., 2006), mirror-

touch (Blakemore, Bristow, Bird, Frith, and Ward, 2005), word-colour (Aleman et al.,

2001; Nunn et al., 2002; Gray, Parslow, Brammer, Chopping, Vythelingum, and

ffytche, 2006), digit-colour (Elias et al., 2003), people-colour (Weiss, Shah, Toni,

Zilles, and Fink, 2001), time-colour (Steven, Hansen, and Blakemore, 2006), time-

space (Steven et al., 2006), sound-vision (Stewart, Mulvenna, Griffiths, and Ward, in

prep), and bidirectional synaesthesia (Cohen Kadosh, Cohen Kadosh, and Henik,

2007). In addition, there have been two diffusion tensor imaging studies (DTI) of

grapheme-colour synaesthesia (Rouw and Scholte, 2007; Jäncke, Beeli, Eulig, and

Hänggi, 2009).

While there is some inconsistency between studies, the majority point to

synaesthetic experience being correlated with activations in brain regions involved in

normal perceptual experience. For example, studies investigating synaesthesia

involving colour tend to report activation of colour area V4 / V8 for synaesthesia

inducing stimuli (e.g. Hubbard et al., 2005; Nunn et al., 2002; Sperling et al., 2006),

although not always (i.e. Paulesu et al., 1995; Weiss et al., 2005; Figure 1.1). The

reasons behind this inconsistency remain unclear, although they may be related to

differences in task demands, statistical power, or qualitative differences between

synaesthetic subjects (Hubbard et al., 2005). Moreover, by correlating performance

on different synaesthetic psychophysical paradigms with fMRI activations, Hubbard

and colleagues (2005) show that synaesthetes who show larger effects on

Chapter 1

16

psychophysical tasks show greater activations in colour selective regions of the visual

cortex (V4).

Figure 1.1 fMRI data from a control and grapheme-colour synaesthetic subject when presented with synaesthesia inducing graphemes. Colour area V4 (as per Wade, Brewer, Rieger, and Wandell, 2002) is shown in purple and the grapheme area (Gr) in blue. The synaesthete shows activation in both V4 and Gr, while the control only shows activation in Gr. Taken from Hubbard and Ramachandran (2005).

Recent research making use of DTI techniques is also consistent with the

notion that inter-individual differences within the synaesthetic population may

contribute to different patterns of brain activation. DTI is a neuroimaging technique

which measures the diffusion of water molecules in the living human brain to enable

analysis of the degree of structural connectivity between brain regions (Basser, 1995).

Using this method, Rouw and Scholte (2007) report that grapheme-colour

synaesthesia is linked with increased structural connectivity (as compared with non-

synaesthetes) in the left superior parietal cortex, right inferior temporal cortex

(adjacent to the fusiform gyrus) and in a bilateral cluster located beneath the central

sulcus. Of these four clusters, greater connectivity in the right inferior temporal

cortex was found to be strongest in ‘projector’ synaesthetes who saw their colours in

Chapter 1

17

the outside world (compared to ‘associator’ synaesthetes who saw their colours in

their mind’s eye).

In addition to shared activations in brain regions involved in normal and

synaesthetic perceptual experience (e.g. V4 / V8 in colour), a number of common

brain activations have been reported across different variants of the condition. Two

brain regions of note are the insula and the intraparietal sulcus (IPS). IPS and insula

activations have been reported in studies of both visual and auditory grapheme-colour

synaesthesia (insula activations - Nunn et al., 2002; Paulesu et al., 1995; IPS

activations - Paulesu et al., 1995; Weiss et al., 2005); synaesthesia involving spatial

number forms (Tang, Ward, and Butterworth, 2008); and studies of sound-vision

synaesthesia (Stewart et al., in prep). Insula activations have also been found in

mirror-touch synaesthesia (Blakemore et al., 2005). Both regions have been

implicated in multi-sensory processing and integration (Bushara, Grafman, and

Hallett, 2001; Hadjikhani and Roland, 1998; Olson, Gatenby, and Gore, 2002),

indicating that they may play a role in integrating synaesthesia inducing materials

with experience.

1.3.4 Transcranial magnetic stimulation (TMS) studies

TMS is a non-invasive technique that uses induced current to depolarize the

cell membrane in the cortex leading to a temporary modulation of neural activity in

the stimulated cortex (Walsh and Rushworth, 1999). This method enables

examination of the necessity of stimulated brain structures for given cognitive

functions. To date, two TMS studies have been conducted to investigate the necessity

of parietal cortex activations in grapheme-colour synaesthesia (Esterman, Verstynen,

Ivry, and Robertson, 2006; Muggleton, Tsakanikos, Walsh, and Ward, 2007).

Esterman et al. investigated the magnitude of synaesthetic interference on a

Chapter 1

18

synaesthetic stroop task in two ‘projector’ synaesthetes following TMS to a parieto-

occipital region in the right hemisphere, to the corresponding region in the left

hemisphere, and to area V1. They found that the magnitude of synaesthetic

interference was reduced following TMS to the right parieto-occipital area, but not for

the other two brain regions (Esterman et al., 2006). Extending upon this, Muggleton

et al. contrasted the effects of TMS over four parietal brain regions (right parieto-

occipital, left parieto-occipital, right parietal and left parietal regions) in five

grapheme-colour synaesthetes (comprised of one ‘projector’ and four ‘associators’).

Consistent with the findings of Esterman et al., these authors also report that the

automaticity of synaesthetic experience (as measured using a synaesthetic stroop task)

was disrupted following stimulation of the right parieto-occipital area only

(Muggleton et al., 2007; Figure 1.2). Therefore both studies suggest that the right

parieto-occipital cortex is necessary for the experience of synaesthesia. In non-

synaesthetes this brain region has been shown to participate in visual feature binding

(Freidman-Hill, Robertson, and Treisman, 1995; Donner, Kettermann, Diesch,

Ostendorf, Villringer, and Brandt, 2002) and one explanation for the selective TMS

disruption observed is that the right parieto-occipital area may act in the spatial

binding of graphemes with synaesthetic colour (Esterman et al., 2006; Muggleton et

al., 2007). If so, this suggests that, in grapheme-colour synaesthesia, synaesthetic

experience acts upon the same cortical pathways that exist in the non-synaesthetic

brain (Cohen Kadosh and Walsh, 2008; Cohen Kadosh, Henik, Catena, Walsh, and

Fuentes, 2009). However, even if synaesthetes use common mechanisms of cross-

modal binding, it remains unclear whether synaesthetic binding follows the same

time-course of processing as veridical cross-modal binding or how the parietal lobe

Chapter 1

19

interacts with processing in other cortical areas (i.e. sensory-selective cortical regions)

to elicit synaesthetic experience.

Figure 1.2 Summary of Muggleton et al. (2007). (a) The location of stimulated right parietal-occipital region (RPO; x = 22, y = -71, z = 27) and right parietal region. (b) Interference between real and synaesthetic colours on synaesthetic stroop task. Performance for individual synaesthetes, divided between projectors and associators, following TMS targeted at the RPO region compared to control condition. Synaesthetes EP and CP were reported by Esterman et al. (2006) and are shown for comparison. Adapted from Muggleton et al. (2007) with permission.

1.3.5 Electrophysiological studies

As with TMS studies, to date there have been few studies utilizing

electrophysiological techniques to investigate the time course neural activity in

synaesthetic experience. Schiltz and colleagues (1999) investigated the

electrophysiological correlates of grapheme-colour synaesthesia (n = 17) and reported

an increased positivity at frontal and central scalp sites of synaesthetes (relative to

controls) occurring around 150 msec after stimulus onset which was sustained until

600 msec. A more recent study by Beeli and colleagues, conducted with grapheme-

colour synaesthetes who only experience colours from spoken letters and words (n =

16), revealed differences (reduced amplitude and / or increased latencies) in the

auditory N1, P2, and N2 deflections (Beeli, Esslen, and Jäncke, 2008). Source

a. b.

Chapter 1

20

localization implicated intracerebral sources of these components to lay in inferior

temporal and orbitofrontal brain regions (although few electrode sites were available).

These authors interpret their finding as evidence for increased cortical wiring in

synaesthetes (c.f. Ramachandran and Hubbard, 2001; Bargary and Mitchell, 2008),

but may also be consistent with accounts of synaesthesia which posit differences in

local mechanisms of cortical inhibition (Cohen Kadosh and Walsh, 2008).

In addition to this, two single case studies and one group study have

investigated auditory-visual synaesthesia. In a single case study of acquired auditory-

visual synaesthesia, Rao and colleagues report that synaesthesia inducing sounds

resulted in a modulation of the auditory evoked N1 deflection (Rao, Nobre,

Alexanader and Cowey, 2007). Rizzo and Eslinger (1989) investigated the

electrophysiological correlates of a single case of developmental auditory-visual

synaesthesia, but restricted analysis to three electrode sites (O1/2, or Oz). No

abnormal potentials were found at these three sites, but this does not rule out the

possibility that abnormal potentials may occur at alternative electrode sites. A more

recent group study of tone-colour synaesthesia (n = 10; Goller, Otten, and Ward,

2009) revealed early onset (around 100msec after stimulus onset) differences in

deflections of the auditory evoked potential (auditory N1, N2, and P2). No posterior

difference was observed, implying that synaesthetic experience may be generated

locally (potentially through mechanisms of local differences in cortical inhibition; c.f.

Cohen Kadosh and Walsh, 2008).

1.3.6 Neurocognitive models of synaesthesia

While much research has determined the authenticity of synaesthesia, the

neurocognitive mechanisms which underpin synaesthesia are a subject of uncertainty.

A current area of dispute in the synaesthesia literature is whether synaesthetic

Chapter 1

21

experience is due to additional structural connectivity (i.e. structural differences),

malfunctions in cortical inhibition (i.e. functional but not structural differences) or a

combination of both (Bargary and Mitchell, 2008; Cohen Kadosh and Henik, 2007;

Cohen Kadosh and Walsh, 2008; Grossenbacher and Lovelace, 2001; Hubbard and

Ramachandran, 2005; Rouw and Scholte, 2007; Smilek, Dixon, Cudahy, and Merikle,

2001).

Supporting evidence for structural connectivity accounts is provided by a DTI

study which reports greater white matter coherence in grapheme-colour synaesthetes

compared to non-synaesthetic control subjects - grapheme-colour synaesthetes show

increased structural connectivity in inferior-temporal, parietal and frontal brain

regions when compared to non-synaesthetes (Rouw and Scholte, 2007). Some

authors have interpreted these findings to be consistent with accounts of synaesthesia

which argue in favour of aberrant connectivity between adjacent cortical regions

(Ramachandran and Hubbard, 2001; Hubbard, 2007). For example, given that the

brain regions involved in the visual recognition of graphemes (i.e. the putative visual

word form area; Cohen and Dehaene, 2004) lie adjacent to brain areas involved in

colour perception (i.e. human V4 - Wade et al., 2002), it has been suggested that

grapheme-colour synaesthesia may arise from direct cross-activation between these

regions as a result of either increased connectivity between adjacent brain regions or

reduced inhibition between adjacent regions. This local cross-activation account has

also been extended to explain sequence-space synaesthesia (i.e. number forms), in

terms of cross-activation between adjacent parietal regions (Ramachandran and

Hubbard, 2001), and may also be important for other variants of synaesthesia (e.g.

lexical-gustatory synaesthesia; Ward, Simner and Auyeung 2005). However, it is of

note that the generality of enhanced structural connectivity in grapheme-colour

Chapter 1

22

synaesthesia is debatable (e.g. see Jäncke et al., 2009) and the extent to which these

differences extend to other variants of synaesthesia (e.g. mirror-touch synaesthesia) or

play a causal role in generating synaesthetic experience remains unknown.

In contrast to aberrant cortical connectivity accounts, other authors have

argued in favour of feedback accounts of synaesthesia which explain the condition in

terms of malfunctions in cortical inhibition, either within (Cohen Kadosh and Henik,

2007; Cohen Kadosh and Walsh, 2008) or between brain regions (e.g. from a

multisensory cortical nexus; Grossenbacher and Lovelace, 2001). According to this

view, synaesthesia is mediated by the same cortical pathways that exist in the non-

synaesthetes’ brain (i.e. aberrant connectivity is not necessary to induce synaesthesia),

but unmasking of these pathways due to alterations in cortical inhibition results in

synaesthetic experience (Grossenbacher and Lovelace, 2001; Cohen Kadosh and

Walsh, 2006; Cohen Kadosh and Henik, 2007; Cohen Kadosh and Walsh, 2008;

Cohen Kadosh, Henik, Catena, Walsh, and Fuetnes, 2009). Evidence that TMS

disruption of the parietal lobe impairs synaesthetic stroop performance (Esterman et

al., 2006; Muggleton et al., 2007); that synaesthetic-like experiences can be induced

following hallucinogenic drugs (i.e. in the absence of altered cortical connectivity;

Aghajanian and Marek, 1999); and that colour synaesthesia can be induced in non-

synaesthetes (individuals without aberrant connectivity) using post-hypnotic

suggestion (Cohen Kadosh et al., 2009) are consistent with this.

1.4 Synaesthesia involving touch

Synaesthesia involving touch has been less well documented than other

variants of synaesthesia. Despite this, there are some reports of both developmental

and acquired synaesthesia involving either a tactile inducer or concurrent. These

cases are discussed below.

Chapter 1

23

1.4.1 Synaesthesia involving tactile inducers

To date much research on synaesthesia involving tactile inducers has centred

on cases of touch-vision synaesthesia in which touch results in visualised photisms.

For example, Armel and Ramachandran (2001) report a case of acquired touch-vision

synaesthesia shown by a patient who suffered blindness due to retinitis pigmentosa.

One year after becoming completely blind the patient began to experience

synaesthetic visual photisms from haptic stimuli. Such sensations were projected onto

the spatial location of the body part touched irrespective of the location of the body

part in space (e.g. a touch to the right hand in left space would elicit photisms in left

space). Detailed investigations indicated that the intensity of tactile stimulation

required to induce synaesthesia was lower when body parts were presented in front of

the patient relative to behind the head (i.e. moving the hands from in front of the head

to behind the head), suggesting that despite the patient being blind a preference was

shown for when the inducer was “in view”. This may be indicative of a body-based

spatial reference that incorporates information about gaze and head orientation. Such

findings are consistent with evidence from non-synaesthetes on normal multi-sensory

interactions between touch and vision indicating that the spatial reference frame

which processes current hand position is modulated by gaze direction (Armel and

Ramachandran, 2001).

In addition to this, cases of blind synaesthetes for whom Braille reading elicits

a visual concurrent have been reported (Wheeler and Cutsforth, 1921; Steven and

Blakemore, 2004). In the latter case, synaesthete JF, who suffered from retinitis

pigmentosa leading to blindness, consistently experienced coloured visual photisms

both when reading Braille and when thinking about Braille characters (Steven and

Blakemore, 2004). Similar geometrical arrangements of Braille dots evoked similar

Chapter 1

24

colours, but photisms were not elicited when touching other textures or objects.

Notably, it has been reported that J.F experienced visual synaesthesia from childhood

(i.e. since before losing his sight; Steven, Hansen, and Blakemore, 2006) so it may be

the case that his synaesthesia reflects a different manifestation of grapheme-colour

synaesthesia in which graphemes are processed haptically rather than visually (Ward,

Banissy, and Jonas, 2008).

There have been relatively few documented cases of developmental touch-

vision synaesthesia. While Day (2005) reports that 4% of synaesthetes report

‘coloured-touch’ these figures are based on self reported cases only (a failure to

objectively confirm these self reports with tests of genuiness may mean that this 4%

claim includes false positive cases; c.f. Simner et al., 2006) and no information is

given regarding the nature of these cases (i.e. developmental or acquired cases).

Recently, two cases of developmental touch-vision synaesthesia in which touch

triggers visual sensations of colour (TV and EB) have been investigated more

systematically (Ward et al., 2008). Each case indicated important distinctions

between the spatial representations which underpin synaesthetic experience.

Moreover, for synaesthete TV coloured photisms were projected onto the spatial

location of the body part touched, whereas for EB photisms were perceived in her

mind’s eye. This distinction appears similar to the projector / associator distinction in

grapheme-colour synaesthesia outlined above (Dixon et al., 2004).

It is of note that while synaesthetes TV and EB appeared consistent on a test

of consistency for their synaesthesia, involving 40 different tactile stimuli across two

testing sessions; they were not shown to be significantly more consistent than control

subjects. This is likely to be related to elevated levels of control consistency (c.f.

Kusnir, MSc Thesis, University of London, 2008) indicating that the touch-vision

Chapter 1

25

synaesthesia reported by TV and EB may rely upon similar multi-sensory principles

which underpin non-synaesthetic touch-vision interactions (Ward et al., 2008).

Moreover, cross-modal correspondences between roughness and luminance (rougher

textures associated to darker colours) and pressure and luminance (higher pressure

associated with darker colours) were found for both synaesthetes and control subjects.

Consistent with this, previous reports of touch-vision synaesthesia have indicated a

relationship between pressure and luminance, in which the synaesthete experienced

dark coloured photisms to hard objects (i.e. higher pressure) and lightly coloured

photisms to soft objects (i.e. lower pressure) (Smith, 1905). Thus, it may be the case

that developmental touch-vision synaesthesia relies upon similar mechanisms of cross

modal transfer as observed in non-synaesthetic multi-sensory interactions between

touch and vision. This would be consistent with findings indicating that other variants

of synaesthesia appear to act upon the ‘normal’ architecture for cross-modal

interactions (e.g. Ward, Huckstep, and Tsakanikos, 2006; Blakemore et al., 2005;

Sagiv and Ward, 2006).

1.4.2 Synaesthesia involving a tactile concurrent

As with cases of synaesthesia in which touch acts to induce synaesthetic

experience, cases of synaesthesia involving a tactile concurrent are less well

documented than other more common variants of synaesthesia. Despite this, there are

reports of acquired auditory-tactile synaesthesia in which sounds elicit tactile

sensations (Ro et al., 2007) and of both acquired and developmental mirror-touch

synaesthesia in which observed touch results in tactile experiences on the observer’s

own body (Halligan, Hunt, Marshall, and Wade, 1996; Bradshaw and Mattingley,

2001; Blakemore et al. 2005; Banissy and Ward, 2007). There is also evidence that

the presence of synaesthesia for colour is linked to a greater incidence mitempfindung

Chapter 1

26

(the referral of a tactile sensation away from the stimulation site; Burrack, Knoch, and

Brugger, 2006).

Synaesthetic interactions involving hearing and touch have rarely been

documented, however recently Ro et al. (2007) report a single case of acquired

auditory-tactile synaesthesia in a female patient following a discrete neurological

lesion to the right ventrolateral thalamus. The synaesthesia was first reported 18

months post lesion when the patient began to feel tactile sensations in response to

sounds. The synaesthetic somatosensations were typically experienced on the

patient’s left upper part of the body and a test of consistency (across three testing

sessions separated by 35 and 15 days) indicated that they were generally consistent

over time. Magnetic resonance imaging (MRI) and diffusion tensor imaging (DTI)

conducted at approximately 3 years post lesion indicated disorganised fibre bundles in

the right ventrolateral thalamus (lesion site) - at 3 years post onset DTI tracking from

the unaffected left hemisphere showed direct projections to motor / premotor cortices,

whereas fibre bundles in the lesioned hemisphere were disorganised and smaller

compared to the unaffected hemisphere. DTI conducted at 1.3 years post onset (i.e.

before synaesthetic experiences were reported) indicated no white matter differences

between the right and left ventrolateral thalamus. The authors suggest that this

disorganisation in cortico-callosal pathways may account for synaesthetic experiences

(Ro et al., 2007; see Chapter 9).

In addition to cases of acquired auditory-tactile synaesthesia there are a

number of accounts of acquired synaesthesia involving vision-touch interactions.

For example, patient D.N., suffered paralysis and loss of sensation in the left side of

his body following stroke. This resulted in D.N. being unable to feel any tactile

stimulation presented to the left side when the touch was hidden from view, however

Chapter 1

27

if tactile stimulation was made visible then D.N. was able to feel touch to the left side.

Similarly, when observing previous videos of his arm being touched and informed

that this reflected live video feedback D.N. reported being able to feel his arm being

touched despite the fact that the experimenter was not actually touching him. In this

sense, observed bodily touch attributed to the patient lead to tactile sensations,

indicating that in some conditions vision alone can be sufficient to elicit tactile

stimulation (Halligan et al., 1996). Such findings appear consistent with research in

the non-synaesthetic population which indicates that observing one’s own body can

lead to enhancements in one’s own tactile sensitivity (Taylor-Clarke, Kennett, and

Haggard, 2004) and with evidence provided by Rorden and colleagues (1999) of a

patient whose tactile detection increased when observing a flash of light to a rubber

hand seen in the same orientation and directly above the patient’s concealed hand (i.e.

when the rubber hand was attributed to the participant’s own body). Taken

collectively these findings highlight the important role of vision, and more so of

observing one’s own body, on haptic perception.

In addition to cases of acquired vision-touch synaesthesia involving one’s own

body there is also one case report involving an interaction between observed and

actual pain (“mirror pain”). This anecdotal report, given to clinicians posthumously

by the patient’s wife, describes a man who experienced observed pain to others as

actual pain to himself (Bradshaw and Mattingley, 2001). Here the inducer is observed

touch to another’s body rather than to one’s own body as described above. The

patient was known to have suffered widespread cancer, but as this case was reported

post-mortem no information about the neural circuitry involved was available. More

recently however, evidence for the interpersonal sharing of observed pain has been

provided (Singer, Seymour, O’Doherty, Kaube, Dolan, and Frith, 2004; Morrison,

Chapter 1

28

Lloyd, di Pellegrino, and Roberts, 2004; Avenanti, Beuti, Galati, and Aglitoi, 2005).

For example, Avenanti and colleagues (2005) report that observing pain to another

person results in significant reductions in motor evoked potentials (MEPs). The

modulation of MEP amplitude correlated with subjective ratings of the sensory

aspects of pain attributed by the observer to the actor and was somatotopically

organised such that the reduced amplitude was specific to the muscles observed in a

painful event. These authors suggest that the findings provide evidence for a mirror-

pain resonance system in which observed pain is matched to the observer’s own

sensorimotor representation of pain. Such interpretation builds upon the findings of

mirror neurons within the monkey premotor cortex and inferior parietal lobule, which

respond both when a monkey performs an action and when the monkey watches

another person perform a similar action (Gallese, Fadiga, Fogassi, and Rizzolatti,

1996; Rizzolatti and Craighero, 2004) and evidence for similar mirror systems in the

human brain for not only action (Buccino et al., 2001), but also touch (Keysers,

Wicker, Gazzola, Anton, Fogassi, and Gallese, 2004; Blakemore et al., 2005; Ebisch,

Perucci, Ferretti, Del Gratta, Luca Romani, and Gallese, 2008), pain (Singer et al.,

2004; Aventani et al., 2005), disgust (Wicker, Keysers¸ Plailly, Royet, Gallese, and

Rizzolatti, 2003) and other emotions (Carr, Iacoboni, Dubeau, Mazziotta, and Lenzi,

2003).

Similar to the case of acquired “mirror pain” described above; developmental

cases of vision-touch or “mirror-touch” synaesthesia have also been documented

(Blakemore et al., 2005; Banissy and Ward, 2007). First reported in a single case

fMRI study (Blakemore et al., 2005), mirror-touch synaesthesia refers to cases of

synaesthesia in which observing touch to another person leads to tactile sensations on

the equivalent part of the synaesthete’s own body. In the original study by Blakemore

Chapter 1

29

and colleagues (2005) the case of synaesthete C was described. C reports

experiencing touch on her own body when observing another person being touched,

but not when observing inanimate objects being touched. These experiences have

been described as being automatic, in so far as they occur whenever she observes

another person being touched, and to have occurred throughout her lifetime. Her

experiences mirror observed touch to another person, such that observing touch to

another person’s left facial cheek leads to a sensation of touch on her own right facial

cheek (i.e. as if looking at a mirror reflection of herself). Using fMRI Blakemore and

colleagues investigated the neural systems underlying C’s synaesthetic experience by

contrasting brain activity when watching videos of humans relative to objects being

touched (the latter did not elicit synaesthesia) in the synaesthete and in 12 non-

synaesthetic control subjects. In controls a network of brain regions were activated

during the observation of touch to a human relative to an object, including primary

and secondary somatosensory cortex, premotor regions and the superior temporal

sulcus. Similar brain regions were also activated during actual touch, indicating that

observing touch to another person activates a similar neural circuit as actual tactile

experience – a “mirror touch” system. A comparison between synaesthete C and non-

synaesthetic subjects indicated that the synaesthete showed hyper activity within a

number of regions within this network (including primary somatosensory cortex) and

additional activity in the anterior insula (Figure 1.3). Thus suggesting that mirror-

touch synaesthesia is a consequence of increased neural activity in the same mirror-

touch network that is evoked in non-synaesthetic controls when observing touch to

another person (Blakemore et al., 2005).

Chapter 1

30

Figure 1.3 Activations resulting from the interaction between mirror-touch synaesthete ‘C’ and non-synaesthetic control participants. The subtraction shown indicates the brain regions more active in synaesthete ‘C’ compared to non-synaesthete controls when observing touch to a human relative to an object. The primary (SI) and secondary somatosensory cortex (SII), bilateral anterior insular and the left premotor cortex were significantly more active in C than in the control group (Blakemore et al., 2005).

More recently, Banissy and Ward (2007) report a behavioural study of ten

developmental mirror-touch synaesthetes, including C. Notably, while all

synaesthetes report similar experiences (i.e. tactile sensations when observing touch to

another person) some important individual differences were found between them. It

appears that mirror-touch synaesthetes can be divided into two subgroups based upon

the spatial mapping between observed and felt (synaesthetic) touch (Figure 1.4).

Some synaesthetes report that an observed touch to the left cheek is felt on their right

cheek (as if the other person is a mirror reflection of oneself – hereafter referred to as

the ‘specular’ subtype) whereas others report synaesthetic touch on their left cheek

when observing touch to another person’s left cheek (as if self and other share the

same anatomical body space – hereafter referred to as the ‘anatomical’ subtype). The

automaticity of these experiences was confirmed by the development of a visuo-

tactile synaesthetic stroop experiment. In the task synaesthetes and matched non-

synaesthetic controls were asked to detect a site touched on their own body (either

SI

SI

Premotor Cortex STS

Anterior Insula

SII SII

Anterior Insula

Chapter 1

31

facial cheeks or hands) while observing touch to another person’s cheek/hands or to a

corresponding object. Participants were asked to report the site of actual touch (left,

right, or no touch) and to ignore observed touch. For synaesthetes, but not controls,

observed touch to humans elicited a tactile sensation whose location was either in the

same position as actual touch (congruent condition – as determined by synaesthetic

self reports) or in a different spatial location (incongruent condition). Synaesthetes,

but not control participants, were faster at detecting the location of actual touch during

the congruent condition relative to the incongruent condition (Figure 1.5b). Further,

synaesthetes produced a higher percentage of errors consistent with their synaesthesia

(hereafter referred to as mirror-touch errors; i.e. reporting touch on trials involving no

actual touch) compared to other error types and to control participants (Figure 1.5c).

Figure 1.4 Specular and anatomical spatial mappings reported by mirror-touch synaesthetes (c.f. Banissy and Ward, 2007). Under a specular frame of reference, mirror-touch synaesthetes report synaesthetic touch as if looking in a mirror. Under an anatomical frame of reference synaesthetic experience is as if self and other share the same anatomical body space.

Introduction 32

300

350

400

450

500

550

600

650

700

750

800

850

Mirror touch Controls

Reaction time (ms)

Congruent

Incongruent

b.

*

0

2

4

6

8

10

12

14

16

18

20

22

24

26

28

30

Percentage of errors

Mirror-touch errors

Other errors

c.

*

*

Mirror-Touch Synaesthetes Non-synaesthetes

Figure 1.5 Summary of Banissy and Ward (2007). (a) Task Protocol. Participants were required to report the site upon which they were actually touched (i.e. left cheek, right cheek, both cheeks or no touch) while ignoring observed touch (and the synaesthetic touch induced from it). Note that although the example given is for a specular mirror-touch synaesthete, both subtypes were tested and congruency was determined according to each synaesthetes’ self-reports. (b) Mirror-touch synaesthetes, but not controls, were significantly faster at detecting the site of real touch in the congruent relative to incongruent condition. (c) Mirror-touch synaesthetes also produced significantly more mirror-touch errors than controls (errors consistent with their synaesthesia), but not other error types. * = p <.05.

Chapter 1

33

1.5 Synaesthesia and models of typical cognition

The preceding sections reviewed evidence for the authenticity of synaesthesia.

While this is now well established, there is growing interest in using the condition to

inform us about non-synaesthetic perceptual and cognitive processing. Following the

logic of cognitive neuropsychology, the positive symptoms related to synaesthesia

may be able to constrain theories on multisensory interactions and inform about the

relationship between multisensory processing and other aspects of cognition (Ward

and Mattingley, 2006; Cohen Kadosh and Henik. 2007).

So far, a number of examples have been cited whereby synaesthetic

interactions have been shown to rely upon similar neurocognitive mechanisms as

those observed in non-synaesthetes and therefore may inform us about general

principles of multisensory interactions (e.g. feature binding in grapheme-colour

synaesthesia; cross-modal interactions in touch-colour synaesthesia; heightened

visual-tactile interactions in mirror-touch synaesthesia). Non-random associations,

which are similar to those found in non-synaesthetic subjects, have also been found

between pitch and lightness in tone-colour synaesthetes (individuals who experience

colour sensations in response to tones) – both synaesthetes and non-synaesthetes show

a tendency to associate low pitches with dark colours and high pitches with light

colours, although only synaesthetes experience these colours consciously (Ward et al.,

2007; also see Parise and Spence, 2009). Evidence of non-random associations in

other variants have also been documented, including number and lightness in digit-

colour synaesthesia (Cohen Kadosh and Walsh, 2008); word form properties and

colour associations in linguistic-colour synaesthesia (Barnett, Feeney, Gormley, and

Newell, 2009); and phonology and tastes in lexical-gustatory synaesthesia (Ward and

Simner, 2003).

Chapter 1

34

Further, in the case of feature binding in synaesthesia, it has been suggested

that an integration of synaesthesia and patient-based research may contribute to our

understanding of the binding problem – how two independently processed features are

combined to be perceived as a unified experience (Robertson, 2003). In the

numerical domain, digit-colour synaesthesia and spatial number-form synaesthesia

have been successfully used as models to make inferences about the mental

representation of two-digit numbers (Seron, Pesenti, Nöel, Deloche, and Cornet, 1992;

Sagiv et al., 2006), the spatial representation of number (Sagiv et al., 2006), and

whether numerical representations are compressed or linear (Cohen Kadosh et al.,

2007).

One aim of this thesis is to use mirror-touch synaesthesia as a model to make

inferences about the role of visual-tactile interactions in cognition and perception. As

noted previously, mirror-touch synaesthesia is thought to arise because of hyper-

activation of the same cortical network (the mirror-touch system) which is active in

non-synaesthetes when observing touch to others (Blakemore et al., 2005). In recent

years there has been much interest in the role brain systems with mirror properties

(e.g. the mirror-touch system) may play in social cognition (Gallese, Keysers, and

Rizzolatti, G, 2004; Gallese, 2006; Keysers and Gazzola, 2006). Moreover, it has

been suggested that brain systems with mirror properties (i.e. common brain regions

in the experience and observation of a particular sensation) may act as a

neurophysiological candidate to facilitate sensorimotor simulation of another’s

experience and thereby promote an understanding of another’s emotions / experience

(Gallese, Keysers, and Rizzolatti, G, 2004; Gallese, 2006; Keysers and Gazzola,

2006). Given that mirror-touch synaesthesia has been linked to neural mechanisms

common to us all when observing touch to another person (i.e. hyper activity within

Chapter 1

35

the tactile mirror system), this variant of synaesthesia highlights one means in which

synaesthesia may be used to investigate vision-touch interactions more generally –

namely what is the impact of heightened sensorimotor simulation on affective

processing. Moreover, mirror-touch synaesthesia is currently one of the only forms of

synaesthesia which depends upon interpersonal interaction and therefore offers a

unqiue opportunity to assess mechanisms of social perception.

1.6 Aims of thesis

This thesis has two primary aims. The first is to investigate cases of mirror-

touch synaesthesia and to document neurocognitive and perceptual profiles associated

with the condition. This includes investigations into the prevalence, characteristics,

and perceptual processing of mirror-touch synaesthesia. The second is to investigate

the function of sensorimotor simulation mechanisms (thought to underpin mirror-

touch synaesthesia) in cognition. This aspect of the thesis aims to evaluate the

importance of somatosensory resources for social cognition and examine the

hypothesis that sensorimotor simulation is critical for understanding the emotions and

thoughts of others (Adolphs, 2002; Adolphs, 2003; Gallese, Keysers, and Rizzolatti,

G, 2004; Gallese, 2006; Gallese and Goldman, 1998; Keysers and Gazzola, 2006;

Oberman and Ramachandran, 2007). Studies involving non-synaesthetic individuals

and studies using synaesthetic participants to inform us about the role of sensorimotor

simulation in affective processing shall be presented.

Chapter 2

36

CHAPTER 2: PREVALENCE AND CHARACHTERISTICS OF

MIRROR-TOUCH SYNAESTHESIA

In so-called ‘mirror-touch synaesthesia’, observing touch to another person induces a

subjective tactile sensation on the synaesthete’s own body. It has been suggested that

this type of synaesthesia depends on increased activity in neural systems activated

when observing touch to others. This is the first study on the prevalence of this variant

of synaesthesia. The findings indicate that this type of synaesthesia is just as

common, if not more common than some of the more frequently studied varieties of

synaesthesia such as grapheme-colour synaesthesia. Additionally, behavioural

correlates associated with the condition are examined further. In a second

experiment, it is shown that synaesthetic experiences are not related to somatotopic

cueing - a flash of light on an observed body part does not elicit the behavioural or

subjective characteristics of synaesthesia. Finally, a neurocognitive model to account

for these characteristics is proposed and the implications of the findings are discussed

in relation to general theories of synaesthesia.

2.1 Introduction

As noted in chapter 1, the term synaesthesia is used to describe a condition in

which one property of a stimulus (the inducer) results in conscious experiences of an

additional attribute (the concurrent). This inducer-concurrent relationship can occur

either within or between modalities. For example, in grapheme-colour synaesthesia a

visually or auditorily presented grapheme can result in synaesthetic experiences of

colour (Ramachandran and Hubbbard, 2001; Cohen Kadosh and Henik, 2007; Rich

and Mattingley, 2002), whereas in lexical-gustatory synaesthesia written or heard

words trigger a subjective sensation of taste (Ward and Simner, 2003).

Early research on the prevalence of synaesthesia indicated that the condition

may have a minimum prevalence rate of 1 in 2000 with a female-to-male ratio of 6:1

(Baron-Cohen, Burt, Smith-Laittan, Harrison, and Bolton, 1996; Rich, Bradshaw, and

Mattingley, 2005). These studies assessed the prevalence of the condition based upon

the number of respondents to newspaper advertisements who pass an objective

measure of synaesthesia (relative to newspaper circulation figures). This method of

assessment does not permit inferences about non-responders and may also lead to an

Chapter 2

37

over inflated female to male ratio. More recent studies, which overcome these

difficulties by screening a large population and supplementing this with the use of

objective measures of different variants of synaesthesia, suggest a higher prevalence

rate of 4% and a female to male ratio of 1:1 (Simner et al., 2006; Ward and Simner,

2005). These studies indicate that the most common forms of the condition include

day-colour synaesthesia (estimated to have a prevalence of 2.8%; Simner et al., 2006)

and grapheme-colour synaesthesia (estimated to have a prevalence of 2%; Simner et

al., 2006).

Since these studies, a new variant of synaesthesia has been documented in

which observing touch to another person induces a tactile sensation on the

synaesthete’s own body (mirror-touch synaesthesia). A single case fMRI study

suggests that this variant of synaesthesia is a consequence of increased neural activity

in a network of brain regions which are also activated in non-synaesthetic control

subjects when observing touch to another person (Blakemore, Bristow, Bird, Frith,

and Ward, 2005). In that study, the authors contrasted brain activity in a single

mirror-touch synaesthete with twelve non-synaesthetic control subjects while

observing humans relative to objects being touched. This indicated that while similar

brain regions were active in the observed touch condition as when participants were

touched (a mirror-touch system present in non-synaesthetes), the synaesthete showed

increased activity within bilateral primary somatosensory cortex (SI), secondary

somatosensory cortex (SII), left premotor cortex and additional activity in the anterior

insula relative to non-synaesthetes. In view of this, it was argued that mirror-touch

synaesthesia reflects hyper-activation of normal (i.e. non-synaesthetic) visual-tactile

interactions in the mirror-touch network (i.e. SI, SII, premotor cortex). Notably, the

general role of SI activations in the mirror-touch system in non-synaesthetes remains

Chapter 2

38

to be clarified, with some authors reporting SI activity when non-synaesthetes observe

touch to another’s face (Blakemore et al., 2005) or arm (McCabe, Rolls, Bilderbeck,

and McGlone, 2008), others reporting SII, but not SI, activation following observed

touch to the legs (Keysers, Wicker, Gazzola, Anton, Fogassi, and Gallese, 2004), and

others reporting SI activity when non-synaesthetes observe intentional but not

unintentional touch (Ebisch, Perrucci, Ferretti, Del Gratta, Romani, and Gallese,

2008).

Extending the single case report, a group study of ten mirror-touch

synaesthetes showed that individuals with mirror-touch synaesthesia can be divided

into two subtypes based upon the spatial mapping between observed and

synaesthetically induced touch. Some synaesthetes report a spatial mapping as if

looking in a mirror (i.e. observed touch to another person’s left cheek induces

synaesthetic touch on their right cheek - specular subtype), while others report a

spatial mapping as if self and other share the same anatomical body space (i.e.

experiencing synaesthetic touch on their left cheek when observing touch to another

person’s left cheek – anatomical subtype; (Banissy and Ward, 2007).

Authenticity and characteristics of synaesthesia

When considering the prevalence of mirror-touch synaesthesia it is important

to note what constitutes synaesthesia in general and the methods used to confirm the

authenticity of the condition. Synaesthesia is typically considered as having three

defining features; 1) experiences are conscious perceptual or percept-like experiences;

2) experiences are induced by an attribute not typically associated with that conscious

experience; 3) these experiences occur automatically (Ward and Mattingley, 2006).

In line with this, mirror-touch synaesthesia requires the conscious experience of a

tactile stimulus which occurs automatically following the observation of touch to

Chapter 2

39

another person (or possibly an object; see Banissy and Ward, 2007). There are

several ways to determine the validity of mirror touch synaesthetes, for example, with

regards to automaticity, Banissy and Ward (2007) developed a visuo-tactile congruity

experiment to explore this aspect of synaesthesia (for description see Chapter 1; also

see Blakemore et al., 2005 methods to assess validity mirror-touch synaesthesia).

Synaesthesia has a number of other important characteristics that also appear

to be found in the mirror-touch variety. Synaesthetic experiences tend to be

consistent over time (e.g. if ‘A’ is red at time 1 then it will be at time 2 several weeks

or months later; Baron-Cohen, Wyke, and Binnie, 1987). Mirror-touch synaesthetes

report their experiences to be enduring and an individual’s spatial sub-type (i.e.

whether they belong to the specular or anatomical category) is consistent both across

time and across different body parts. Further, whilst it was once believed that

synaesthetic experiences reflect random but consistent associations this view is no

longer widely held. For example, non-random associations have been found between;

pitch and lightness (Ward, Huckstep, and Tsakanikos, 2006); number and lightness

(Cohen Kadosh, Henik, and Walsh, 2007); grapheme frequencies and colour (Simner

et al., 2005); and phonology and tastes (Ward and Simner, 2003). More overt

semantic links are also found: it is not uncommon for the word “sausage” to taste of

sausage (and similarly for other food names; Ward, Simner and Auyeung 2005) or for

the word “red” to be coloured red (and similarly for other colour names; Gray et al.,

2002; Rich et al., 2005). The mappings in mirror-touch synaesthesia are non-arbitrary

in that somatotopy is generally preserved between the observed and felt touch.

Here two studies investigating the prevalence and the characteristics of mirror-

touch synaesthesia are presented. In Experiment 1, the prevalence of mirror-touch

synaesthesia is investigated by screening a large population and confirming self

Chapter 2

40

reports using a behavioural paradigm designed to test for the authenticity of the

condition. Then potential factors which may contribute to the behavioural correlates

observed are addressed. Experiment 2 examines the nature of the synaesthetic inducer

and considers the role of somatotopic cueing on synaesthetic experience. Finally, the

factors which may underpin synaesthetic experience are discussed and a

neurocognitive model of mirror-touch synaesthesia is outlined.

2.2 Experiment 1: Prevalence of mirror-touch synaesthesia

This study investigates the prevalence of mirror-touch synaesthesia and

compares new cases with previously studied cases of mirror-touch synaesthesia

(Banissy and Ward, 2007) to ascertain the main cognitive characteristics of the

condition.

Method

All participants (n = 567) were recruited from the University College London

and University of Sussex undergraduate communities. Each participant was given a

written and verbal description of synaesthesia including examples of what did and did

not constitute synaesthesia. Participants were then administered a questionnaire

asking about different variants of synaesthesia with one question specifically related

to mirror-touch synaesthesia (Appendix 1). Participants were asked to indicate on a

five point scale the extent to which they agreed with the question “Do you experience

touch sensations on your own body when you see them on another person’s body?”

Following initial screening, all participants who gave positive responses to the above

question (n = 61; approximately 10.8% of all subjects) were contacted and

interviewed about their experiences. This included them being shown a series of

online videos showing another person, object, or cartoon face being touched.

Chapter 2

41

Participants were asked to indicate the location (if any) in which they experienced a

tactile stimulus and the type of experience. Typical responses of potential mirror-

touch synaesthetes (n = 14; approximately 2.5% of all subjects) included reports that

observing touch elicits a tingling somatic sensation in the corresponding location on

their own body, and that a more intense and qualitatively different sensation is felt for

painful stimuli (i.e. videos of a pin pricking a hand rather than observed touch to the

hand).

In an attempt to investigate reports of mirror-touch synaesthesia the

performance of each potential synaesthete was compared to ten age and gender

matched non-synaesthetic control subjects on the paradigm developed by Banissy and

Ward (2007). In the task, participants were required to detect a site touched on their

own face (left, right, both or none) while observing touch to another person’s face or

to a corresponding object (a lamp). For synaesthetes, but not for controls, observed

touch elicited a synaesthetic sensation in a congruent or incongruent location as actual

touch (Figure 2.1). The tactile stimuli were administered via two miniature solenoid

tappers attached to the face with a Velcro strap. Each tapper was controlled using a

Dual Solenoid Tapper Controller (MSTC3-2, M and E Solve) and the intensity of taps

was filtrated to account for individual sensitivity of the participant (as in Banissy and

Ward, 2007). The visual stimuli were presented on a 17” CRT monitor with a refresh

rate of 100Hz, were sized to fit the screen, and consisted of two presentations of 100

ms each followed by a third stimulus which remained on the screen until the

participant responded. The first two stimuli showed the approach of the hand towards

the face and the third showed contact with the face. After a 10 ms presentation of the

final slide participants received a tap to either their left, right or both cheeks. The

location of the felt touch (left, right, both or none) was indicated with a button press

Chapter 2

42

and the need for both speed and accuracy was emphasised. Following this, there was

a gap of 1500 ms with a fixation cross before the start of the next trial. A train of

white noise was presented via headphones for the duration of each trial in order to

prevent participants from using auditory cues (i.e. the sound of the taps) to determine

the location of actual touch (c.f. Banissy and Ward, 2007 for more details on task

methodology) .

A total of 80 congruent trials, 80 incongruent trials and 80 trials involving no

actual touch were completed. For each potential synaesthete, congruency was

determined according to self reports when observing videos showing another person

being touched. Within each condition, 60 trials involved observed touch to either a

female or male actor, with the remaining 20 trials involving observed touch to a

corresponding object. The order of trials was randomised over 3 blocks of 80 trials

(preceded by 5 practice trials). Reaction times and error rates were measured. Based

upon previous findings synaesthetes were expected to be faster at identifying a site

touched in the congruent compared to incongruent condition and / or to show a higher

proportion of mirror-touch errors compared to non-synaesthetic controls. The control

data were scored according to the reported sub-type of the corresponding mirror-touch

synaesthete (i.e. anatomical versus specular congruency).

Chapter 2

43

Figure 2.1 (a) Summary of the task used to confirm potential cases of mirror-touch synaesthesia in experiment 1. Participants were asked to detect the site of real touch while observing another person being touched. For mirror-touch synaesthetes observed touch elicited a tactile sensation which could either be in a congruent or incongruent location as the site real touch. (b) Example of congruent and incongruent trials including error types for a specular mirror-touch synaesthete. On a congruent trial, real touch was applied to the same side of the face as synaesthetic experience. On an incongruent trial real touch was applied to the side of the face which was opposite to synaesthetic experience. Participants were asked to report the location of real touch and to ignore synaesthetic touch. ‘Mirror-touch’ errors could be produced on incongruent trials if the subject was to report real touch to both cheeks (despite real touch being applied to one cheek only) or if the subject was to report synaesthetic rather than real touch. All other error types were classified as ‘Other’ error types.

Chapter 2

44

Results and Discussion

Behavioural performance of each potential synaesthete was compared to an

age and gender matched non-synaesthetic control group using Crawford’s modified t-

test (Crawford and Garthwaite, 2002). Reaction time performance (filtered prior to

analysis, ± 3 s.d. and all errors removed) and the percentage of error types on human

and object trials were compared separately (Table 2.1). For reaction time

performance, the size of congruency effect (incongruent minus congruent reaction

time) was used as an index of synaesthetic experience. For errors, the percentage of

mirror-touch errors (errors consistent with synaesthetic experience) and other error

types were compared. Subjects who showed either significantly larger reaction time

differences or significantly more mirror-touch errors relative to controls were counted

as synaesthetes. Using this method nine cases (seven female)1 of mirror-touch

synaesthesia were confirmed on reaction time performance, the percentage of mirror-

touch errors produced, or both (Table 2.1). This indicates a prevalence rate of 1.6%.

In comparison to previous prevalence estimates of other types of synaesthesia this

places mirror-touch synaesthesia as one of the most common forms of synaesthesia

along with grapheme-colour synaesthesia (1.4% prevalence) and day-colour

synaesthesia (2.8% prevalence; Simner et al., 2006).

Comparison of the prevalence group with previously reported cases

In order to ensure that these cases were consistent with previously reported

cases of mirror-touch synaesthesia, these characteristics of synaesthetic experience

were considered further by contrasting synaesthetes recruited through the prevalence

study (n = 9) with mirror-touch synaesthetes recruited via self referral including some

1 Participants came from courses with a higher female-male ratio and gender information for non-respondants was not available, so no empirical claims about female-male ratio of mirror-touch synaesthetes are made.

Chapter 2

45

previously reported cases (n = 12). Reaction time (Congruency x Group) and the

percentage of error types (Error Type x Group) were compared separately using a 2 x

2 ANOVA (Figure 2.2a, 2.2b – for comparison non-synaesthetic control data, n = 20,

is also shown, but not included in analysis). One participant from the self referral

group was withdrawn from analysis of reaction times due to an insufficient number of

correct responses (< 25% correct responses in any one condition).

Analysis of reaction time data revealed a significant main effect of

congruency, with subjects performing faster overall on trials which were congruent

with their synaesthesia compared to incongruent trials [F(1, 18) = 13.98, p = < .01].

Analysis of error type data revealed a significant main effect of error type, which was

due to a higher proportion of mirror-touch errors being produced relative to other

error types [F(1, 19) = 11.18, p = < .01]. No significant interaction or main effect of

group was found for reaction time (Group: [F(1, 18) = .048, p = .829]; Group x Cong:

[F(1. 18) = .095, p = .761]) or error type analysis (Group: [ F(1, 19) = 2.77, p = .113];

Group x Cong: [F(1. 19) = 2.75, p = .114]). This indicates that the prevalence and

self-referred mirror-touch synaesthete group come from the same population with

regard to congruency effects. Therefore, both prevalence and self-referred cases are

combined to consider additional cognitive characteristics of mirror-touch

synaesthesia.

For the majority of cases, the effects of spatial congruity are found for bodies

but not objects and this corresponds well with their phenomenological reports. There

are, however, a minority of synaesthetes who do report tactile experiences when

watching objects being touched (4 out of 21). For some of these synaesthetes, this

experience is reported in the finger tip that is touching the objects, but for others

synaesthetic touch is mapped onto particular body locations which are thought to

Chapter 2

46

spatially correspond to the object being touched (e.g. when looking directly at a

monitor the experience maps onto the face, but when standing in front of the monitor

the experience maps onto the trunk). In addition, another minority of synaesthetes (6

out of 21, including I., Z. and H.G. in Table 2.1) show an effect of spatial congruity

for both bodies and objects despite initially claiming to experience synaesthesia for

touched bodies alone. One possibility is that this reflects the fact that object trials are

interleaved with the more frequent human trials and this leads to objects being treated

more like human bodies than expected. In the normal population, fMRI studies

suggest that the tactile mirror system does respond to objects under some

circumstances (Ebisch et al., 2008; Keysers et al., 2004).

Of the 21 cases of mirror-touch synaesthesia reported to date, seventeen (nine

of which come from the prevalence sample) report a specular frame of reference and

four report an anatomical frame of reference. This finding is consistent with studies

on imitation behaviour which demonstrate that both adults and children tend to imitate

in a specular mode (Schofield, 1976; Franz, Ford, and Werner, 2007). The relative

bias in synaesthetes could be due to the fact that one’s own head is only ever seen

from a mirror-reflected perspective and this regularity may drive the choice of spatial

frame. However, it is to be noted that those synaesthetes who adopt a specular frame

for the head also do so with the hands (Banissy and Ward, 2007) even though this part

of one’s own body is not normally viewed from a reflected perspective.

A general characteristic of synaesthesia is that different variants of

synaesthesia tend to co-occur (Simner et al., 2006). Some preliminary evidence based

upon self reports suggests that this may also be the case with mirror-touch

synaesthetes. While, some mirror-touch synaesthetes only report mirror-touch

synaesthesia (implying that the overall prevalence of synaesthesia may be higher than

Chapter 2

47

the previously assumed 4% estimate), nine of the twenty-one mirror-touch

synaesthetes sampled also report genders or personalities for graphemes and/or certain

other linguistic stimuli (e.g. 3 is a bossy male; Simner and Holenstein, 2007; Smilek,

Malcolmson, Carriere, Eller, Kwan, and Reynolds, 2007). Five of these cases have

been confirmed using behavioural tests for this phenomenon (N. Sagiv, personal

communication). Additionally, seven report synaesthetic experiences of colour for

linguistic stimuli. Notably, this data is preliminary because the sample contains a

mixture of randomly (i.e. prevalence group) and non-randomly sampled participants

(i.e. self-referred group), and because members of the prevalence group were not

systematically tested for other variants of synaesthesia.

C

hapt

er 2

48 Synaesthete

Human Trials

Object Trials

Reaction time

% Mirror-touch

% Other

Reaction time

% Mirror-touch

% Other

D

431.24***

5.81**

0.58

11.26

0

0

I 38.97

10.29***

3.43**

-51.11

10.34***

5.17*

Z

-34.98

6.86**

0.57

-28.52

6.67**

1.67

E

79.45*

1.14

0

-18.2

0

0

K

84.84*

2.25

4.49***

-20.2

1.67

0

J

53.96

6.62***

0.60

7.77

0

1.75

R

532.13***

6.43***

0.58

54.38

0

0

H.S

214.25***

0.68

0.68

-7.05

0

0

H.G

136.38**

24.02***

0.56

52.67

10.17***

1.69

Tabl

e 2.

1 R

eact

ion

time

perf

orm

ance

(inc

ongr

uent

con

ditio

n re

actio

n tim

e m

inus

– c

ongr

uent

con

ditio

n re

actio

n tim

e) a

nd p

erce

ntag

e of

mir

ror-

touc

h an

d ot

her

erro

r ty

pes

for

pote

ntia

l sy

naes

thet

es w

hen

obse

rvin

g a

hum

an o

r co

rres

pond

ing

obje

ct b

eing

tou

ched

. T

hree

syn

aest

hete

s sh

owed

beh

avio

ural

cor

rela

tes

of m

irror

-touc

h sy

naes

thes

ia o

n re

actio

n tim

e on

ly, t

hree

on

mirr

or-to

uch

erro

rs o

nly,

and

thre

e on

bot

h re

actio

n tim

es a

nd e

rror

s (*

p = < 0

.05;

** p =

< 0

.01;

***

p = < 0

.001

rela

tive

to c

ontr

ol p

erfo

rman

ce).

C

hapt

er 2

49 a.

b.

550

600

650

700

750

800

850

900

950

1000

1050

1100

1150

Reaction Time (msec)

Congruent

Incongruent

Prevalence

Self referral

Control

**

05

10

15

20

25

30

% Error

Mirror-touch

Other

Prevalence

Self referral

Control

*

*

Figu

re 2

.2 M

ean

reac

tion

time

perf

orm

ance

(a)

and

per

cent

age

of e

rror

type

s (b

) on

hum

an tr

ials

for

mir

ror-

touc

h sy

naes

thet

es re

crui

ted

with

in

the

prev

alen

ce s

tudy

com

pare

d to

syn

aest

hete

s re

crui

ted

via

self

-ref

erra

l (co

ntro

l gro

up p

erfo

rman

ce is

sho

wn

for c

ompa

rison

). ±

s.e

.m (*

p =

<

0.05

).

Chapter 2 50

2.3 Experiment 2: Behavioural correlates and somatotopic cueing While the results from experiment 1 establish evidence for the authenticity of

mirror-touch synaesthesia and suggest that behavioural correlates are related to

‘observed bodily touch’, it remains unclear if the behavioural data could also be

consistent with ‘observed bodily cueing’ – whereby an observed visual event cues a

particular location on the body. There is growing evidence from research

investigating visual-tactile interactions that non-informative vision associated with

one’s own body can influence tactile processing (e.g. Johnson, Burton, and Ro, 2006).

In order to establish whether the findings could be related to somatotopic cueing, the

performance of mirror-touch synaesthetes and non-synaesthetic subjects was

compared on a condition in which a human face is observed but is accompanied by a

flash of light on the cheek rather than a touch. As these stimuli did not induce

synaesthesia it was expected that the pattern of effects shown by synaesthetes on

experiment 1 would be related specifically to ‘observed bodily touch’ and that

differences between synaesthetes and non-synaesthete controls would not be found for

‘observed bodily cueing’.

Method

Ten mirror-touch synaesthetes (7 females and 3 male, mean age ± Std. Error =

30.1 ± 11.17 years; 1 anatomical and 9 specular) and ten non-synaesthetic controls

matched for age and gender (7 females and 3 males, mean age ± Std. Error = 31 ±

13.23 years) took part. Congruency was determined according to synaesthetes’ self

reports when observing touch to another person. Controls were randomly allocated to

either a specular or anatomical congruency group to match the synaesthetic group.

The experimental task and procedure was the same as Experiment 1, with the

exception of the stimuli presented (Figure 2.3). For the human trials, rather than

Chapter 2

51

observing touch to the cheek(s), a flash of light appeared on the observed person’s

cheek(s). As before, the visual stimuli consisted of 3 frames which were sized to fit

the screen. The first stimulus, lasting 100 ms, depicted a male or female face. The

second stimulus, also lasting 100 ms, was the same as the first except that a patch of

white light appeared on the person’s left/right/both cheek(s). The flash was then

removed for the third stimulus which remained on the screen until the participant

responded. The tactile stimulus was applied 10 ms after the flash, i.e. at the onset of

the third stimulus. For the control trials, all pictures of the person were replaced by a

blank screen with a 100ms flash of light on the left, right or both sides of space

immediately before the tactile event. A total of 306 trials were completed, of which

180 involved human stimuli and 126 involved control stimuli.

Figure 2.3 Summary of the task used for somatotopic cueing experiment. Participants observed a flash of light on the left/right/or both cheek(s) of another person. Immediately following the light flash, subjects were touched on their own facial cheeks (either left, right or both cheeks). Participants were asked to report the site of real touch.

C

hapt

er 2

52

400

450

500

550

600

650

700

750

800

850

900

Reaction time (ms)

Congruent

Incongruent

Mirror-touch Syns

Control

05

10

15

20

25

30

% Error

Mirror-touch

Other

Controls

Mirror-touch Syns

500

550

600

650

700

750

800

850

900

950

1000

1050

Reaction time (ms)

Congruent

Incongruent

Mirror-touch Syns

Control

05

10

15

20

25

30

% Error

Mirror-touch

Other

Controls

Mirror-touch Syns

Figu

re 2

.4

Mea

n re

actio

n tim

e pe

rfor

man

ce a

nd p

erce

ntag

e of

err

or ty

pes

for

mirr

or -

touc

h sy

naes

thet

es a

nd n

on-s

ynae

sthe

tic c

ontro

l sub

ject

s ob

serv

ing

a lig

ht fl

ash

on a

noth

er p

erso

n’s

face

(a; b

) or a

ligh

t fla

sh o

nly

(c; d

). ±

s.e

.m.

a b

c d

Chapter 2 53

Results and Discussion

The results are summarised in Figure 2.4. Reaction times and error rates were

measured. Reaction time data were filtered prior to analysis (± 3 s.d. and all errors

removed). A 2 (Group) x 2 (Congruency) ANOVA conducted on reaction times

revealed no significant main effects or interactions (p = > .05 in all cases; Figure

2.4a). Although the direction of the effect was the same as in Experiment 1 the effect

was not significant. Analysis of the percentage of error types made by participants on

human trials also revealed no significant main effects or interactions (p = > .05 in all

cases; Figure 2.4b). Similarly, no significant differences were observed on control

trials (Figure 2.4c, d). These findings are unlikely to be due to the fact that the flash

of light is less salient than the hand, because the synaesthetes also failed to show an

effect in Experiment 1 when a hand was used on a non-human object.

To further validate that the performance of mirror-touch synaesthetes

significantly differed between experiment 1 and 2 a within-group comparison on the

size of congruency effect (incongruent minus congruent trial reaction time) shown by

synaesthetes across each task was conducted. This revealed that synaesthetes showed

a significantly greater effect of congruency on trials involving observed touch to a

human face in experiment 1 (mean ± s.e.m = 208.24 ± 52.32 msec) compared to a

flash of light shown on a human face in experiment 2 (mean ± s.e.m = 51.49 ± 34.18

msec), [t(9) = 2.98, p = < .02]. Thus the findings from Experiment 1 are related

specifically to ‘observed bodily touch’ and cannot be attributed to somatotopic

cueing.

Chapter 2

54

2.4 General Discussion Taken together, these measures detail the prevalence and characteristics of

mirror-touch synaesthesia. In relation to prevalence, the findings suggest that:

• mirror-touch synaesthesia is one of the more common forms of

synaesthesia

• there are two sub-types (specular and anatomical) depending on the

visuo-tactile spatial transformation used

• the specular (mirror-reflected) sub-type is the more common

• the effects are quite specific to observed touch to a human body.

In many respects, mirror-touch synaesthesia shares common ground with other

types of synaesthesia; for instance, with regards to phenomenology, automaticity,

consistency (of the spatial mapping), reliability over time, and possibly with regards

to associated traits (e.g. attributing personalities and genders to graphemes).

However, when one turns to consider its neural basis the similarities are less apparent.

A current area of debate in the synaesthesia literature is whether synaesthetic

experience is due to cross-activation between brain regions or cortical disinhibition

(Bargary and Mitchell, 2008; Cohen Kadosh, Henik, Catena, Walsh, and Fuetnes,

2009; Cohen Kadosh and Walsh, 2008; Grossenbacher and Lovelace, 2001; Hubbard

and Ramachandran, 2005; Rouw and Scholte, 2007). Thus far, accounts of

synaesthesia in terms of cross-activation have mainly focussed on grapheme-colour

synaesthesia and highlight the role of adjacency between visual grapheme and colour

processing areas in the fusiform gyrus (Ramachandran and Hubbard, 2001). It is

possible that adjacency is one of several biasing principles that influence which forms

of synaesthesia will, and will not, be found. Another biasing principle may be the

‘normal’ architecture for multi-sensory interactions. As noted before, there is now

Chapter 2

55

good evidence for a visuo-tactile mirror system in humans (Blakemore et al., 2005;

Ebisch et al., 2008; Keysers et al., 2004) and mirror-touch synaesthesia could be

construed as hyper-activity within this network (either as a result of cortical

disinhibition or cross-activation).

Below I propose a model of this type of synaesthesia.

A Neurocognitive Model of Mirror-Touch Synaesthesia: What, Who, Where.

In this model, mechanisms thought to underpin synaesthetic experience are

divided into processes involved in identifying the visual stimulus touched (“what”

mechanism – shown in red boxes), discriminating between self and other (“who”

mechanism – shown in blue boxes), and locating where on the body and in space

observed touch occurs (“where” mechanism – shown in green boxes). Connections

between processes common to all subjects are shown in black; connections between

processes necessary for an anatomical reference frame in purple; connections between

processes contributing to a specular reference frame are shown in orange (Figure 2.5).

Visual Encoding: “What” Mechanisms

With regards to the tactile mirror system, the putative “what” mechanisms are

needed to implement several discriminations. Is this a human or object? Is this a face

or body? One potential brain region which may be crucial to human body perception

in mirror-touch synaesthesia is the extrastriate body area (EBA; Downing, Jiang,

Shuman and Kanwisher, 2001). The EBA is a body-selective visual region which

responds more to bodies and body parts, than faces, objects and object parts (Downing

et al., 2001). This is in contrast to the fusiform body area (FBA; Peelen and

Downing, 2005), a further body selective visual region, which appears more important

for processing body parts into wholes (Taylor, Wiggett and Downing, 2007).

Chapter 2

56

Figure 2.5 The ‘What, Who, Where Model of Mirror-Touch Synaesthesia’. ‘What’ mechanisms are shown in red boxes and are involved in defining the stimulus touched. ‘Who’ mechanisms implement discriminations between self and other, and are shown in blue boxes. ‘Where’ mechanisms are shown in green boxes and are involved in locating where on the body and in space observed touch occurs. Processes necessary for all subjects are shown with black arrows, necessary for specular mirror-touch synaesthetes with orange arrows, and for anatomical mirror-touch synaesthetes with purple arrows. Brain regions represented are considered with regard to importance for mirror-touch synaesthesia. AI = Anterior Insula; EBA = Extrastriate Body Area; FBA = Fusiform Body Area; FFA = Fusiform face area; IFG = Inferior Frontal Gyrus; IPL = Inferior Parietal Lobule; IPS = Intraparietal Sulcus; LO = Lateral Occipital Cortex; SI = Primary Somatosensory Cortex; SII = Secondary Somatosensory Cortex; STS = Superior Temporal Sulcus; TPJ = Temporoparietal Junction.

In addition to the EBA, object selective visual regions and their interactions

along higher-order visual systems may then be crucial for distinguishing between

those synaesthetes for whom observing touch to objects elicit synaesthetic interactions

and for those in which no synaesthetic interaction is experienced. In cases where

observing touch to objects evokes synaesthesia, the processing of object information

Chapter 2

57

via the dorsal stream to areas along the medial bank of the intraparietal sulcus (IPS;

Konen and Kaster, 2008) may be particularly important. The IPS forms part of the

tactile mirror system (Blakemore et al., 2005) and is known to contain visual-tactile

body maps which are important for dynamic multisensory body representations

(Bremmer et al., 2001; Duhamel, Colby and Goldberg, 1998; Iriki, Tanaka and

Iwamura., 1996; Macaluso and Driver, 2003; also see Colby, 1998; Maravita and

Iriki, 2002 for review). Therefore the degree to which observing touch to an object is

able to elicit visual-tactile synaesthetic interactions may depend upon the extent to

which the object is incorporated into visual-tactile representations of the body,

potentially within the IPS.

Visual Encoding: “Who” Mechanisms

The most crucial distinction to be made by the putative “who” mechanism is

that between self and other. Is it my body/face that is seen?

One can consider mirror-touch synaesthesia as a breakdown in the

mechanisms that normally distinguish self from other. A dedicated module to

distinguish between self and other is not proposed; rather, this discrimination will

emerge out of other processes involved in linking visual representations with internal

representations of bodies. Namely, there may be a tendency to over-incorporate

viewed bodies within the observer’s current body schema (Coslett, 1998; Gallagher,

1995; Head and Holmes, 1911-1912; Sirigu, Grafman, Bressler, and Sunderland,

1991). This process is likely to depend on a variety of factors: the perspective of the

viewed body part; the current posture of the mirror-touch synaesthete; and the

similarity (facial or otherwise) between the perceiver and perceived.

Chapter 2

58

Figure 2.6 The influence of perspective on synaesthetic experience. (a) Observing touch to another person from one’s own perspective induces touch on an anatomically corresponding hand for both the anatomical and specular subtypes (i.e. observing touch to another’s left hand evokes synaesthesia on the synaesthete’s left hand). (b) For anatomical mirror-touch synaesthetes, synaesthetic touch is still evoked on the anatomically corresponding hand when observing touch to another person’s hand from another’s perspective (i.e. observing touch to another’s left hand evokes synaesthesia on the synaesthete’s left hand). (c) For specular mirror-touch synaesthetes, this not the case. When observing touch to another person’s hand from another’s perspective, synaesthesia is evoked on the mirrored right hand (i.e. observing touch to another’s left hand evokes synaesthesia on the synaesthete’s right hand). See Banissy and Ward (2007). Blue dots correspond to the location of the synaesthetic sensation evoked.

The perspective of the seen body part provides one way of discriminating

between self and other. The importance of discriminations between first-person and

third-person perspectives (Figure 2.6) also varies between synaesthetic subtypes when

observing touch to body parts (excluding the face) and this may require more

computations for specular compared to anatomical synaesthetes. For specular

Chapter 2

59

synaesthetes, touch to the hands from a first-person perspective induces synaesthetic

touch to the anatomically corresponding hand (i.e. right hand to right hand), but from

a third-person perspective induces synaesthetic touch to the mirrored hand (i.e. right

hand to left hand). In contrast, for anatomical synaesthetes, observed touch from

either perspective elicits synaesthetic touch to the anatomically corresponding hand

(c.f. Kusnir, MSc Thesis, University of London, 2008). The response of the right

EBA is greater for body parts in the third-person than first-person perspective (Saxe,

Jamal and Powell, 2006) and this brain region may contribute to this distinction. .

With regards to faces, viewing one’s own face activates a different network of

brain regions from other faces including famous or personally familiar ones (Uddin,

Iacoboni, Lange, and Keenan, 2007). FMRI research has highlighted the role of a

right-fronto-parietal network in this process, including the right inferior parietal lobule

(IPL) and right inferior frontal gyrus (IFG; Sugiura, Watanabe, Maeda, Matsue,

Fukuda, and Kawashima, 2005; Uddin, Kaplan, Molnar-Szakacs, Zaidel, and

Iacoboni, 2005). These two regions form part of the classical mirror neuron system in

humans (Rizzolatti and Craighero, 2004) and it has been suggested that they may be

necessary to not only establish shared representations, but also to implement

mechanisms to distinguish between self and other (Uddin, Molnar-Szakacs, Zaidel,

and Iacoboni, 2006). It may be the case that this same sensorimotor network is over-

active in mirror-touch synaesthetes when viewing faces other than their own, causing

the body part to be incorporated into the observer’s own body representations. One

prediction is that mirror-touch synaesthetes (at least the specular sub-type) will show

little behavioural or phenomenological differences on the spatial congruity task used

Chapter 2

60

here if the unfamiliar faces were replaced with images of their own faces2. However,

controls may begin to show similar behavioural performance to the synaesthetes if

images of their own face are displayed. In accordance with this, Serino and

colleagues (2008) report that, for non-synaesthetes, observing touch to one’s own or

another’s face increases tactile sensitivity on the observers own face (also see

Haggard, 2006 for similar evidence of interpersonal enhancements of touch). This

visual-tactile enhancement was maximised when observing touch to one’s own face

rather than another’s face, indicating that self-similarity can modulate the extent of

visuo-tactile resonance (Serino, Pizzoferrato, and Làdavas, 2008).

Perspective Taking: “Where” Mechanisms

The third class of mechanism that is considered to be relevant involves linking

visual representations of body with tactile representations based on proprioception

and somatosensation. One distinction that has been made in the literature is between

“embodied” and “disembodied” representations of body (Giummarra, Gibson,

Georgiou-Karistianis, and Bradshaw, 2007; also see Brugger, 2002 for a discussion of

similar spatial aspects of autoscopic phenomena). Evoked potential mapping

indicates that the right temporoparietal junction (TPJ) is related to disembodied

perspective taking (judging left/right from someone else’s perspective), while left

EBA activation is linked with embodied perspective taking (judging left/right from

own perspective; Arzy, Thut, Mohr, Michel and Blanke, 2006). Moreover,

stimulation of the TPJ has been shown to lead to disembodied experiences in

neurological patients (Blanke, Landis, Spinelli and Seeck, 2004; Blanke, Ortigue,

Landis and Seeck, 2002).

2 The predictions for synaesthetes with the anatomical sub-type are unclear because their usual synaesthetic phenomenology would contradict their own prior experiences of observing their own face in a mirror (e.g. when shaving or putting on make-up).

Chapter 2

61

This distinction is similar to the specular-anatomical division between mirror-

touch synaesthetes. For the specular sub-type, the visual representation of the other

body is spatially processed as if it is a mirror-image of one’s own (embodied) body.

For the anatomical sub-type, the spatial mapping is more disembodied in that one’s

own body is placed in the perspective of the other person (or one’s own body and that

of the other person are copied into some other shared bodily template). If this is the

case, it makes a specific and testable prediction - namely, that the anatomical sub-type

will be associated with greater activity in the TPJ than the specular sub-type.

Somatosensory Processes

A final component within the model is the role of somatosensation in mirror-

touch synaesthesia. Previous fMRI findings indicate that the condition is linked with

increased activations in SI, SII and additional activations in bilateral anterior insula

(Blakemore et al., 2005). The specific role of these regions in the experience of

synaesthetic touch remains unclear. For example, the anterior insula is connected

with both somesthetic cortex and visual association areas (Mesulam and Mufson,

1982; Mufson and Mesulam, 1982) which may make this brain region a potential

candidate for accounts of mirror-touch synaesthesia in terms of mechanisms of

disinhibition or hyper-connectivity. This brain region also contains tactile receptive

fields in the absence of activations of primary somatosensory cortices (Olausson et al.,

2002) and is important in processing the affective consequences of touch (Craig,

2002). In this sense, anterior insula activations observed in mirror-touch synaesthesia

may reflect processing of tactile and affective consequences of synaesthetic

experience; self reports indicate that the synaesthetic tactile sensation varies with the

type of touch observed (i.e. pain versus touch) and has differing affective

consequences accordingly. Alternatively, the anterior insula is also important in

Chapter 2

62

distinguishing between self and other (Fink, Markowitsch, Reinkemeier, Bruckbauer,

Kessler, and Heiss, 1996; Kircher et al., 2001; Ruby and Decety, 2001) and this

region could be involved in misattributing observed touch to oneself through

mechanisms of self-other discrimination (Blakemore et al., 2005). The use of brain

imaging to investigate more closely the interactions between activations in the

anterior insula and primary somatosensory cortices observed in mirror-touch

synaesthesia may shed light on these issues.

Summary

In summary, by investigating the prevalence and characteristics of mirror-

touch synaesthesia it has been shown that this variant of the condition may be one of

the most common forms of synaesthesia. Furthermore, there are a number of

important characteristics which indicate that the condition goes beyond a simple one

to one mapping between observed and synaesthetic touch. A neurocognitive model is

proposed (Figure 2.5), which distinguishes between subtypes of mirror-touch

synaesthesia and suggest potential neural mechanisms to account for how differences

in the interpersonal body maps adopted may lead to different cognitive processes

related to synaesthetic experience.

Chapter 3

63

CHAPTER 3: SENSORY PROCESSING IN SYNAESTHESIA

The studies presented in chapter 2 explored the prevalence and characteristics of

mirror-touch synaesthesia. Here I investigate the perceptual characteristics

associated with the presence of mirror-touch synaesthesia. Previous findings imply

that synaesthetic experience may have consequences for sensory processing of stimuli

that do not themselves trigger synaesthesia. For example, synaesthetes who

experience colour show enhanced perceptual processing of colour compared to non-

synaesthetes. This study aimed to investigate whether enhanced perceptual

processing was a core property of synaesthesia by contrasting tactile and colour

sensitivity in synaesthetes who experience either colour, touch, or both touch and

colour as evoked sensations. For comparison the performance of non-synaesthetic

control subjects was also assessed. There was a relationship between the modality of

synaesthetic experience and the modality of sensory enhancement. Synaesthetes who

experience colour have enhanced colour sensitivity and synaesthetes who experience

touch have enhanced tactile sensitivity. These findings suggest the possibility that a

hyper-sensitive concurrent perceptual system is a general property of synaesthesia

and are discussed in relation to theories of the condition.

3.1 Introduction

As noted previously, synaesthesia is a developmental condition in which one

property of a stimulus results in conscious perceptual or ‘percept like’ experiences of

an additional attribute. The authenticity of the condition is now well established (for

reviews see Cohen Kadosh and Henik, 2007; Hubbard and Ramachandran, 2005; Rich

and Mattingley, 2002) and a number of psychophysical studies indicate that

synaesthetic experience resembles veridical sensory perception (but see Hong and

Blake, 2008), e.g. in grapheme-colour synaesthetes, synaesthetic and real colours

interact in binocular rivalry (Kim, Blake, and Palmeri, 2006); can induce orientation

contingent colour adaption after-effects such as a synaesthetic ‘McCollough Effect’

(Blake, Palmeri, Marois, and Kim, 2004); and can combine to produce apparent

motion (Kim et al., 2006). Under certain circumstances, synaesthetic experience may

also aid veridical sensory perception. Vision-sound synaesthetes (synaesthetes for

whom seeing visual motion triggers auditory perception) show an advantage at

Chapter 3

64

perceiving visually presented rhythmic patterns compared to non-synaesthetes.

Typically, non-synaesthetes are superior at recognising auditory compared to

equivalent visual rhythmic patterns, but vision-sound synaesthetes recode the visual

information aurally leading to superior visual rhythm perception (Saenz and Koch,

2008).

Recent ERP evidence indicates that the presence of synaesthesia may also

exert a wider influence over veridical sensory processing (Barnett et al., 2008; Goller,

Otten, and Ward, 2009; Yaro and Ward, 2007). Barnett and colleagues (2008) report

that, compared to non-synaesthetes, linguistic-colour synaesthetes show differences in

early components of the visual evoked potential (VEP) when presented with simple

visual stimuli which do not evoke synaesthesia. VEP differences were observed

following the presentation of high spatial frequency Gabor patches which

preferentially activate the parvocellular pathways (pathways highly responsive to

colour; Kaplan, 1991) of the visual system (Barnett et al., 2008). Goller et al. (2009)

report similar early VEP differences between tone-colour synaesthetes and non-

synaesthetic controls following the presentation of coloured visual stimuli. These

findings indicate electrophysiological differences in two groups of synaesthetes for

stimuli which do not themselves trigger synaesthetic experience, implying that

synaesthesia may be linked with general differences in veridical sensory perception.

Behavioural correlates of this may include enhanced perceptual processing for stimuli

related to synaesthetic experience. For example, Yaro and Ward (2007) report that

synaesthetes who experience colour show better perceptual discrimination of colour

relative to non-synaesthetic control subjects.

Although synaesthetic experiences are nearly always unidirectional in terms of

conscious experience (e.g. graphemes or sounds trigger colour but not vice versa)

Chapter 3

65

there is good evidence of implicit bidirectionality (Cohen Kadosh and Henik, 2006;

Cohen Kadosh, Cohen Kadosh and Henik, 2007). It is possible that previously

reported sensory enhancements in colour processing (Yaro and Ward, 2007) are not

localised within the colour domain but reflect back-coding, e.g. into the verbal

domain. As such it is important to test other varieties of synaesthesia in which this

possibility is less likely. For example, ‘mirror-touch’ synaesthetes experience tactile

sensations on their own body when observing touch to another person (Banissy and

Ward, 2007; Blakemore, Bristow, Bird, Frith, and Ward, 2005). To address this, the

current study investigates tactile and colour perception in three groups of synaesthetes

and a group of non-synaesthetic control subjects. The experiment was a 2x2 between

subjects design in which I contrasted presence/absence of mirror-touch synaesthesia

with presence/absence of synaesthetic colour experiences (including but not limited to

grapheme-colour synaesthesia). The group reporting an absence of both types of

synaesthesia are termed ‘normal’ control group. The three synaesthetic groups either

have both types (hereafter referred to as dual-synaesthetes), or only one of these types.

Based on previous research synaesthetes were expected to show enhanced perceptual

sensitivity, but it remains to be shown whether this is specific to the modality (or

modalities) that participate in the synaesthesia.

3.2 Methods

Participants

The touch-synaesthete group were comprised of six mirror-touch synaesthetes

who do not experience any other forms of synaesthesia (4 female and 2 male; mean

age ± s.e.m = 35.5 ± 3.93 years). Eight mirror-touch synaesthetes who also experience

some form of colour synaesthesia (i.e. grapheme-colour, digit-colour, tone-colour)

formed the dual-synaesthete group (6 female and 2 male; mean age ± s.e.m = 39.14 ±

Chapter 3

66

3.74 years). Eight synaesthetes who experience synaesthetic perceptions of colour

only (7 digit-colour synaesthetes and 1 letter-colour synaesthete who also reports

digit-colour) were recruited for the colour-synaesthete group (7 female and 1 male;

mean age ± s.e.m = 30.83 ± 3.01 years). None of the colour synaesthetes reported that

colour elicited synaesthesia (e.g. colour-to-sound). Additionally, 20 non-synaesthetic

control subjects were recruited for the experiment (16 female and 4 male; mean age ±

s.e.m = 31.4 ± 3.66 years).

Cases of mirror-touch synaesthesia were confirmed on a visual-tactile spatial

congruity paradigm designed to provide evidence for the authenticity of the condition

(Banissy and Ward, 2007; described in Chapter 1 and 2). All cases of synaesthesia

involving colour were confirmed using tests of consistency over time, with subjects

demonstrating test-retest consistency of 85% (for letters, numbers, or other verbal

stimuli) or a score of ≤ 0.75 on the Eagleman Synaesthesia Test Battery (Eagleman,

Kagan, Nelson, Sagaram, and Sarma, 2007).

Materials and Procedure

Subjects completed two tests of sensory perception in a counterbalanced order:

The Farnsworth-Munsell Colour Hue Test and the Gratings Orientation Test (Van

Boven and Johnsen, 1994).

The Farnsworth-Munsell Colour Hue Test is a test of colour discrimination.

The apparatus is a palette of different colour hues with identical luminance and

saturation. Each hue is presented as a coloured cap and when arranged correctly the

set forms a regular colour series transforming from one hue to another. Four colour

series are presented in different trays, each containing 23 or 24 colours showing a

distinct colour transformation. The procedure for the task is as follows. For each tray,

the coloured caps are removed and arranged on the table in front of the participants.

Chapter 3

67

Two coloured caps remain in the tray, which represent the two end points of the

colour sequence (e.g. a purple and pink cap). The participant is given 2 minutes to

arrange the remaining caps into an ordered colour series from one hue to another (e.g.

purple through violet through pink). The correct order for the hues can be identified

by the experimenter from the numeric coding on the underside of each cap. A

deviation score is calculated by considering how far each colour cap deviates from the

correct location in the sequence. For example, in a correct ordering such as 4–5–6;

colour number “5” has a score of 2 because it is 1 unit from 4 and 1 unit from 6. An

incorrect ordering such as 2–5–9 would yield a score of 7 for colour “5” because it is

3 units from “2” and 4 units from “9”. The error score is the difference between the

actual score obtained and the expected score based on flawless ordering. The same

procedure was used for each of the four trays and the order of trays was randomized

across participants.

In order to investigate tactile discrimination, the Gratings Orientation Test

(GOT) was used to measure tactile acuity on the index finger tip. The GOT is a well

established method for measuring the spatial resolution of touch. It consists of a

series of square wave gratings with varying ridge widths (0.35mm – 3mm; Van Boven

and Johnsen, 1994). Each grating is applied to the finger tip in one of two orthogonal

orientations (across or along the axis of the finger tip). The task is to report the

orientation of the probe. Typically as ridge width decreases, accuracy decreases. The

GOT is thought to reflect cortical representations of the finger tip in SI (Van Boven

and Johnsen, 1994) and in this vein provides a method for accurate threshold

estimates of sensory function.

Six spatial grating probes (0.35mm, 0.5mm, 0.75mm, 1mm, 1.25mm, 1.5mm)

were used to investigate each participant’s tactile sensory threshold. Using a blocked

Chapter 3

68

design (20 trials per block; Van Boven and Johnsen, 1994) each probe was applied

manually to the participant’s right index finger tip in one of two different orientations

(across or along the axis of the finger tip). Manual application was chosen because

performance in spatial resolution is generally insensitive to the force of application

(Johnsen and Philips, 1981) and the receptive fields of the afferent population

involved in grating orientation detection are relatively independent of skin indentation

(Vega-Bermudez and Johnsen, 1999). Participants were asked to indicate the

orientation of each probe by giving a verbal response (i.e. “across” or “along”). In

total participants completed 20 trials per probe (120 trials in total) which were

randomised across blocks. For each probe half of the trials were orientated across the

finger tip and the remainder were applied along the finger tip. Participants were

blindfolded during the task to prevent any visual cues to orientation. Prior to

threshold measurement, participants completed two practice blocks, using 2mm and

3mm gratings, in which feedback was given on participants’ responses. No feedback

was given on trials involved in threshold detection.

3.3 Results

Farnsworth-Munsell Colour Hue Test The results from the Farnsworth-Munsell Colour Hue Test are shown in figure

3.1. The test is measured according to a total error score (TES) based on the deviation

from the expected ordering (it is not a percentage error). Superior performance is

reflected by a lower TES.

To investigate if the presence of synaesthesia for colour or touch was linked

with superior performance, a 2 (presence/absence of colour synaesthesia) x 2

(presence/absence of touch synaesthesia) ANOVA was conducted. It was predicted

that synaesthetes who experience colour would show superior performance on the

Chapter 3

69

colour perception task compared to those who do not. This was found to be the case,

a significant main effect of Colour Group was found [F(1,38) = 4.61, p = <.05]. The

presence of colour synaesthesia was linked with a lower TES (mean ± s.e.m = 48.22 ±

7.7), and therefore better colour discrimination, than the absence of colour

synaesthesia (mean ± s.e.m = 75.54 ± 10.1). This replicates previous reports (Yaro

and Ward, 2007). No main effect of touch group or any interaction between groups

was found. Therefore, the presence of colour synaesthesia, but not touch

synaesthesia, was linked with enhanced colour discrimination.

To further delineate the contribution of different variants of synaesthesia to

performance, the performance between each subgroup of subjects (colour-

synaesthetes; touch-synaesthetes; dual-synaesthetes; non-synaesthete controls) was

compared using a one-way ANOVA (Figure 3.1). The main effect of group

approached significance [F(3,38) = 2.45, p = .078]. In order to examine the basis of

this strong trend, and to test the a priori assumption that synaesthetes would show

enhanced colour sensitivity relative to control subjects, a series of planned t-tests were

carried out (cf. Howell, 2002, pg. 372-373). This revealed that synaesthetes who

experience colour (but not touch) as their induced experience showed superior colour

discrimination relative to non-synaesthetic control subjects [t(26) = 2.04, p = <.05].

This was also true of dual-synaesthetes – synaesthetes who experience touch and

colour significantly outperformed non-synaesthetic control subjects [t(25.90) = 2.47, p =

<.05]. No significant differences were found between the colour discrimination

abilities of synaesthetes who only experience touch-synaesthesia and non-synaesthetic

control subjects [t(24) = .62, nsig]. This indicates that the presence of ‘colour’

synaesthetic experience (rather than synaesthesia per se) is related to enhancements in

the perceptual processing of colour relative to non-synaesthetic control subjects.

Chapter 3

70

Figure 3.1 Synaesthetes who experience colour outperformed individuals who do not experience synaesthetic colour on a measure of colour perception (a). This pattern of performance was shown by synaesthetes who experience colour only relative to non-synaesthetic controls and by synaesthetes who experience both colour and touch relative to non-synaesthetic controls, but not by synaesthetes who experience touch only relative to controls (b). Error scores are based on the deviation from the expected ordering of hues. Superior colour performance is indicated by a lower error score. * = p < .05. Gratings Orientation Test Figure 3.2 shows the average tactile discrimination thresholds for all subjects

on the gratings orientation test. Enhanced tactile discrimination is reflected in a lower

threshold value (in millimetres). Thresholds were calculated using the following

formula and provide an estimate of the grating level which would lead to a 75%

response level:

g75 = glow + ((0.75 -plow)/(phigh - plow)) (ghigh - glow) g = grating spacing

20

30

40

50

60

70

80

90

100

Group

Colour Sensitivity: Total Error Score

Touch-Synaesthetes

Dual-Synaesthetes

Colour-Synaesthetes

Non-synaesthetes

*

Chapter 3

71

p = trials correct / n n = number of trials high=the grating spacing on the lowest grating spacing on which the participant responded correctly better than 75% of the time. low=the grating spacing on the highest grating spacing on which the participant responded correctly less than 75% of the time. g75 =the interpolated grating spacing on which the subject would have scored 75% had it been present. A 2 (presence/absence of colour synaesthesia) x 2 (presence/absence of touch

synaesthesia) ANOVA revealed a significant main effect of Touch Group (F(1,40) =

13.44, p = <.01). This was because synaesthetes who experience touch (either touch

only or dual-synaesthetes) showed heightened tactile sensitivity (mean ± s.e.m = 0.79

± 0.05 mm) compared to participants who do not experience synaesthetic touch (mean

± s.e.m = 1.25 ± 0.08 mm). No significant main effect of Colour Group or interaction

was observed, indicating that the presence of synaesthetic touch was linked with

heightened tactile sensitivity but not the presence of synaesthesia in general.

As with colour discrimination performance, tactile discrimination performance

between all four groups (colour-synaesthetes; touch-synaesthetes; dual-synaesthetes;

non-synaesthete controls) was compared using a one-way ANOVA. A significant

main effect of group was observed [F(3,38) = 4.50, p = .008]. Post-hoc comparisons

(corrected using Fisher’s LSD) revealed that this was because synaesthetes who only

experienced mirror-touch synaesthesia showed superior tactile discrimination relative

to both colour-synaesthetes (p = < .05) and non-synaesthetic control subjects (p = <

.05). Dual-synaesthetes also significantly outperformed both colour-synaesthetes (p =

< .05) and non-synaesthetic control subjects (p = < .01). No significant differences

were found between ‘colour-only’ synaesthetes and non-synaesthetic (p = .694)

controls or between touch-synaesthetes and dual-synaesthetes (p = .891). Therefore

the presence of synaesthesia for touch was related to enhancements in the perceptual

processing of touch relative to the absence of synaesthesia for touch (Figure 3.2).

Chapter 3

72

Figure 3.2 Synaesthetes who experience touch outperformed individuals who do not experience synaesthetic touch on a measure of tactile perception (a). This pattern of performance is shown by synaesthetes who experience touch only relative to non-synaesthetic and synaesthetic control subjects, and by synaesthetes who experience touch and colour relative to non-synaesthetic and synaesthetic control subjects. No differences in tactile perception were found between synaesthetes who experience colour only and non-synaesthete control subjects (b). Superior tactile sensitivity is indicated by a lower tactile threshold (mm). * = p < .05. 3.4 Discussion

This study extends previous reports of enhanced perceptual processing in

grapheme-colour synaesthesia (Yaro and Ward, 2007) and aimed to clarify whether

synaesthesia in other modalities has similar repercussions for perceptual processing of

stimuli in that modality. Using a sample of synaesthetes who experience either colour,

touch, or both touch and colour as evoked sensations, the findings first replicate

previous reports that synaesthetes who experience colour show superior perceptual

0.25

0.5

0.75

1

1.25

1.5

1.75

Group

Tactile Threshold (mm)

Touch-Synaesthetes

Dual-Synaesthetes

Colour-Synaesthetes

Non-synaesthetes

*

Chapter 3

73

discrimination of colour relative to non-synaesthetic subjects. They extend this by

showing:

i) synaesthetes who experience only touch show enhanced perceptual

discrimination of touch but not of colour,

ii) synaesthetes who experience both a tactile and visual concurrent show

enhanced perceptual processing of both touch and colour (although the

robustness of differences in colour processing was less strong than for tactile

processing),

iii) synaesthetes who experience only colour do not show enhanced sensory

processing in modalities outside of vision.

These findings suggest that enhanced perceptual processing is a core property

of synaesthesia, which is not limited to colour but occurs in each affected sensory

modality.

There are two possible accounts for why synaesthetes should demonstrate an

oversensitive concurrent perceptual system: 1) enhanced perceptual processing is a

consequence of the additional synaesthetic percepts which are experienced in

everyday life (i.e. enriched perceptual experience leads to enhanced perceptual

processing) or 2) enhanced perceptual processing is related to differences in brain

development as a function of synaesthesia (which may be either a cause or

consequence of synaesthesia; i.e. widespread differences in cortical connectivity or

mechanisms of cortical unmasking; Rouw and Scholte, 2007; Cohen Kadosh et al.,

2009).

Under the first account, an oversensitive concurrent perceptual system would

be explained as a proximal consequence of synaesthetic experience. For example, the

presence of stable synaesthetic associations may impact on the internal structure of

Chapter 3

74

sensory representations. In the case of colour, it has been suggested that linguistic

labels are necessary to categorize colours across a perceptual continuum (Davidoff,

2001) and cross-cultural differences in the number of colour labels has been shown to

influence colour perception and memory (Robertson, Davies, and Davidoff, 2000). It

may be the case that for some colour synaesthetes (i.e. grapheme-colour

synaesthesia), the presence of stable associations with colour increases the number of

colour terms and thereby impacts on the internal structure of colour space (Yaro and

Ward, 2007). In accordance with this, Simner and colleagues (2005) report that

grapheme-colour synaesthetes produce a greater depth of colour descriptions and use

more colour terms than non-synaesthete control participants. It is unclear however,

how this would extend to other variants and concurrent perceptual systems such as

enhanced tactile acuity in mirror-touch synaesthesia.

Additionally, insights into the influence of enriched perceptual experience on

sensory enhancement in the deprived brain would suggest that it is unlikely that extra

synaesthetic percepts are the cause of superior perceptual processing (Pascual-Leone,

Amedi, Fregni, and Merabet, 2005). For example, blind subjects have been shown to

be superior to sighted subjects on the grating orientations test, however this

superiority does not correlate with Braille reading experience or differ between Braille

readers and blind non-readers (Goldreich and Kanics, 2003; Van Boven, Hamilton,

Kauffman, Keenan, and Pascual-Leone, 2000). This indicates that mechanisms of

cross-modal plasticity following visual deprivation (rather than increased tactile

experiences) drives tactile acuity enhancement in the blind (Goldreich and Kanics,

2003). Consistent with this, in sighted-subjects visual deprivation induced by long-

term blindfolding results in temporary enhancements of passive tactile acuity, but

enriched tactile experience on the same finger does not (Kauffman, Théoret, Pascual-

Chapter 3

75

Leone, 2002) – further implying that elevated sensory sensitivity may be linked to

mechanisms of cross-modal plasticity rather than additional sensory experience.

Two mechanisms of cross-modal plasticity have been suggested to account for

compensatory changes in the deprived brain: cortical unmasking of pre-existing

connections and cortical reorganisation (e.g. Pascual-Leone et al., 2005; Wittenberg et

al., 2004). Mechanisms of cortical unmasking (through processes such as

disinhibition) involve the strengthening of existing within and between region

anatomical pathways (i.e. functional but not structural differences), while mechanisms

of cortical reorganisation involve the establishment of new local and widespread

anatomical connections (i.e. structural differences). The role of these mechanisms on

compensatory change is thought to reflect differences in the speed of plasticity, with

unmasking representing a form of rapid change which if sustained leads to long

lasting cortical changes such as the establishment of new anatomical connections

(slow acting mechanism; Pascual-Leone et al., 2005). For example, in the case of

temporary enhancements in tactile acuity following blindfolding, unmasking of

existing connections may offer a fast-acting mechanism of cross-modal plasticity to

maintain functional behaviour (i.e. perception of the environment through rapid

enhancements in tactile processing). In comparison, in the case of enhanced tactile

processing in the blind, sustained unmasking of existing connections may lead to new

local and widespread anatomical pathways resulting in long lasting enhancements in

tactile acuity which aid in daily life (Pascual-Leone et al., 2005).

There is growing evidence that synaesthesia may act upon the ‘normal’

architecture for cross-modal interactions (see Sagiv and Ward, 2006 for review) and

parallels may be drawn between mechanisms of cross-modal plasticity following

compensatory changes in the deprived brain and those which have been suggested to

Chapter 3

76

underlie synaesthetic experience (Cohen Kadosh and Walsh, 2006). A current area of

debate is whether synaesthetic experience is a consequence of additional structural

connectivity between brain regions (i.e. structural differences akin to cortical

reorganisation following sensory deprivation), malfunctions in cortical inhibition (i.e.

functional but not structural differences akin to cortical unmasking following sensory

deprivation), or a combination of both (Bargary and Mitchell, 2008; Cohen Kadosh

and Henik, 2007; Cohen Kadosh and Walsh, 2008; Grossenbacher and Lovelace,

2001; Hubbard and Ramachandran, 2005; Rouw and Scholte, 2007; Smilek et al.,

2001). Supporting evidence for structural connectivity accounts is provided by

diffusion tensor imaging findings that grapheme-colour synaesthetes show increased

structural connectivity in inferior-temporal, parietal and frontal brain regions when

compared to non-synaesthetes (Rouw and Scholte, 2007). Evidence for inhibition

accounts is provided by findings that synaesthetic-like experiences can be induced

following hallucinogenic drugs (Aghajanian and Marek, 1999) and that grapheme-

colour synaesthesia can be induced in non-synaesthetes (individuals without aberrant

connectivity) using post-hypnotic suggestion (Cohen Kadosh et al., 2009). It is

plausible that enhanced sensory perception in synaesthesia may reflect a combination

of these mechanisms. Mechanisms of reduced inhibition may act by unmasking local

anatomical pathways (akin to visual deprivation studies in sighted subjects; Cohen

Kadosh and Henik, 2007; Cohen Kadosh and Walsh, 2008), while altered cortical

connectivity may lead to enhanced perceptual sensitivity through aberrant circuitry

within the concurrent perceptual system (akin to sensory sensitivity enhancements in

the blind). It will be interesting to determine if altered connectivity in synaesthesia

(Rouw and Scholte, 2007) may reflect sustained unmasking of existing connections

Chapter 3

77

(e.g. Cohen Kadosh et al., 2009) and what implications this may have for veridical

sensory processing.

In summary, this study extends previous findings that grapheme-colour

synaesthetes show enhanced perceptual processing of colour (Yaro and Ward, 2007)

to suggest that an oversensitive concurrent perceptual system is a core property of

synaesthesia. Mirror-touch synaesthetes were shown to have enhanced tactile

sensitivity only, synaesthetes who experience both mirror-touch and a form of colour

synaesthesia were shown to demonstrate enhanced tactile and colour perception, and

synaesthetes who only experience colour were shown to have enhanced perceptual

processing of colour only. These findings imply that the presence of synaesthesia has

repercussions for sensory processing for stimuli which do not themselves induce

synaesthetic experience. It remains to be determined whether an oversensitive

concurrent perceptual system is a cause or consequence of synaesthesia.

Chapter 4

78

CHAPTER 4: MIRROR-TOUCH SYNAESTHESIA AND

EMPATHY

In the preceding chapters I investigated the behavioural correlates and perceptual

consequences of mirror-touch synaesthesia. Here I consider the implications of

mirror-touch synaesthesia for general cognitive processing. Previous fMRI findings

link mirror-touch synaesthesia to heightened activations in the mirror-touch system

(the same neural system activated in non-synaesthetes when observing touch to

others). It has been suggested that components of the mirror-touch system may act to

facilitate processes such as empathy and emotion recognition because they provide

the perceiver with a neurophysiological mechanism to simulate what it would “feel

like” to be in the same situation. To examine this possibility, two experiments were

conducted to investigate the influence of heightened sensorimotor simulation in

mirror-touch synaesthesia on empathy. Experiment 1, ‘Mirror-touch synaesthesia

and empathy’, demonstrates that mirror-touch synaesthesia, but not other variants of

synaesthesia, is linked with heightened empathic abilities for specific components of

empathy. Experiment 2, ‘Empathy and personality’, extends the findings from

experiment 1 by demonstrating that differences in empathy are ‘other’ rather than

‘self’ orientated reactions.

4.1 Introduction

Empathy is a higher order psychological construct and is considered to have

two main strands: (i) cognitive empathy – predicting and understanding another’s

mental state by using cognitive processes (i.e. role / perspective taking), and (ii)

affective empathy – experiencing an appropriate emotional response as a consequence

of another’s state (Baron-Cohen and Wheelwright, 2004; Decety and Jackson, 2004;

Preston and de Waal, 2002). Evolutionary perspectives suggest that there are several

possible systems which mediate this division, including phylogentically early

emotional contagion systems and more recently evolved cognitive perspective taking

mechanisms (De Waal, 2007), with the former thought to play a crucial role in

supporting the ability to empathize emotionally (e.g. I feel sad when I see someone

else sad) and the later considered to be linked to more complex empathic cognitions

including perspective taking and mentalizing.

Chapter 4

79

Recent research has suggested that one neurophysiological mechanism which

may mediate people’s abilities to empathize and understand the emotions of others is

shared affective neural systems in which common brain areas are activated during

both experience and passive observation of other’s experiences. Moreover, building

on the discovery of mirror neurons in the monkey brain (Gallese, Fadiga, Fogassi, and

Rizzolatti, 1996; Rizzolatti and Craighero, 2004), functional brain imaging has

suggested the existence of mirror systems in humans not only for actions (e.g.

Buccino et al., 2001), but also for sensations and emotions (e.g. disgust: Jabbi, Swart

and Keysers, 2006; Wicker, Keysers¸ Plailly, Royet, Gallese, and Rizzolatti, 2003;

touch: Blakemore, Bristow, Bird, Frith, and Ward, 2005; Ebisch, Perucci, Ferretti, Del

Gratta, Luca Romani, and Gallese, 2008; Keysers, Wicker, Gazzola, Anton, Fogassi,

and Gallese, 2004; pain: Avenanti, Beuti, Galati, and Aglitoi, 2005; Bufalari, Aprile,

Avenanti, Di Russo, and Aglioti, 2007; Morrison, Lloyd, di Pellegrino, and Roberts,

2004; Singer, Seymour, O’Doherty, Kaube, Dolan, and Frith, 2004; emotion: Carr,

Iacoboni, Dubeau, Mazziotta, and Lenzi, 2003). These systems may be crucial for

empathy because they enable the observer to simulate another’s experience by

activating the same brain areas that are active when the observer experiences the same

emotion or state (Gallese, 2006; Gallese and Goldman, 1998; Keysers and Gazzola,

2006; Oberman and Ramachandran, 2007). Consistent with this, is evidence that

increased activations in the auditory mirror system are correlated with high self

reported empathy (Gazzola, Aziz-Zadeh, and Keysers, 2006); that increases in trait-

cognitive empathy are correlated with increases in sensorimotor simulation when

viewing others’ pain (Avenanti, Minio-Paluello, Bufalari, and Aglioti, 2009); and that

participants self reported empathy skills are positively correlated with levels of

cortical mirroring of when witnessing disgust (Jabbi et al., 2006). Furthermore, there

Chapter 4

80

is a growing body of evidence suggesting that individuals with autistic spectrum

disorder (ASD) have impaired activity in the action mirror system (Dapretto, Davies,

Pfeifer, Scott, Sigman, Bookheimer, and Iacoboni, 2006; Oberman, Hubbard,

McCleery, Altschuler, Ramachandran, and Pineda, 2005), which may lead to the

deficits in imitation and empathy observed in ASD (Iacoboni and Dapretto, 2006;

Oberman and Ramachandran, 2007; but see Southgate and Hamilton, 2008).

As discussed in preceding chapters, previous functional magnetic resonance

imaging findings indicate that synaesthetic tactile experiences in mirror-touch

synaesthesia are associated with hyperactivity in the same mirror-touch network that

is evoked by observed touch in non-synaesthete controls in which no overt tactile

experience is elicited (Blakemore et al., 2005). As such, mirror-touch synaesthesia

may offer a unique opportunity to explore the role that the tactile mirror system has in

empathy because it enables investigations into the relationship between heightened

sensorimotor simulation in the mirror-touch system and empathic sensitivity.

To address this possibility two experiments were conducted. In experiment 1,

the empathic abilities of ten mirror-touch synaesthetes were compared to a

synaesthetic and non-synaesthetic control group. In experiment 2, potential factors

which may contribute to heightened empathy were investigated by contrasting mirror-

touch synaesthetes with non-synaesthetic participants on empathy and personality

measures.

4.2 Experiment 1: Mirror-touch synaesthesia and empathy

Participants

Ten mirror-touch synaesthetes (6 females and 4 males, mean age ± Std. Error

= 37.6 ± 5.59 years) and twenty non-synaesthetic controls matched for age and gender

(12 females and 8 males, mean age ± Std. Error = 32.95 ± 3.24 years) took part in the

Chapter 4

81

study. All cases of mirror-touch synaesthesia were confirmed on the visual-tactile

spatial congruity paradigm described previously (Banissy and Ward, 2007; also see

Chapter 1 and Chapter 2 of this thesis for a description of the task).

In addition to this, twenty-five synaesthetes (22 females and 3 males, mean

age ± Std. Error = 43.96 ± 3.38 years) experiencing other forms of synaesthesia

(minimally grapheme-colour synaesthesia) but not mirror-touch synaesthesia took

part. These synaesthetes acted as a synaesthetic control group and demonstrated test-

retest consistency of ≥ 85% for letters, numbers and other verbal stimuli. The

synaesthete control group were included to ensure that any differences in empathy

were not due to a general feature of synaesthesia.

Materials and Procedure

All participants completed the Empathy Quotient (Baron-Cohen, Richler,

Bisarya, Gurunathan, and Wheelwright, 2003; Baron-Cohen and Wheelwright, 2004).

The EQ is a self report scale designed to empirically measure empathy. As noted

previously, empathy is a higher order construct and has been theorised as having two

main strands: (i) cognitive empathy – predicting and understanding another’s mental

state by using cognitive processes (i.e. role / perspective taking), and (ii) affective

empathy – experiencing an appropriate emotional response as a consequence of

another’s state (Baron-Cohen and Wheelwright, 2004; Preston and de Waal, 2002).

The EQ was developed to measure both cognitive and affective components of

empathy, has been validated on both clinical and control groups (Baron-Cohen et al.,

2003), and it has been shown to distinguish between these groups (Baron-Cohen et al.,

2003; Baron-Cohen and Wheelwright, 2004). In addition to this, the EQ has been

validated on measures of concurrent validity (Lawrence, Shaw, Baker, Baron-Cohen,

and David, 2004) and has been shown to have high test-retest reliability over 12

Chapter 4

82

months (Baron-Cohen et al., 2003). The scale is comprised of 40 test items and 20

filler items. All items are a series of statements (e.g. ‘I can tune into how someone

feels rapidly and intuitively) and responses are given on a 4 point scale ranging from

‘strongly agree’ to ‘strongly disagree’. Responses score 1 or 2 points for an empathic

response and 0 points for all other responses. Principal component analysis has

indicated that the EQ is comprised of three main factors (i) cognitive empathy, (ii)

emotional reactivity and (iii) social skills (Lawrence et al., 2004; Muncer and Ling,

2006). Confirmatory factor analysis has indicated that the EQ may be better

conceived as comprising of this three factor structure rather than a 40 item unifactorial

scale (Lawrence et al., 2004; Muncer and Ling, 2006).

Results and Discussion

The empathic ability of mirror-touch synaesthetes was compared with non-

synaesthetic control participants and controls that report other types of synaesthesia

but do not report mirror-touch synaesthesia. Results from non-synaesthetic controls

and synaesthetes lacking mirror touch did not differ and were therefore combined.

Empathy scores for each component of the EQ are summarised in Figure 4.1. Mirror-

touch synaesthetes showed significantly higher scores on the emotional reactivity

subscale of the EQ relative to controls [t(53) = 2.15, p = < .05]. This subscale is

thought to reflect affective components of empathy, and instinctive empathic

responses to others (Baron-Cohen and Wheelwright, 2004; Lawrence et al., 2004).

There was also a non-significant trend for mirror-touch synaesthetes to outperform

control subjects on the cognitive empathy (i.e. mentalizing / perspective taking)

subscale [t(53) = 1.92, p = .061]. Scores on the social skills subscale did not approach

significance [t(53) = 1.22, p = .227]. Therefore, mirror-touch synaesthetes showed

heightened empathy on some, but not all aspects of empathy. This supports the notion

Chapter 4

83

that empathy is multi-faceted (Baron-Cohen and Wheelwright, 2004; Decety and

Jackson, 2004; Lawrence et al., 2004; Muncer and Ling, 2006; Preston and de Waal,

2002) and implies that sensorimotor simulation may modulate some, but not all,

aspects of this ability. Further, the evidence that enhanced empathy is not found in

other types of synaesthesia suggests that heightened empathy relates specifically to

mirror-touch synaesthesia (and the neural system which underpins this condition).

6

8

10

12

14

16

18

20

22

24

EQ Score

Mirror-Touch Synaesthetes

Controls*

Cognitive Empathy Emotional Reactivity Social Skills

Figure 4.1 Mirror-touch synaesthetes showed significantly higher scores than controls on the emotional reactivity component, but not other components, of the Empathy Quotient (mean ± s.e.m). * = p < .05.

Previous functional magnetic imaging findings indicate that, in healthy adults,

emotional empathy engages the cortical sensorimotor network (including the premotor

cortex, primary somatosensory cortex and motor cortex) more than cognitive empathy

(Nummenmaa, Hirvonen, Parkkola, and Hietanen, 2008). Further,

neuropsychological findings have demonstrated a functional and anatomical double

dissociation between deficits in cognitive empathy and emotional empathy, with

Chapter 4

84

emotional empathy being linked to lesions to the human mirror system and cognitive

empathy being associated to lesions to the ventromedial prefrontal cortices (Shamay-

Tsoory, Aharon-Peretz, and Perry, 2009). This functional coupling between

emotional and cognitive empathy suggests that emotional empathy may be linked

more closely to sensorimotor simulation of another’s state and the evidence that

mirror-touch synaesthetes only significantly differ from controls on levels of

emotional reactivity is consistent with this.

Two qualifications are apposite: (a) it remains unclear whether heightened

emotional reactivity reflects differences in the nature of the individual (i.e. more

emotional or more distressed by emotional scenes as opposed to more empathic /

concerned), and (b) evidence of a borderline significant difference on cognitive

empathy suggests that differences in empathy may not be limited to emotional

empathy per se. To further address these issues a second experiment was conducted.

4.3 Experiment 2: Empathy and personality in mirror-touch synaesthesia

While findings from experiment 1 indicate that mirror-touch synaesthesia is

related to heightened emotional empathy, it remains unclear if the emotional reactivity

component of the EQ reflects the nature of the individual (i.e. more emotional / more

distressed) rather than emotional empathy per se (Muncer and Ling, 2006).

Moreover, the emotional reactivity subscale of the EQ fails to consider the role of

personal distress (an emotionally specific response to one’s own state rather than an

emotional response related to emotional empathy) and thus it is difficult to confirm

whether responses on the emotional reactivity component are indeed other (e.g.

feeling compassion or sorrow towards to another person) rather than self-oriented

(e.g. feeling distress from an unpleasant scene rather than feeling sorrow or concern;

c.f. Batson, 1991; Davis, 1994; Lamm, Batson, and Decety, 2007; Saarela,

Chapter 4

85

Hlushchuk, Williams, Schürmann, Kalso, and Hari, 2007). The implications of this

distinction for behaviour are that other-oriented empathy may promote altruistic

motivations to help another person, while self-orientated empathy may lead to egoistic

motivations to reduce the personal distress felt by the observer and ultimately

counteract positive empathic behaviours (e.g. helping behaviour; Batson, 1991;

Eisenberg, 2000).

This study aimed to investigate this, by contrasting mirror-touch synaesthetes

with age and gender matched non-synaesthetes on the EQ and an additional measure

of empathy - the Interpersonal Reactivity Index (IRI; Davis, 1980). The IRI contains

a component relating to ‘personal distress’ that is regarded as being self-oriented

emotional reactivity and so is able to clarify if heightened emotional reactivity in

mirror-touch synaesthetes is other rather than self-oriented. In addition, the

relationship between individual differences in emotional reactivity (in both the

synaesthetes and controls) and personality traits were investigated using the ‘Big Five

Inventory (BFI; John, Donahue, and Kentle, 1991). If emotional reactivity is self-

oriented then a correlation with the neuroticism (or emotional stability) trait is

expected, but not if it is other-oriented. A previous study found no relationship

between empathy and emotional stability, but it did not divide empathy into

component factors (Del Barrio, Aluja, and Garcia, 2004), thus my study is the first to

consider the importance of personality on different facets of empathy as indexed by

the EQ and IRI.

4.3.1 Methods

Participants

In the first part of the study, sixteen mirror-touch synaesthetes (mean age ±

s.e.m = 35.5 ± 3.58 years) and sixteen age and gender matched non-synaesthete

Chapter 4

86

controls (mean age ± s.e.m = 35.69 ± 3.45 years) were compared. All cases of mirror-

touch synaesthesia were confirmed on the visual-tactile spatial congruity paradigm

designed to provide evidence for the authenticity of the condition (cf. Banissy and

Ward, 2007; Chapter 1 and Chapter 2 this thesis). Of the sixteen synaesthetes, five

took part in experiment 1.

For analysis of the relationship between empathy and personality, an

additional 88 non-synaesthete control subjects were recruited. These controls were

combined with mirror-touch synaesthetes and non-synaesthete controls from the first

part of the study to provide a sample of 120 subjects (mean age ± s.e.m = 27.98 ± 2.78

years). The additional controls were included to provide a range of scores on the

empathy and personality measures, with mirror-touch synaesthetes thought to reflect

the top end of a spectrum on empathy scores.

Materials and procedure

All subjects completed the EQ, IRI, and BFI.

The EQ is a self report measure designed to empirically measure empathy and

is described in experiment 1 of this chapter (pp. 78-79).

The IRI is a 28 item self-report empathy measure (Davis, 1980). It is

comprised of four subscales; perspective taking, fantasizing, empathic concern and

personal distress. Each subscale contains seven items which are measured on a five

point Likert scale ranging from 0 (“Does not describe me well”) to 4 (“Describes me

very well”). For each subscale, a minimum score of 0 or maximum score of 35 is

possible. For the perspective taking, fantasizing and empathic concern subscales

higher scores reflect heightened empathy. For the personal distress subscale higher

scores are reflective of self-orientated emotional reactivity (Davis, 1994).

Chapter 4

87

The BFI is a 44-item scale designed to measure components of the Big Five

personality traits (extraversion, agreeableness, conscientiousness, neuroticism, and

openness; John et al., 1991). Respondents are asked to indicate on a five point Likert

scale the extent to which a series of statements related to each personality trait best

describe their own characteristics. Responses are given from 1 (“Disagree strongly”)

to 5 (“Agree strongly”). Analysis of the reliability of the scale (John and Srivastava,

1999) indicates a coefficient alpha of 0.83 and the BFI shows good convergent

validity with TDA (Goldberg, 1992) and NEO-PI personality measures (Costa and

McCrae, 1985).

4.3.2 Results

Mirror-touch synaesthetes compared to non-synaesthete controls

The empathic abilities of mirror-touch synaesthetes and controls on each

component of the EQ and IRI were compared. On the EQ (Figure 4.2), mirror-touch

synaesthetes scored significantly higher than non-synaesthete controls on the

emotional reactivity subscale [t (30) = 2.29, p = < .05]. Despite synaesthetes scoring

higher than controls on the cognitive empathy (CE) and social skills (SS) subscales

these did not approach significance [CE: t(30) = 1.20, p = .239; SS: t(30) = 1.37, p =

.181]. This is consistent with findings from experiment 1 where mirror-touch

synaesthetes differed significantly from controls on the emotional reactivity subscale,

but not cognitive empathy or social skills subscale of the EQ.

Chapter 4

88

4

6

8

10

12

14

16

18

20

22

EQ Score

Mirror-Touch Synaesthetes

Controls

*

Cognitive Empathy Emotional Reactivity Social Skills

Figure 4.2 In experiment 2, mirror-touch synaesthetes showed significantly higher scores than controls on the emotional reactivity component, but not other components, of the Empathy Quotient (mean ± s.e.m). * = p < .05.

In addition, on the IRI mirror-touch synaesthetes showed significantly

elevated scores on the fantasizing subscale [t(30) = 2.35, p = < .05], but not on the

alternative subscales (Figure 4.3). Of note, is the comparable performance of

synaesthetes and controls on the perspective taking subscale and higher scores of

controls relative to synaesthetes on the personal distress subscale, which indicates that

differences were not simply due to a tendency for synaesthetes to provided higher

self-reported values overall (also see Figure 4.4).

Chapter 4

89

8

10

12

14

16

18

20

22

24

26

28

IRI Score

Mirror-Touch Synaesthetes

Controls*

PT F PD EC

Figure 4.3 Mirror-touch synaesthetes showed significantly higher scores than controls on the fantasizing component, but not other components, of the Interpersonal Reactivity Index (mean ± s.e.m). * = p < .05. PT = Perspective Taking, F = Fantasizing, PD = Personal Distress, EC = Empathic Concern.

18

20

22

24

26

28

30

32

34

36

38

40

42

44

BFI Score

Mirror-Touch Synaesthetes

Controls

*

O A C N E

Figure 4.4 Mirror-touch synaesthetes showed significantly higher scores than controls on the Openness subscale, but not other components, of the BFI (mean ± s.e.m). * = p < .05. E = Extraversion, A = Agreeableness, C = Conscientiousness, N = Neuroticism, O = Openness.

Chapter 4

90

Synaesthetes also differed from controls on the openness to experience trait of

the BFI [t(30) = 2.62, p = .014], with synaesthetes demonstrating higher scores on this

personality trait. No significant differences were found on the other BFI subscales

(Figure 4.4).

Empathy and personality

Pearson’s correlations were conducted to investigate the relationship between

five components of personality (extraversion, agreeableness, conscientiousness,

neuroticism and openness) with empathic abilities on each component of the EQ

(cognitive empathy, emotional reactivity, social skills). This analysis revealed that

the cognitive empathy subscale of the EQ showed a moderate correlation with

extraversion [n = 120, r = .401, p = < .001], agreeableness [n = 120, r = .463, p = <

.001], and conscientiousness [n = 120, r = .383, p = < .001]; but not neuroticism or

openness traits. Similar associations were observed for the emotional reactivity

subscale, which was associated with extraversion [n = 120, r = .292, p = < .01],

agreeableness [n = 120, r = .455, p = < .001], conscientiousness [n = 120, r = .248, p

= < .01] and openness [n = 120, r = .180, p = < .05]. Importantly, emotional reactivity

was not correlated with neuroticism / emotional stability [n = 120, r = .016, p = .862].

The social skills subscale displayed a positive relationship with extraversion [n = 120,

r = .337, p = < .001], agreeableness [n = 120, r = .336, p = < .001], and

conscientiousness [n = 120, r = .324, p = < .001]; a negative relationship with

neuroticism [n = 120, r = -.311, p = < .01]; but no relationship with openness.

Additionally, the relationship between IRI scores and personality were

explored. This revealed a positive association between the perspective taking

subscale and agreeableness [n = 120, r = .237, p = < .01]; the fantasizing subscale and

openness [n = 120, r = .319, p = < .001]; the empathic concern subscale and

Chapter 4

91

extraversion [n = 120, r = .185, p = < .05], agreeableness [n = 120, r = .524, p = <

.001], and conscientiousness [n = 120, r = .201, p = < .05]. The personal distress

subscale was found to correlate negatively with extraversion [n = 120, r = -.256, p = <

.01] and conscientiousness [n = 120, r = -.283, p = < .01], but positively with the

neuroticism trait [n = 120, r = .532, p = < .001].

4.4 Discussion

Experiment 2 sought to examine findings of heightened emotional reactivity in

mirror-touch synaesthesia documented in experiment 1. The study aimed to clarify if

enhanced emotional reactivity was specific to enhanced other-orientated emotional

empathy or to self related processes, and to investigate previous trends towards

significantly higher levels of cognitive empathy in mirror-touch synaesthetes. Using a

larger sample of mirror-touch synaesthetes and a new control group, the findings first

replicate previous reports of heightened emotional reactivity, but not other

components of empathy, in mirror-touch synaesthesia. They then confirm that

heightened emotional reactivity in mirror-touch synaesthesia is not linked with

heightened personal distress, indicating that heightened empathy in mirror-touch

synaesthesia is indeed other rather than self-oriented. Further, the findings indicate

that emotional reactivity does not reflect a less emotionally stable personality type –

scores on emotional reactivity component of the EQ were not related to neuroticism /

emotional stability personality trait.

Mirror-touch synaesthetes also differed from non-synaesthete controls on the

fantasizing scale of the IRI. This subscale reflects an individual’s ability to match

another’s feelings and behaviours onto their own. Previous findings indicate that

increased scores on the fantasizing subscale of the IRI are related with heightened

activations within the anterior insula and frontal operculum when witnessing others’

Chapter 4

92

gustatory emotions (Jabbi et al., 2006). In a previous brain imaging study on mirror-

touch synaesthesia, the only brain region to differ between mirror-touch synaesthetes

and non-synaesthetes when observing touch to others was the anterior insula

(Blakemore et al., 2005). Shared representations within this brain region may be

important for this component of empathy.

Finally, analysis of correlations between personality and measures of empathy

provide important insights into the EQ and IRI measures. Evidence that heightened

extraversion, agreeableness and conscientiousness are related with higher responses

on all components of the EQ is consistent with findings implicating these personality

traits in social cognition more generally; extraversion has been suggested to be a

measure of social skills (John and Srivastava, 1999); agreeableness has been linked to

altruistic behaviour (Barrick and Mount, 1991); and conscientiousness correlates

negatively with psychoticism (Aluja, Garcia, and Garcia, 2002). Of note, is the lack

of association between emotional reactivity and neuroticism, but presence of a strong

positive correlation between the personal distress subscale of the IRI and neuroticism.

This is consistent with the notion that the neuroticism personality trait would indicate

self rather than other-oriented processes when correlated with empathy. Further to

this, the openness trait appears specifically related to one’s ability to match another’s

emotional state with one’s own (as indicated by the positive relationship between the

openness subscale and emotional reactivity subscale of the EQ; and between the

openness subscale and fantasizing subscale of the IRI) and this was the only

personality trait where synaesthetes significantly differed from controls. Notably, it is

difficult to determine the nature of the higher levels of openness to experience

observed in mirror-touch synaesthetes relative to controls because the mirror-touch

group included self-referred cases who have already demonstrated openness to

Chapter 4

93

experience by contacting unknown researchers for a study. Therefore, it is difficult to

determine whether differences in the levels of openness to experience would extend to

larger randomly recruited populations of mirror-touch synaesthetes (i.e. whether

higher levels of openness to experience are more prominent in self-referred rather

than randomly sampled populations of synaesthetes).

General Summary

In sum, the studies presented in this chapter indicate that mirror-touch

synaesthetes show heightened levels of emotional, but not other components, of

empathy. In experiment 1, mirror-touch synaesthetes scored significantly higher on

emotional reactivity components of empathy, but not on social skills or cognitive

empathy. A control group of synaesthetes who experience other types of synaesthesia

but not mirror-touch did not differ from non-synaesthete controls, indicating that

differences in empathy were specific to this subtype of synaesthete. Experiment 2

extended findings in experiment 1 to demonstrate that the heightened emotional

reactivity observed in mirror-touch synaesthetes reflects other, rather than self-

orientated, emotional reactions. Mirror-touch synaesthetes were also shown to differ

from non-synaesthetes on an alternative measure to that used in experiment 1. On

both measures synaesthetes showed heightened affective empathy (but not cognitive

empathy), implying that sensorimotor simulation is important for some, but not all

components of empathy. Given that mirror-touch synaesthesia has been linked to

heightened sensorimotor simulation (Blakemore et al., 2005), these findings appear

consistent with accounts of empathy that posit a role for sensorimotor simulation

mechanisms (Gallese, 2006; Gallese and Goldman, 1998; Keysers and Gazzola, 2006;

Oberman and Ramachandran, 2007) and are consistent with functional brain imaging

Chapter 4

94

(Nummenmaa et al., 2008) and neuropsychological findings (Shamay-Tsoory et al.,

2009) which indicate that emotional empathy may be linked more closely to

sensorimotor simulation of another’s state.

Chapter 5

95

CHAPTER 5: FACIAL EXPRESSION RECOGNITION IN MIRROR-TOUCH SYNAESTHESIA

The findings from chapter 4 indicate that mirror-touch synaesthesia is linked with

heightened affective empathy. Simulation models of expression recognition contend

that in order to understand another’s facial expressions individuals map the

perceived expression onto the same sensorimotor representations which are active

during the experience of the perceived emotion. To investigate this view, the present

study examines facial expression and identity recognition abilities in mirror-touch

synaesthesia. Mirror-touch synaesthetes outperformed non-synaesthetic controls on

measures of facial expression recognition, but not on control measures of face

memory or face perception. These findings imply a role for sensorimotor simulation

in the recognition of facial affect, but not facial identity.

5.1 Introduction

The ability to perceive a face is one of the most highly developed visual skills

in humans, important not only for our ability to recognise the identity of others but

also to facilitate social interaction. Neurocognitive models of face perception

highlight the role of a number of face-specific and domain-general mechanisms in this

process, and distinguish between those involved in the recognition of facial identity

and those involved in the recognition of expressions at different stages of cortical

processing (Bruce and Young, 1986; Calder and Young, 2005; Haxby, Hoffman, and

Gobbini, 2000).

Simulation accounts of expression recognition contend that to understand

another’s facial emotion the observer simulates the sensorimotor response associated

with generating the perceived facial expression (Adolphs, 2002; Adolphs, 2003;

Gallese, Keysers, and Rizzolatti, 2004; Goldman, and Sripada, 2005; Keysers and

Gazzola, 2006). This is supported by evidence that electromyographic responses in

expression relevant facial muscles are increased during subliminal exposure to

emotional expressions (Dimberg, Thunberg, and Elmehed, 2000); that preventing the

Chapter 5

96

activation of expression relevant muscles impairs expression recognition (Oberman,

Winkielman, and Ramachandran, 2007); and that perceiving another person’s facial

expressions recruits similar premotor and somatosensory representations as when the

perceiver generates the same emotion or expression (Carr, Iacoboni, Dubeau,

Mazziotta, and Lenzi, 2003; Hennenlotter et al., 2005; Montgomery and Haxby, 2008;

van der Gaag, Minderaa, and Keysers, 2007; Winston, O’Doherty, and Dolan, 2003).

Further, neuropsychological findings indicate that focal brain damage to right

somatosensory cortices is associated with expression recognition deficits (Adolphs,

Damasio, Tranel, Cooper, and Damasio, 2000), and transcranial magnetic stimulation

findings demonstrate the necessity of the right somatosensory cortex for facial

expression recognition abilities in healthy adults but not face identity recognition

(Pitcher, Garrido, Walsh and Duchaine, 2008). These findings imply that purely

visual face-processing mechanisms interact with sensorimotor representations to

facilitate expression recognition. This thought to differ to facial identity recognition,

in which there is no clear indication of how one could simulate another’s identity

(Calder and Young, 2005).

While much has been learnt from studies involving a disruption of simulation

mechanisms, an alternative approach is to consider whether facilitation of these

mechanisms promotes expression recognition. One example of facilitated

sensorimotor simulation is the case of mirror-touch synaesthesia (Blakemore, Bristow,

Bird, Frith, and Ward, 2005). As noted previously, in mirror-touch synaesthesia,

simply observing touch to others elicits a conscious tactile sensation on the

synaesthete’s own body. Functional brain imaging indicates that this variant of

synaesthesia is linked to heightened neural activity in a network of brain regions

which are also activated in non-synaesthetic control subjects when observing touch to

Chapter 5

97

others (the mirror-touch system; Blakemore et al., 2005). The mirror-touch system is

comprised of brain areas active during both the observation and passive experience of

touch (including primary and secondary somatosensory cortices, and premotor cortex;

Blakemore et al., 2005; Ebisch, Perucci, Ferretti, Del Gratta, Luca Romani, and

Gallese, 2008; Keysers, Wicker, Gazzola, Anton, Fogassi, and Gallese, 2004). It has

been suggested that brain systems with mirror properties may be crucial for social

perception because they provide a probable neural mechanism to facilitate

sensorimotor simulation of another’s perceived state (Gallese, Keysers, and Rizzolatti,

2004; Keysers and Gazzola, 2006). In this sense, mirror-touch synaesthesia can be

viewed as a case of heightened sensorimotor simulation, which may be able to inform

on the role of sensorimotor simulation mechanisms in social cognition. Consistent

with this, in chapter 4 I report that mirror-touch synaesthetes show heightened

emotional empathy compared to control subjects. Enhanced empathy was not found in

other types of synaesthesia, suggesting that it relates specifically to this variety of

synaesthesia.

This study sought to establish whether this type of synaesthetes differed in

another aspect of social perception, namely facial expression recognition. To do so,

the performance of mirror-touch synaesthetes and non-synaesthetic control subjects

on tasks of facial expression recognition, identity recognition and identity perception

were compared. Based upon the hypothesis that mirror-touch synaesthetes have

heightened sensorimotor simulations mechanisms it was predicted that synaesthetes

would show superior performance on expression recognition tasks due to heightened

sensorimotor simulation mechanisms, but not on the identity recognition or face

perception control tasks that are less dependent on simulation.

Chapter 5

98

5.2 Method

Participants

Eight mirror-touch synaesthetes (6 female and 2 male; mean age ± s.d = 45.6 ±

11.7 years) and twenty non-synaesthetic control subjects (15 female and 5 male; mean

age ± s.d = 35.6 ± 13.6 years) took part in the study. All cases of mirror-touch

synaesthesia were confirmed using a previously developed visual-tactile congruity

paradigm designed to provide evidence for the authenticity of the condition (Banissy

and Ward, 2007; also see Chapters 1 and 2 of this thesis for a description of the task

used).

Materials and Procedure

Participants completed four tasks in a counterbalanced order. These tasks are

detailed below.

Films Facial Expression Recognition

This task investigated participants’ abilities to recognize the emotional

expressions of others. In each trial participants were presented with an adjective

describing an emotional state followed by three images (each image shown for 500

msec, with a 500msec ISI) of the same actor or actress displaying different facial

expressions. Participants were asked to indicate which of the three images best

portrayed the target emotional adjective. There was no fixed inter-trial interval as

participants began each trial with a key press (i.e. the task was self-paced).

In order to portray subtle facial expressions, expression stimuli were captured

from films (Figure 5.1a). Fifty-eight target images (preceded by three practice trials)

from 15 films were used. All films were from a non-English speaking country to

decrease the probability that participants had seen them or were familiar with the

Chapter 5

99

actors. Target and distracter stimuli were selected based on four pilot studies (see

Garrido et al., in press for description). Each stimulus was shown once during the test

and trials were presented in a pseudo-random order over two blocks (29 trials per

block).

Cambridge Face Memory Test Long Form (CFMT+)

To test face recognition the performance of synaesthetes and non-synaesthetes

on the CFMT+ was compared (Russell, Duchaine, and Nakayama, 2009). The task is

an adapted version of the Cambridge Face Memory Test (Duchaine and Nakayama,

2006; Figure 5.1b) and was designed to distinguish normal from super-normal ability

at recognising faces (Russell et al., 2009). During the task subjects learn to recognize

six unfamiliar male faces from three different views (left 1/3 profile, frontal, right 1/3

profile) and are then tested on their ability to recognise these faces in a three-

alternative forced-choice task. The test is comprised of four sections, each more

difficult than the previous. The first three sections are taken from the original CFMT

(Duchaine and Nakayama, 2006) and the final section forms the longer CFMT+

(Russell et al., 2009).

The test begins by testing recognition with the same images that were used

during training (i.e. participants are are asked to memorise an unfamiliar male face

from three different views and are then tested on their memory for the trained face).

In the first phase each training image is shown for three seconds and immediately

followed by three trials per face, resulting in eighteen trials overall. This relatively

easy introduction is followed by a further training phase in which participants are

shown six frontal views of the faces for twenty seconds. The participants’ memory

for each face is then assessed using novel images that show the target faces from un-

trained views and lighting conditions (thirty trials). Following a further twenty

Chapter 5

100

second training phase in which participants are shown six frontal views of the face, a

third section consisting of novel images with visual noise added (twenty-four trials) is

completed. In the CFMT+, this is followed by a final section containing thirty very

difficult trials in which distracter images repeat much more frequently, targets and

distracters contain more visual noise than the images in the third section, cropped (i.e.

only showing internal features) and uncropped images (i.e. showing hair, ears, and

necks) are used, and images showing the targets and distracters making emotional

expressions are included. An inter-trial interval of 1 second is used throughout. The

percentage of correct responses for each section and overall are measured. Feedback

is not provided during the test.

Cambridge Face Perception Test (CFPT)

To investigate face perception the CFPT was administered (Duchaine,

Germine, and Nakayama, 2007). This test assesses the ability to perceive differences

between facial identities. Memory demands are minimal because faces are presented

simultaneously. During the task, subjects are shown a target face (from a ¾

viewpoint) and six faces (from a frontal view) morphed between the target and

distracter in varying proportions (88%, 76%, 64%, 52%, 40%, 28%) so that they vary

systematically in their similarity to the target face (Figure 5.1c). Subjects are asked to

sort the six faces by similarity to the target face and are given one minute to do so.

Participants sorted the faces by clicking on the face which they wished to move and

then indicating where the face should be by clicking in the area between two faces.

The desired face was then moved to the chosen location by the program. At the end

of each trial participants then clicked an option on screen to begin the next trial (i.e.

the task was self-paced). The task involves eight upright and eight inverted trials that

alternate in a fixed pseudo-random order. This allows investigation of the inversion

Chapter 5

101

effect for face perception. Performance is measured by an error score. This is

calculated by summing the deviations from the correct position for each face, with

one error reflecting each position that a face must be moved in order to be in the

correct location. For example, if a face was one position from the correct location

than this leads to an error score of one. If it is three positions away this is an error

score of three.

Same-Different Expression and Identity Matching Task

This experiment investigated participants’ abilities to match another’s facial

identity or facial expressions under identical experimental conditions.

In the expression matching task, participants were presented with a “sample”

face (250 msec) followed by a fixation cross (1000 msec), and a “target” face (250

msec). Participants were asked to indicate whether the target facial expression

matched, or was different to, the sample facial expression. On half of the trials, the

target and sample face expressed the same emotion and half the sample-target pairs

showed different emotions (Figure 5.1d). A total of 72 trials (split between 2 blocks)

were completed. Each image showed one of six female models making one of six

basic facial expressions: anger, disgust, fear, happiness, sadness or surprise. Each

stimulus was a greyscale image taken from the Ekman and Friesen (1976) facial affect

series. Stimuli were cropped with the same contour to cover the hair and neck using

Adobe Photoshop. In the expression task, identity always changed between sample-

target pairs and each expression was presented an equal number of times.

In the identity matching task, the same stimuli and procedure were used.

Participants were asked to indicate whether the sample and target face were the same

or a different person. Half of the trials showed pairs with the same identity and half

Chapter 5

102

with a different identity. Expression always changed between the sample and target

face, and the six models were presented an equal number of times.

Figure 5.1 Summary of the tasks used. (a) Films Facial Expression Task. This task investigated participants’ abilities to categorize the emotional expressions of others. Participants were presented with a target adjective describing an emotional state followed by three images shown consecutively for 500 msec each. Participants were asked which of the three images best portrayed the target emotion. In the actual task colour stimuli were used. (b) Cambridge Face Memory Test Long Form. This task investigated participants’ abilities to memorize facial identity and was derived from the Cambridge Face Memory Test (shown in figure). During the task participants memorized six unfamiliar male faces. They were then tested on their ability to recognize the faces in a three-alternative-forced-choice paradigm. The task involves four sections (for stimuli from the final section see Russell et al., 2009), each more difficult than the preceding section. (c) Cambridge Face Perception Test. This task investigated participants’ abilities to perceive faces in the absence of memory. Participants were shown a target face and six faces morphed between the target and a distracter face. Participants sorted the six faces by similarity to the target face. Faces were presented upright and inverted in a fixed pseudo-random order. (d) Same-Different Expression-Identity Matching Task. This task investigated participants’ abilities to match another’s facial identity or facial expressions. Participants were presented with a sample face followed by a fixation cross, and then a target face. In the expression matching task participants indicated whether the expression in the target face matched the expression in the sample face. In the identity matching task participants indicated whether the identity of the target face and the prime face matched.

Chapter 5

103

5.3 Results

Films Facial Expression Recognition

Accuracy and reaction time performance were compared separately using a

one way between subjects ANCOVA. Participant age was used a covariate on all

analyses because of slight trend for synaesthetes to differ from controls on age [t26 =

1.84, p = .078]. One control subject was withdrawn from analysis due to difficulties in

understanding the meaning of expression adjectives and performing more than three

standard deviations below the control group mean on accuracy and reaction time

measures.

Synaesthetes showed superior abilities at recognizing the emotional

expressions of others (Figure 5.2). Analysis of accuracy performance revealed that

mirror-touch synaesthetes outperformed control subjects on expression recognition

[F1,24 = 16.38, p = < .001] (Figure 5.2a). This difference was not due to a speed-

accuracy trade off as no significant effect of group (synaesthete or control) was found

for reaction time performance, and in fact synaesthetes tended to perform faster than

controls [F1,24 = .962, p = .336] (Figure 5.2b). These findings suggest that mirror-

touch synaesthetes show superior facial expression recognition, which may be due to

heightened sensorimotor simulation mechanisms.

C

hapt

er 5

104

Fi

gure

5.2

Mea

n ac

cura

cy (

a) a

nd r

eact

ion

time

perf

orm

ance

s (b

) of

syn

aest

hete

s an

d co

ntro

ls o

n th

e fi

lms

faci

al e

xpre

ssio

n ta

sk (

± s.

e.m

).

Syna

esth

etes

wer

e si

gnif

ican

tly m

ore

accu

rate

at

cate

goriz

ing

the

emot

iona

l fa

cial

exp

ress

ions

of

othe

rs (

a).

Thi

s w

as n

ot d

ue t

o a

spee

d ac

cura

cy tr

ade

off

as n

o si

gnif

ican

t diff

eren

ces

wer

e fo

und

for

reac

tion

time

and

syna

esth

etes

sho

wed

a tr

end

to b

e fa

ster

than

con

trol s

ubje

cts

(b).

75

80

85

90

95

Percentage Correct

Synaesthetes

Controls

*a

400

500

600

700

800

Reaction Time (msec)

Synaesthete

Control

b

C

hapt

er 5

105

Figu

re 5

.3 M

ean

accu

racy

per

form

ance

s (±

s.e

.m)

on t

he C

FMT

(a)

and

CFM

T+

(b)

for

syna

esth

etes

and

con

trols

. T

he p

erfo

rman

ce o

f sy

naes

thet

es a

nd c

ontr

ols

did

not s

igni

fica

ntly

dif

fer o

n th

e C

FMT

(fir

st th

ree

sect

ions

of t

ask)

or C

FMT+

.

70

75

80

85

Percentage Correct

Synaesthete

Controls

a

60

65

70

75

Percentage Correct

Synaesthete

Control

b

Chapter 5

106

Cambridge Face Memory Test Long Form

Accuracy performance from the Cambridge Face Memory Test (first three

sections) and Cambridge Face Memory Test Long Form are shown in figure 5.3. No

significant differences were observed between synaesthetes and controls on either the

CFMT [F1,25 = .023, p = .880] (Figure 5.3a) or the CFMT + [F1,25 = .095, p = .761]

(Figure 5.3b). Therefore unlike facial expression recognition, synaesthetes and

controls did not differ in their ability to memorize facial identity.

Cambridge Face Perception Test

Error scores on the eight upright and eight inverted trials were summed to

determine the total number of upright and inverted errors. A 2 (Group) x 2 (Trial

Type) ANCOVA revealed a significant effect of trial type [F1,25 = 5.81, p = .024]

which was due to an inversion effect, whereby overall participants were less accurate

on inverted (mean ± s.e.m = 70 ± 3) compared with upright trials (mean ± s.e.m =

41.5 ± 3.21). Importantly, this effect did not interact with group [F1,25 = .37, p = .549]

and no main effect of group was found [F1,25 = .253, p = .619] (Figure 5.4).

Therefore, unlike expression recognition, synaesthetes and controls did not

significantly differ in their abilities to detect another’s facial identity.

C

hapt

er 5

107

Figu

re 5

.4 M

ean

erro

r sco

re (±

s.e

.m) f

or s

ynae

sthe

tes

and

cont

rols

on

upri

ght (

a) a

nd in

verte

d (b

) tria

ls o

f the

CFP

T.

No

sign

ific

ant d

iffe

renc

es

wer

e fo

und

betw

een

the

perf

orm

ance

of

syna

esth

etes

and

con

trol

s on

a m

easu

re o

f fa

cial

ide

ntity

per

cept

ion.

Er

ror

scor

es a

re b

ased

on

the

devi

atio

n fr

om th

e ex

pect

ed c

orre

ct o

rder

ing

of th

e co

rrec

t loc

atio

n of

the

targ

et fa

ce.

Supe

rior p

erfo

rman

ce is

refl

ecte

d in

a lo

wer

err

or s

core

.

30

35

40

45

Upright Errors

Synaesthete

Control

a

65

70

75

80

Inverted Errors

Synaesthete

Control

b

C

hapt

er 5

108

Figu

re 5

.5 M

ean

accu

racy

per

form

ance

s (±

s.e

.m) o

n th

e ex

pres

sion

mat

chin

g (a

) and

iden

tity

mat

chin

g (b

) tas

k fo

r syn

aest

hete

s an

d co

ntro

ls.

A

2 (T

ask)

x 2

(G

roup

) A

NC

OV

A r

evea

led

a si

gnif

ican

t ta

sk x

gro

up i

nter

actio

n. O

n th

e ex

pres

sion

tas

k, t

here

was

a t

rend

for

syn

aest

hete

s to

ou

tper

form

con

trols

whe

reas

on

the

iden

tity

task

, the

re w

as a

tren

d fo

r con

trols

to o

utpe

rfor

m s

ynae

sthe

tes.

With

in-g

roup

com

pari

sons

bet

wee

n th

e ta

sks

reve

aled

that

con

trols

wer

e si

gnif

ican

tly m

ore

accu

rate

in th

e id

entit

y co

mpa

red

to th

e ex

pres

sion

task

. Sy

naes

thet

es d

id n

ot s

how

this

bi

as to

war

ds id

entit

y m

atch

ing

- exp

ress

ion

and

iden

tity

mat

chin

g pe

rfor

man

ces

wer

e co

mpa

rabl

e.

70

75

80

85

Percentage Correct

Synaesthete

Controls

aExpression matching

75

80

85

90

Percentage Correct

Synaesthete

Control

bIdentity matching

Chapter 5

109

Same-Different Expression and Identity Matching Task

A 2 (Group) x 2 (Task) mixed ANCOVA was conducted. Participant age was

used as a covariate. No main effect of task or group was found. No relationship

between task and age was observed. There was however a significant interaction

between task and group [F1,25 = 4.507, p = .044]. Controls were more accurate, and

therefore showed an advantage, on the identity matching task relative to the emotion

matching task [F1,18 = 5.10, p = .037]. Synaesthetes did not show this pattern -

analysis of within-subject effects revealed no significant difference between the two

tasks for the synaesthetic group [F1,6 = .759, p = .417]. There was also a non-

significant trend for synaesthetes to outperform controls on the expression matching

task (Figure 5.5a), but for controls to outperform synaesthetes on the identity

matching task (Figure 5.5b).

5.4 GENERAL DISCUSSION

This study investigated expression and identity face processing in mirror-touch

synaesthetes and non-synaesthete control participants. It was predicted that

heightened sensorimotor simulation mechanisms would result in superior expression

recognition, but would not affect the identity recognition abilities of mirror-touch

synaesthetes. Consistent with these predictions, mirror-touch synaesthetes were

superior when recognizing the facial expressions, but not facial identities of others.

These findings are consistent with simulation accounts of expression recognition

which suggest that in order to understand another’s emotional expressions individuals

must simulate the sensorimotor response associated with generating the perceived

facial expression (Adolphs, 2002; Adolphs, 2003; Gallese, Keysers, and Rizzolatti, G,

2004; Goldman, and Sripada, 2005; Keysers and Gazzola, 2006).

Chapter 5

110

A variety of sources indicate that recognizing another’s identity and

expressions relies upon multiple stages of representation, including purely visual,

multimodal, expression-general and expression-specific mechanisms (e.g. Adolphs et

al., 2000; Anderson, Spencer, Fulbright, and Phelps, 2000; Calder, Lawrence, and

Young, 2001; Calder, Keane, Lawrence, and Manes, 2004; Lawrence, Calder,

McGowan, and Grasby, 2002; Lewis et al., 2003; Keane, Calder, Hodges, and Young,

2002; Pitcher et al., 2008; Sprengelmeyer et al., 1996). Simulation accounts of

expression recognition contend that one mechanism involved in expression, but not

identity, recognition is an internal sensorimotor re-enactment of the perceived

expression (Adolphs, 2002; Adolphs, 2003; Gallese, Keysers, and Rizzolatti, 2004;

Goldman, and Sripada, 2005; Keysers and Gazzola, 2006). Functional brain imaging

(Carr et al., 2003; Hennenlotter et al., 2005; Montgomery and Haxby, 2008; van der

Gaag, et al., 2007; Winston et al., 2003), neuropsychological (Adolphs et al., 2000),

and transcranial magnetic stimulation studies (Pitcher et al., 2008) suggest a key role

for somatosensory resources in expression recognition. The findings that individuals

who show increased levels of somatosensory simulation (mirror-touch synaesthetes)

demonstrate superior expression, but not identity perception, are consistent with this

view. The task specific nature of the findings also indicate that the superior

performance shown by mirror-touch synaesthetes on the expression recognition tasks

are not linked to heightened motivation on the part of the synaesthetic subjects.

The experiments in chapter 4 documented that mirror-touch synaesthetes show

heightened emotional reactivity compared to controls, but do not differ on other

components of empathy. The findings from the current investigation indicate that

mirror-touch synaesthesia is not only linked with some components of empathy, but

also with superior emotion recognition. It remains to be established whether

Chapter 5

111

heightened emotion sensitivity displayed by mirror-touch synaesthetes is a cause or

consequence of this type of synaesthetic experience. While it is assumed that mirror-

touch synaesthetes form part of the synaesthetic population, and are therefore a unique

group of subjects, the principles which bias what type of synaesthesia will or will not

be developed are a matter of debate (Bargary and Mitchell, 2008; Cohen Kadosh,

Henik, and Walsh, 2009; Cohen Kadosh and Walsh, 2008; Grossenbacher and

Lovelace, 2001; Hubbard and Ramachandran, 2005; Ramachandran and Hubbard,

2001; Rouw and Scholte, 2007; Sagiv and Ward, 2006). Conceptually there are two

possibilities: i) mirror-touch synaesthetes reflect the top end of a spectrum along

which emotion sensitivity ranges (e.g. the ‘normal’ architecture for multi-sensory

interactions) and this biases them towards interpersonal synaesthetic experience, or ii)

mirror-touch synaesthetes are a unique population whose extra sensory experiences

predispose superior emotion sensitivity.

In sum, this study demonstrates that mirror-touch synaesthesia is associated

with superior facial expression recognition abilities. The observed superiority in face

processing is restricted to expression recognition. Mirror-touch synaesthetes show

enhanced emotional expression abilities. They did not differ from controls on identity

perception measures. Given that mirror-touch synaesthesia has been linked to

heightened somatosensory simulation these findings are consistent with simulation

based accounts of expression recognition and indicate that somatosensory resources

are an important facet in our ability to recognise the emotions of others.

Chapter 6

112

CHAPTER 6: METHODOLOGICAL INTRODUCTION TO TMS

This chapter outlines the methodological principles for using transcranial magnetic

stimulation (TMS) to disrupt normal cognitive functioning. The main principles,

ethical considerations, spatial and temporal constraints, and types of TMS are

discussed. In chapters 7 and 8 continuous theta burst TMS was performed to

investigate the role of sensorimotor simulation in expression recognition. This TMS

paradigm is introduced here and it’s spatial and temporal effects are discussed.

6.1 INTRODUCTION

Transcranial magnetic stimulation (TMS) is a non-invasive experimental

technique that is capable of suppressing or facilitating activity in the brain. The effect

of this modulation of neural activity can be measured using a variety of standard

behavioural (e.g. reaction time, accuracy, thresholds) and physiological (e.g. evoked

potentials, functional brain imaging) measures; is temporally discrete (c.f. Walsh and

Cowey, 2000); and shows good spatial specificity (e.g. Pitcher, Charles, Devlin,

Walsh, and Duchaine, 2009). The technique provides a unique tool to the cognitive

neuroscientist because it permits the opportunity to assess the causality of a particular

brain region to a given cognitive task. For example, one can use TMS to disorganize

neural activity in a given brain region and investigate the effects of this disruption on

functionally specific cognitive tasks (e.g. motion priming and human V5 / MT

stimulation – Campana, Cowey, and Walsh, 2002; face processing and Occipital Face

Area stimulation – Pitcher, Walsh, Yovel, and Duchaine, 2007). In this sense, TMS is

similar to both human (e.g. Milner, 1966; Shallice, 1988) and animal (e.g. Cowey and

Gross, 1970; Walsh and Butler, 1996) lesion studies in which one is able to make

inferences about the necessity of specific brain areas based upon impairment to

cognitive functioning.

Chapter 6

113

There are also important differences between lesion and TMS studies, which

lead to a number of benefits over patient-based research. Firstly, the nature of lesion

studies means that the experimenter is required to make inferences about the normal

architecture based on an abnormal system. This has a number of caveats including the

influence of mechanisms of compensatory plasticity in the abnormal system, which

may lead to changes in function or task performance. For example, it is often months

after brain injury that the experimenter is able to examine patient performance

systematically and it is difficult to disentangle whether one is measuring the removal

of a region or the ability of other brain regions to compensate the function being

investigated (Lomber, 1999; Robertson and Murre, 1999). Further, brain lesions are

rarely spatially discrete and removal of a brain area may also incur damage at distal

sites (e.g. via severed vessels, ablated white matter; Robertson and Murre, 1999;

Walsh and Pascual-Leone, 2003). In contrast to this, TMS permits investigation of

spatially specific brain regions in normal subjects and overcomes problems of neural

compensation / reorganization because the main effects occur in a discrete temporal

window (lasting a few tens of milliseconds to minutes depending on the type of

stimulation used). In addition, because behavioural performance can be measured

within-subjects (during both the application and the absence of stimulation), it is

possible for subjects to act as their own control group, thereby strengthening the

validity of the conclusions that can be drawn from a TMS experiment.

To consider the method further, this chapter discusses the principles, ethical

aspects, spatial resolution, temporal constraints, and alternative TMS paradigms.

Chapter 6

114

6.2 What is TMS?

Attempts to modulate human brain function using magnetic fields can be

traced back to the late 19th century and developed over the next 100 years (c.f. Walsh

and Pascual-Leone, 2003). However, these attempts rarely systematically measured

the effects of magnetic stimulation and it was not until the 1980s that TMS (as we

know it) was first introduced as a tool for cognitive neuroscience - Barker and

colleagues (1985) reported the first successful attempts to disrupt normal cortical

functioning when they applied magnetic stimulation over the motor cortex in human

subjects and recorded the resulting muscle twitches via motor evoked potentials

(MEPs).

The technique builds upon Faraday’s observations of electromagnetic

induction (1831), in which electric current passed along one wire coil (coil A)

generates a magnetic field that induces electrical current in another wire coil (coil B).

In TMS, rapidly changing electrical current is passed through a coil (i.e. coil A in

Faraday example) located on the participant’s scalp to generate a brief magnetic field

which passes through the skull and induces electrical fields in the underlying cortex

(i.e. coil B in Faraday example). The induced current alters the electrical state inside

and outside of nerve axons, leading to membrane depolarisation and the initiation of

action potentials (Nagarjan, Durand, and Warman, 1993). Thus, the electrical field

induced by the TMS coil causes changes in the resting potential of the underlying

neurons and effectively disorganises neural processing in a stimulated cortical region.

The duration of this disorganisation of neural processing is linked to the size

of the induced current, which is related to the amplitude and the rate of change of the

current passing through the TMS coil. Typically the current in the coil is large, up to 8

kiloamperes (kA), with a swift rise time of roughly 200 milliseconds (ms) and an

Chapter 6

115

overall duration of roughly 1 ms (Figure 6.1). The extent to which the resulting TMS

pulse disrupts neural processing in the targeted area depends on the orientation of the

coil and the orientation of the underlying nerve fibres (Amassian, Eberle, Maccabee,

and Cracco, 1992). If the induced field is uniform across the cell membrane then no

current will be induced. The TMS effects are optimised when the electric field is

tangential to the orientation of the nerve fibre either due to the electric field

orientation being perpendicular to a straight axon or an axon bending relative to the

orientation of the induced field (Figure 6.2).

Two types of coils are typically used in TMS studies. They are the figure-of-

eight and circular coils (Figure 6.3). All of the studies reported in this thesis use a

figure-of-eight coil, which has been shown to produce the most focal effects of TMS

(Ueno, Tashiro, and Harada, 1988). In the figure-of-eight coil, current flows in

opposite directions around each of the windings and converges on the centre point of

the coil where the electrical currents summate. This leads to focal neural stimulation

with the largest effect occurring in the cortex situated directly under the centre-point

of the coil. Because the outer-windings of the coil are away from the surface of the

scalp they are unlikely to induce an additional disruptive magnetic field. The

stimulation effects dissipate gradually as distance from the maximal point increases

(Figure 6.3).

Chapter 6

116

Figure 6.1 The sequences of events for a transcranial magnetic stimulation pulse (taken with permission from Walsh and Cowey, 2000). (a) A capacitor generates an electric current (up to 8kA). (b) This discharges into the TMS coil generating a magnetic pulse of up to 2 T. (c) The pulse has a rise time of roughly 200ms and lasts for 1ms, which changes rapidly due to its intense and brief nature. (d) The magnetic field generates an electrical field. (e) The magnetic field causes neural activity or changes in the resting potential of the underlying neurons. Note that monophasic pulses are shown in the figure (but biphasic pulses are used for repetitive TMS).

Chapter 6

117

Figure 6.2 The effects of fibre orientation and electric field orientation for the application of a TMS pulse.

Figure 6.3 TMS-induced electrical fields produced by circular (top left) and figure-of-eight (top right) shaped coils. Maximal intensity with a circular coil is located directly under the winding, but with a figure-of-eight coil it is at the intersection of the two windings. The intensity of the induced current dissipates with a radial distance from the area of maximum intensity (diagram taken from the Magstim Guide to Magnetic Stimulation).

Chapter 6

118

6.3 The spatial resolution of TMS

Figure 6.4 illustrates the spatial and temporal specificity of TMS in relation to

other experiment methodologies.

Figure 6.4 The spatial and temporal resolution of TMS compared with other experimental techniques. TMS benefits from high spatial and temporal resolution and is capable of interfering with brain function (taken with permission from Walsh and Cowey, 2000).

As noted in the last section, the magnetic field induced by TMS dissipates

gradually as distance from the maximal point of stimulation decreases. This raises a

concern of how confident one can be in the spatial and functional specificity of TMS.

Put another way, how can one be sure that the effects observed in a TMS study are

due to stimulation of the target region or due to more widespread dissipating effects of

the induced magnetic field. The answer to this lies in a number of converging lines of

evidence which demonstrate that while in theory, the magnetic field induced by TMS

Chapter 6

119

is infinite (with the induced electrical field decreasing from the centre of the

stimulation focal point), in practice the size of the electrical field capable of disrupting

normal neuronal activity is limited (i.e. it is anatomically and functionally specific).

For example, TMS of the motor cortex results in visible and measurable (via MEPs)

motor twitches in the muscles. This effect however is neither random nor non-

specific – TMS to the motor cortex has been shown to evoke muscular twitches from

selective muscle groups in a manner compatible with the functional layout of the

motor homunculus, with stimulation at target sites varying from 0.5 to 1 centimetres

apart leading to selective activation of each muscle type (Singh, Hamdy, Aziz, and

Thompson, 1997; Wassermann, McShane, Hallet, and Cohen, 1992). The functional

focality of the method is further demonstrated in the visual domain, where a similar

spatial resolution (less than 1 cm on the scalp) has been reported through the study of

the physiological effects induced by TMS stimulation of the primary visual cortex

(V1; c.f. Walsh and Cowey, 2000). Moreover, stimulation to V1 leads to the

generation of phosphenes, which are spatially distributed in a manner that corresponds

with the retinotopic organisation of V1 (Kammer, 1999).

Outside of the primary motor and sensory areas, the effective spatial

resolution3 of TMS has regularly been demonstrated by functional dissociations

following TMS to spatially discrete regions of the cortex. For example, in the same

subjects, TMS targeted at the right occipital face area (thought to be functionally

specialised for face processing) has been shown to impair face but not body or object

recognition, while stimulation at a region of lateral occipital cortex (LO; thought to be

functionally specialised for object processing) has been shown to impair object but

not face or body recognition, whilst stimulation at the right extra-striate body area 3 It is of note, that TMS does not only stimulate the neuron in a 1cm region, rather, it is that this represents the physiologically effective resolution of TMS.

Chapter 6

120

(EBA; thought to be functionally specialised for body processing) has been shown to

impair body but not face or object recognition (Pitcher et al., 2009) – this dissociation

is particularly striking given the anatomical adjacency between these regions.

Functional dissociations such as the triple dissociation described above demonstrate

the functional focality of TMS and have been regularly observed across a variety of

domains (e.g. Ashbridge, Walsh, and Cowey, 1997; Stewart, Walsh, Frith, and

Rothwell, 2001).

A further approach to assess the spatial specificity of TMS is to combine the

approach with methods such as fMRI, MEG or PET. To date, studies that have

combined these methodologies have demonstrated a good correspondence between

TMS defined functional regions and the areas revealed with high spatial resolution

brain imaging techniques (Bestmann, Baudewig, Siebner, Rothwell, and Frahm, 2004;

Bohning, Shastri, McConnell, Nahas, Lorberbaum, and Roberts, 1999; Ruff et al.,

2006; Siebner et al., 1998; Terao et al., 1998).

In sum, while the effects of TMS will dissipate from the targeted region to

other cortical areas, the functionally effective resolution is much more discrete

(approximately 1cm). This has been demonstrated across a variety of studies by

systematically measuring the effect on behaviour as the coil is moved away from an

optimal stimulation site (e.g. by stimulating adjacent areas of cortex [Figure 6.5]

which demonstrate functionally different characteristics or examining direct

physiological effects).

Chapter 6

121

Figure 6.5 Functional dissociation method that can be employed using TMS (taken with permission from Walsh and Cowey, 2000). Using TMS one can infer the importance of a targeted region of stimulation for a particular function. Stimulation to an adjacent or functionally alternative region enables refined inferences to be made, including functional-anatomical attributions. The ‘coils’ and induced fields in this figure are illustrative of the methodological rationale and do not represent real configurations and effects. 6.4 Offline and online TMS: The temporal resolution of TMS

When considering the temporal effects of TMS a distinction needs to be drawn

between offline and online paradigms. Online paradigms involve stimulation

concurrent with behavioural performance (i.e. when a participant is doing a task) and

have short-lasting effects (e.g. 1 msec per pulse), while offline paradigms take place

prior to task performance and have more long-lasting effects during an experimental

session (e.g. 20-60 minutes of altered brain activity).

For online TMS experiments, a typical single pulse of TMS is very brief,

around 1ms. An important consideration for online TMS experiments is when is the

most appropriate time window for neural disruption to produce behavioural

impairment. A number of approaches have been developed to address this, including

delivering single (Amassian, Cracco, Maccabee, Cracco, Rudell, and Eberle, 1993) or

paired-pulses (Juan and Walsh, 2003; O’Shea, Muggleton, Cowey, and Walsh, 2004)

of TMS to the target region at different time points after stimulus onset or the start of

Chapter 6

122

trial. The later has the advantage of reducing the number of temporal conditions in a

TMS timing experiment.

It is also possible that the disruption induced by multiple pulses of TMS will

summate, therefore inducing larger behavioural impairments. The potential for

summation of the disruptive effect has been further exploited by repetitive TMS

(rTMS) protocols. For example, Rushworth and colleagues (2001) delivered TMS at

a frequency of 10 Hz for 500 ms and showed dissociations between parietal regions

(supramarginal gyrus and angular gyrus) for mediating modality-specific attentional

processes. To date there is no corroborating physiological evidence that the five

pulses of TMS actually do summate, but despite this the approach has proven to be a

robust online TMS protocol for demonstrating function-specific involvement of a

wide variety of cortical areas across a number of domains (e.g. Beck, Muggleton,

Walsh, and Lavie, 2006; Bjoertomt, Cowey, and Walsh, 2002; Campana et al., 2002;

Lavidor and Walsh, 2003; Pitcher et al., 2008; Pitcher et al., 2009; Wig, Grafton,

Demos, and Kelley, 2005).

In contrast to online TMS paradigms which rely upon observing effects in

very short-lasting temporal windows, a recently introduced paradigm of offline

continuous theta-burst stimulation (cTBS) provides a more long lasting window for

one to examine changes in cortical function on behaviour. cTBS is a form of rTMS

based on the burst patterns used to induce long lasting changes in synaptic

effectiveness in animal experiments. The approach uses high frequency stimulation

bursts (3 pulses at 50Hz), which are repeated at intervals of 200 milliseconds (i.e.

5Hz). In the motor system cTBS to M1 suppresses the excitability of motor cortical

circuits for 20-60 minutes depending on the cTBS parameters used (Di Lazzoro et al.,

2005; Di Lazzoro et al., 2008; Huang, Edwards, Rounis, Bhatia, and Rothwell, 2005;

Chapter 6

123

Huang et al., 2009; Stefan, Gentner, Zeler, Dang, and Classen, 2008). In paradigms in

which 20 seconds of cTBS (300 TMS pulses) is applied to the motor cortex, MEP

amplitude is reduced for approximately 20-30 minutes; in paradigms in which 40

seconds of cTBS (600 TMS pulses) is applied MEP amplitude is reduced for

considerably longer (up to one hour; Huang et al., 2005). This rapid method of

suppressing cortical activity offers a unique opportunity to examine the functional role

of a given brain region on behaviour in a relatively large time window. It also

overcomes a number of potential confounds related to online stimulation, including

the peripheral and proprioceptive effects of online TMS (e.g. muscular twitching;

Terao et al. 1997) that may impact on task performance. The potential for this

paradigm to be used to study cognitive processing has recently been fulfilled in

several domains. Firstly, Vallesi and colleagues (2007) used cTBS to show a critical

role for the right dorsolateral prefrontal cortex, but not left dorsolateral prefrontal

cortex or right angular gyrus, in temporal processing. More recently, Kalla and

colleagues used cTBS targeted at the dorsolateral prefrontal cortex to demonstrate the

necessity of this brain region, but not the vertex or MT/V5 (TMS to MT / V5

facilitated processing), in conjunction visual search (Kalla, Muggleton, Cowey, and

Walsh, 2009). The studies reported in chapters 7 and 8 used cTBS to investigate the

role of sensorimotor cortices in expression recognition from visual and auditory cues.

6.5 The safety of TMS as an experimental tool

The primary concern in any TMS experiment is the health and safety of the

subjects (c.f. Wasserrmann, 1998 for detailed safety guidelines). The main ethical

concern in the use of TMS is the possible side effects of this method. TMS is

sometimes associated with minor discomfort, muscular pain, and occasionally mild

headache. These are all treatable with simple pain killers such as aspirin, and any

Chapter 6

124

discomfort can mostly be alleviated by repositioning the TMS coil. All volunteers

should be informed of these possible effects and they should be minimised in each

individual. As with all studies, participants should be informed that they can withdraw

from the experiment at any time, without having to give a reason why. Sessions in

which subjects are perceived to be uncomfortable but do not report this should also be

terminated by the experimenter. The concern most commonly associated with TMS is

that in rare circumstances TMS has induced seizures. These are most likely to occur in

individuals already susceptible to seizures (i.e. with a history of epilepsy) and those

taking neuroleptic medication (Stewart, Ellison, Walsh, and Cowey, 2001). For the

studies reported in this PhD, only healthy adult subjects (aged from 18-50 years), with

no psychiatric or neurological history and no family history of seizures took part in

the proposed research. All safety guidelines regarding the use of TMS (Wassermann,

1998) were followed and all the experiments reported in the thesis were approved by

the local ethics committee at University College London.

Chapter 7

125

CHAPTER 7: THE ROLE OF SENSORIMOTOR SIMULATION

IN AUDITORY EMOTION DISCRIMINATION

Functional neuroimaging studies indicate that activity in primary somatosensory and

premotor cortex is evoked during the perception of emotion. In the visual domain,

right somatosensory cortex activity has been shown to be critical for facial emotion

recognition. However, the importance of these representations in modalities outside

of vision remains unknown. This study used continuous theta-burst transcranial

magnetic stimulation (cTBS) to investigate whether neural activity in the right

primary somatosensory cortex (rSI) and right lateral premotor cortex (rPM) is

central to non-verbal auditory emotion recognition. Two groups of participants

completed same-different tasks on auditory stimuli, discriminating between either the

emotion expressed or the speakers’ identities, prior to and following cTBS targeted at

rSI, rPM or the vertex (control site). A task-selective deficit in auditory emotion

discrimination was observed. Stimulation to rSI and rPM resulted in a disruption of

participants’ abilities to discriminate emotion, but not identity, from vocal signals.

7.1 Introduction

Our ability to recognise the emotions of others is a crucial feature of human

social cognition. The neurocognitive processes which underpin this have recently

been described as being achieved through simulation processes (Adolphs, 2002;

Adolphs, 2003; Damasio, 1990; Gallese, Keysers, and Rizzolatti, 2004; Goldman and

Sripada, 2005; Keysers and Gazzola, 2006). These models suggest that understanding

another’s emotions requires individuals to map the observed state onto their own

representations which are active during the experience of the perceived emotion. The

discovery of mirror neurons in the primate brain (Gallese, Fadiga, Fogassi, and

Rizzolatti, 1996), and evidence of not only a ‘classical’ action mirror system (Buccino

et al., 2001; Fadiga, Fogassi, Pavesi, and Rizzolatti, 1996; Gazzola, Aziz-Zadeh, and

Keysers, 2006), but also ‘extended’ mirror systems in the human brain (involved in

mirroring sensation and emotion; Avenanti, Bueti, Galati, and Aglioti, 2005;

Blakemore, Bristow, Bird, Frith, and Ward, 2005; Carr, Iacoboni, Dubeau, Mazziotta,

and Lenzi, 2003; Keysers, Wicker, Gazzola, Anton, Fogassi, and Gallese, 2004;

Chapter 7

126

Singer, Seymour, O’Doherty, Kaube, Dolan, and Frith, 2004; Wicker, Keysers¸

Plailly, Royet, Gallese, and Rizzolatti, 2003) has provided a candidate

neurophysiological mechanism for such shared representations in emotion

recognition. These brain regions may, in part, aid emotion recognition because they

enable the observer to match the observed emotion within cortical areas active during

the observer’s own experience of the perceived emotion (Carr et al., 2003;

Hennenlotter et al., 2005; Jabbi, Swart and Keysers, 2007; Leslie, Johnsen-Frey, and

Grafton, 2004; van der Gaag, Minderaa, and Keysers, 2007). Consistent with this,

functional brain imaging studies indicate that components of classical and extended

mirror systems (including premotor cortex and primary somatosensory cortex) are

recruited when perceiving others’ facial emotions (Hennenlotter et al., 2005; Leslie,

Johnsen-Frey, and Grafton, 2004; Montgomery and Haxby, 2008; van der Gaag,

Minderaa, and Keysers, 2007); that primary somatosensory cortex is activated when

judging another’s facial emotion (Winston, O’Doherty, and Dolan, 2003); and that the

auditory-motor mirror system is activated during the perception of non-vocal emotion

expressions (e.g. hearing somebody laughing; Warren et al., 2006). Further, in

chapters 4 and 5 I show that facilitated sensorimotor simulation (in mirror-touch

synaesthesia) is linked to heightened emotional empathy and emotional expression

recognition. In attempt to assess what impact suppressing sensorimotor activity has

on the expression recognition abilities of healthy adults, here (and in chapter 8) I use

TMS in non-synaesthetes to assess whether sensorimotor activity plays a central role

in our ability to recognize the emotions of others.

In the visual domain, there is growing evidence that sensorimotor activity

plays a causal role in facial emotion recognition. Neuropsychological findings

indicate that deficits in the recognition of facial affect are related to damage within

Chapter 7

127

right hemisphere somatosensory-related cortices (Adolphs, Damasio, Tranel, Cooper,

and Damasio, 2000) and transcranial magnetic stimulation (TMS) findings in healthy

adults are consistent with this (Pitcher, Garrido, Walsh and Duchaine, 2008). In the

study by Pitcher and colleagues (2008), rTMS targeted at right primary

somatosensory cortex resulted in a disruption of participants’ abilities to discriminate

the facial expressions, but not facial identities, of others. It remains unclear if neural

activity within these systems is necessary for the recognition of affect from alternative

modalities. Moreover, if sensorimotor resources are vital for global processing of

emotion then these resources should also be central for emotion recognition abilities

of healthy adults in modalities outside of visual perception. An example of this would

be in the auditory domain. Primates are highly sensitive to vocal cues and the

affective contents of vocal signals are reliably recognised by humans (Bryant and

Barett, 2007; Sauter and Scott, 2007; Schröder, 2003). An fMRI study indicates that

adult human listeners activate the sensorimotor cortices when listening to emotional

vocalisations of others (Warren et al., 2006), however whether this activity is

necessary for affect recognition remains unknown. To address this, the studies

presented in this chapter use continuous theta burst TMS (cTBS; Di Lazzaro et al.,

2005; Huang, Edwards, Rounis, Bhatia, and Rothwell, 2005; Vallesi, Shallice, and

Walsh, 2007; Kalla, Muggleton, Cowey, and Walsh, 2009), an offline (i.e. conducted

while the participant is at rest) TMS paradigm following which neural activity may be

suppressed for several minutes (Di Lazzaro et al., 2005; Huang et al., 2005), to

examine whether neural activity in the right lateral premotor (rPM) and right primary

somatosensory cortex (rSI) is involved in discriminating affect from vocal signals.

Right hemisphere representations were selected based on previous fMRI,

neuropsychological and TMS findings demonstrating the importance of right

Chapter 7

128

hemisphere activity in affect recognition (e.g. Adolphs et al., 2000; Mitchell and

Crow, 2005; Pitcher et al., 2008; Pourtois et al., 2004; Van Lancker and Fromkin,

1973).

Two experiments were conducted. Experiment 1 sought to establish the

effects of cTBS targeted at rPM, rSI, or the vertex (cTBS control site) on participants’

abilities to complete a same-different auditory emotion recognition task (Figure 7.1;

Figure 7.2). Non-verbal emotional vocalisations (such as laughter or screams) were

used. These vocalisations are reliably recognised by human listeners (Sauter and

Scott, 2007; see also Meyer, Zysset, von Cramon, and Alter, K, 2005; Schröder, 2003)

and can be considered to be closer to emotional facial expressions than emotional

speech because they do not contain the segmental structure of emotionally inflected or

nonsense speech (Dietrich, Szameitat, Ackermann, and Alter, 2006; Scott, Sauter, and

McGettigan, in press; Scott, Young, Calder, Hellawell, Aggleton, and Johnsen, 1997).

In experiment 2, identical stimuli and cTBS parameters were used, but a new group of

participants were instructed to complete a same-different auditory identity

discrimination task. This enabled examination of any non-specific effects of cTBS

and whether the effects observed in experiment 1 were selective to affective

processing. Based on simulation accounts of emotion recognition it was predicted

that cTBS targeted at rPM and rSI would result in a disruption of participants’ ability

to discriminate the auditory emotions, but not identity, of others.

7.2 Methods

Participants

Twenty healthy naïve adult participants, 11 female and 9 male (aged 20 to

35years), took part in the study. All were right handed, had normal or corrected-to-

normal vision, and gave informed consent in accordance with the ethics committee of

Chapter 7

129

University College London. Ten participants took part in each experiment

(Experiment 1: 6 female and 4 male aged 20 to 30 years; Experiment 2: 5 female and

5 male aged 20 to 35 years).

Materials

Identical stimuli were used in experiments 1 and 2. Stimuli were one of four

categories of non-verbal auditory emotions (amusement, sadness, fear, or disgust).

These stimuli were adapted from a previously validated set of non-verbal

vocalisations (Sauter, PhD thesis, University of London, 2006; Sauter and Scott,

2007; Sauter and Eimer, in press; Warren et al., 2006) and two of these emotions

(amusement and fear) were adapted (trimmed to 500msec) from stimuli used in a

previous fMRI study investigating the role of sensorimotor resources in non-verbal

auditory emotion perception (Warren et al., 2006). Ten stimuli, produced by four

different actors (two male / two female), per emotion type were used. All were 500

milliseconds in duration and were presented aurally via headphones.

Procedure

Both experiments consisted of three testing sessions conducted over three non-

consecutive days. At each testing session one of the three brain regions was

stimulated (rSI, rPM or Vertex). The order of site of stimulation was pseudo-

randomised between participants in an ABC-BCA-CAB fashion. Participants

completed the experimental task twice within each session, one run prior to cTBS

(baseline performance) and the other following cTBS.

In experiment 1, the task comprised of 120 trials (preceded by 20 practice

trials) divided between two blocks of 60 trials. Each trial began with the presentation

of a fixation cross (1500 ms) followed by the presentation of the prime stimulus. 500

Chapter 7

130

milliseconds after the offset of the prime stimulus a second emotion was presented

aurally. Concurrent with the presentation of the second emotion, participants were

asked to indicate if the second non-verbal emotion was the same or different from the

first using a key press (Figure 7.1). The need for speed and accuracy were

emphasised. Each block lasted approximately 10-15 minutes.

In experiment 2, the same stimuli and paradigm were used, but participants

were instructed to indicate if the prime and target emotions were expressed by the

same or different person.

Chapter 7

131

Figure 7.1 Summary of cTBS and task protocol. Participants completed a same-different auditory emotion (Experiment 1) or identity (Experiment 2) matching task. Both experiments consisted of three testing sessions conducted over three non-consecutive days. At each testing session one of the three brain regions was stimulated (rSI, rPM or the vertex). In each session, the task was completed prior to and 5 minutes following cTBS to each site. The 5 minute rest period was based on the observed time course of effects seen in the motor cortex. During the task, trials began with a fixation cross followed by the presentation of a prime emotion via headphones. Non-verbal auditory emotions of amusement, sadness, fear or disgust were used. After 500 milliseconds a second emotional expression was presented which was either the same or different from the target. In experiment 1, participants were asked to indicate whether the emotional expression was the same or different to the prime using a key press. In experiment 2, a new group of participants were asked to indicate whether the emotional expressions were produced by the same or different person. The need for speed and accuracy were emphasised.

TMS parameters and coregistration

TMS was delivered via a figure of eight coil with a 70mm diameter using a

Magstim Super Rapid Stimulator (Magstim, UK). An offline cTBS paradigm was

used, which consisted of a burst of 3 pulses at 50Hz repeated at intervals of 200ms for

20 seconds, resulting in a total of 300 pulses. This paradigm was used to prevent any

influence of online auditory and proprioceptive effects of TMS on task performance

Chapter 7

132

(Terao et al., 1997). Based upon previous findings (Di Lazzaro et al., 2005; Huang et

al., 2005; ; Kalla et al., 2009; Valessi et al., 2007) the time window of reduced

excitability following theta burst stimulation was expected to last between 20-30

minutes and a 5 minute rest period after stimulation offset was implemented for each

site stimulated.

TMS machine output was set to 80% of each participant’s motor threshold

with an upper limit of 50% of machine output. Motor threshold was defined using

visible motor twitch of the contralateral first dorsal interosseus following single pulse

TMS delivered to the best scalp position over motor cortex. Motor threshold was

calculated using a modified binary search paradigm (MOBS [Tyrell and Owens, 1988;

see also Thilo, Santoro, Walsh and Blakemore, 2004 for example use). For each

subject, motor threshold was calculated following pre-cTBS baseline and prior to

coregistration.

Locations for cTBS were identified using Brainsight TMS-magnetic resonance

coregistration system (Rogue Research, Montreal, Canada). FSL software (FMRIB,

Oxford) was used to transform coordinates for each site to each subject’s individual

MRI scan (Figure 7.2). Coordinates for rSI (27, -27, 69) were taken from Blakemore

and colleagues (2005) and were averages for twelve neurologically normal

participants in an fMRI study following touch to their own face. The coordinates for

rPM (54, -2, 44) were the averages of neurologically normal participants in an fMRI

study of non-verbal auditory emotion processing (Warren et al., 2006). The vertex

was identified as the point midway between the inion and the nasion, equidistant from

the left and right intertragal notches.

Chapter 7

133

Figure 7.2 Summary of TBS sites stimulated, rSI (a), rPM (b). Locations of TMS were determined using the Brainsight coregistration system. The co-ordinates for rSI (27, -27, 69) were taken from a study on the tactile mirror-system (Blakemore et al., 2005), the co-ordinates for rPM (54 -2 44) were taken from a study on the role of sensorimotor resources in auditory emotion recognition (Warren et al., 2006). cTBS parameters were used to stimulate each site. To ensure that any differences observed were not due to non-specific effects of cTBS, the vertex was stimulated as a TMS control site.

7.3 Results

Reaction times were trimmed prior to analysis (± 3 standard deviations and all

errors removed) and were corrected for accuracy (mean RT divided by percentage of

correct).

The role of r SI and rPM in recognizing emotions and identity from auditory cues

Baseline performance did not differ significantly across sites in either task

(Emotion task group: F(2,18) = 1.64, n.sig; Identity task group: F(2,18) = .815, n.sig).

To assess the effects across tasks and across sites, the difference between the post

Chapter 7

134

cTBS and pre-cTBS baseline reaction times (i.e. baseline RT corrected for accuracy

minus post cTBS RT corrected for accuracy) was compared for each site stimulated.

A 2 (Task Group) x 3 (TMS Site) mixed ANOVA was then conducted to determine

the effects of cTBS on participants’ abilities to recognize identity and emotion from

auditory signals. The overall main effect of TMS Site was not significant [F(2,36) =

.361, p = .699], however a significant Task Group x TMS Site interaction was found

[F(2, 36) = 3.43, p = < .05]. This was because the effects of cTBS significantly

differed across sites on the emotion [F(2, 18) = 4.78, p = < .05], but not on the

identity task [F(2, 18) = .574, p = .573] (Figure 7.3). The main effect of TMS Site on

the emotion task was due to significant impairments following cTBS targeted at rPM

compared to the vertex (p = < .05) and following cTBS targeted at rSI relative to the

vertex (p = < .05). Therefore, cTBS stimulation of rSI and rPM disrupted

participants’ abilities to discriminate between the auditory emotions, but not the vocal

identities, of others - indicating that neural activity within these brain regions is

important for the emotion discrimination abilities of healthy adults.

Between-group comparisons also revealed a main effect of Task Group [F(1,

18) = 12.81, p = < .005]. This task-specific dissociation was modulated by site of

stimulation, with cTBS targeted at rSI (p = < .01) and rPM (p = < .01) resulting in a

different pattern of performance between the emotion and identity tasks (Figure 7.3).

This pattern of effects was not found following stimulation at the vertex (cTBS

control site), where there was a trend for facilitation in both tasks (p = .841). Thus,

the cTBS impairments observed at rSI and rPM in the emotion task are not due to

general impairments in processing following cTBS, but reflect a task specific

impairment on emotion discrimination performance.

C

hapt

er 7

135

-200

-150

-100

-500

50

100

150

200

250

300

Baseline minus cTBS RT (msec)

rSI

rPM

Vertex

*

Emotion Discrimination

a

-200

-150

-100

-500

50

100

150

200

250

300

Baseline minus cTBS RT (msec)

rSI

rPM

Vertex

Identity Discrimination

b

Figu

re 7

.3 M

agni

tude

of d

isru

ptio

n or

faci

litat

ion

(mea

n ±

s.e.

m) i

n m

illis

econ

ds f

ollo

win

g cT

BS

targ

eted

at r

SI, r

PM a

nd th

e ve

rtex

acro

ss ta

sk

grou

ps. I

n or

der

to d

eter

min

e if

the

mag

nitu

de o

f im

pairm

ent

follo

win

g cT

BS

stim

ulat

ion

diff

ered

acr

oss

the

site

s an

d ta

sks,

the

diff

eren

ce

betw

een

the

post

cT

BS

and

pre-

cTB

S ba

selin

e re

actio

n tim

es (

± 3

stan

dard

dev

iatio

ns a

nd a

ll er

rors

rem

oved

; and

cor

rect

ed f

or a

ccur

acy)

wer

e ca

lcul

ated

for

eac

h co

nditi

on (

i.e. b

asel

ine

RT/

Acc

urac

y m

inus

pos

t cT

BS

RT/

Acc

urac

y fo

r ea

ch s

ite s

timul

ated

acr

oss

task

s). A

dis

rupt

ion

in

reac

tion

times

fol

low

ing

stim

ulat

ion

is s

how

n by

a n

egat

ive

valu

e an

d fa

cilit

atio

n by

a p

ositi

ve v

alue

. (a

) Fo

r th

e em

otio

n di

scri

min

atio

n ta

sk,

parti

cipa

nts

(n =

10)

wer

e im

pair

ed i

n th

eir

abili

ties

to d

iscr

imin

ate

betw

een

the

audi

tory

em

otio

ns o

f ot

hers

fol

low

ing

stim

ulat

ion

to r

SI a

nd

rPM

com

pare

d to

stim

ulat

ion

at t

he v

erte

x (c

TBS

cont

rol

site

). (

b) T

his

was

not

fou

nd t

o be

the

cas

e w

hen

parti

cipa

nts

(n =

10)

had

to

disc

rimin

ate

audi

tory

iden

tity

in th

e ex

perim

ent -

the

effe

cts

of c

TBS

targ

eted

at r

SI, r

PM a

nd th

e ve

rtex

did

not s

igni

fica

ntly

dif

fer b

etw

een

the

site

s st

imul

ated

, and

ther

e w

as a

tren

d fo

r fa

cilit

atio

n at

all

site

s. B

etw

een-

grou

p co

mpa

riso

ns a

lso

reve

aled

that

the

disr

uptio

n in

per

form

ance

on

the

emot

ion-

disc

rim

inat

ion

task

fol

low

ing

cTB

S to

rSI

and

rPM

was

sig

nifi

cant

ly d

iffe

rent

to th

e fa

cilit

atio

n sh

own

in th

e id

entit

y ta

sk.

No

sign

ific

ant d

iffer

ence

bet

wee

n th

e em

otio

n di

scrim

inat

ion

and

iden

tity

disc

rimin

atio

n ta

sk p

erfo

rman

ce w

as fo

und

follo

win

g cT

BS

at th

e ve

rtex

.

* = P

<.0

5.

Chapter 7

136

Are some emotions affected more than others?

To clarify whether the overall disruption of auditory emotion discrimination

observed at rSI and rPM in experiment 1 was linked to a greater impairment for

specific emotions or was expression-general, the effects of cTBS (corrected baseline

RT minus corrected post cTBS RT) for each emotion type (amusement, disgust, fear,

sadness) in the emotion task group were compared using a 3 (TMS Site) x 4

(Emotion-Type) repeated measures ANOVA. This revealed a main effect of TMS

Site, [F(2, 18) = 4.97, p = <.05], due to the overall impairment in auditory expression

matching following cTBS targeted at rSI and rPM relative to the vertex noted

previously, but no significant interaction [F(6, 54) = 1.32, p = .266] or main effect of

emotion-type [F(3, 27) = .812, p = .498]. Therefore the overall impairment in

emotion discrimination following cTBS to rSI and rPM was not modulated by

emotion-type.

7.4 Discussion

The current study investigated whether neural activity in rSI and rPM was

important for discriminating affect from non-verbal vocal signals. Using

neuronavigation procedures to co-register targeted sites onto each participant’s

structural MRI scan, stimulation targeted at rSI and rPM led to a significant disruption

in the ability to discriminate the auditory emotions, but not identities, of others

(Figure 7.3). This pattern was not found following cTBS at the vertex, indicating that

the differences observed were not due to non-specific effects of cTBS. Therefore

consistent with predictions, rSI and rPM stimulation reduced the ability to

discriminate the auditory emotions, but not identities, of others.

These findings extend upon research demonstrating the involvement of right

somatosensory cortex representations in facial affect recognition (Adolphs et al.,

Chapter 7

137

2000; Pitcher et al., 2008) to suggest that activity in rSI is central to the perception of

emotion across different modalities. Further, in recent years a number of functional

brain imaging studies have documented the role of premotor cortex activity in the

mirroring of actions and emotions of others (Dapretto et al., 2006; Hennenlotter et al.,

2005; Leslie, Johnsen-Frey, and Grafton, 2004; Montgomery and Haxby, 2008; van

der Gaag, Minderaa, and Keysers, 2007; Warren et al., 2006). Using stimuli adapted

from one such study (Warren et al., 2006), the findings show that neural activity in

rPM plays a central role in non-verbal auditory emotion discrimination in healthy

adults. These findings are consistent with simulation-based accounts of emotion

processing, which contend that perceived emotions are mapped onto an individual’s

own somatosensory and motor representations to facilitate emotion recognition

(Adolphs, 2002; Adolphs, 2003; Damasio, 1990; Gallese, Keysers, and Rizzolatti,

2004; Goldman and Sripada, 2005; Keysers and Gazzola, 2006).

The task specific nature of the findings further supports the role of rSI and

rPM activity as a substrate for a mechanism that facilitates emotion processing.

Under equivalent conditions to experiment 1, cTBS targeted at rSI and rPM did not

impair participants’ ability to discriminate another’s identity, indicating that the

changes in reaction time are not simply due to a general reduction in reaction times

following cTBS stimulation of these regions or more widespread suppression of

neural activity. In contrast to a disruption in emotion discrimination abilities, there

was a trend for facilitation when participants were asked to discriminate the identity

of others. This facilitation is non-specific because it is seen over all sites stimulated in

the identity task, and does not differ significantly between sites. The nature of the

effect may reflect practice in the post-cTBS blocks or intersensory facilitation

following cTBS (Marzi et al., 1998; Walsh and Pascual-Leone, 2003).

Chapter 7

138

The findings are also compatible with recent TMS findings documenting the

necessity of the right fronto-parietal operculum in emotional prosody (van Rijn,

Aleman, van Diessen, Berckmoes, Vingerhoets, and Kahn, 2005; Hoekert, Bais,

Kahn, and Aleman, 2008). They extend upon them by examining the role of

somatosensory and motor cortices in non-verbal auditory emotion processing. These

kind of auditory signals differ from emotionally inflected speech because they do not

have the segmented structure of speech and provide relatively “pure” vocal

expressions of emotion (Scott et al., 1997; Scott, Sauter, and McGettigan, in press;

Dietrich, Szameitat, Ackermann, and Alter, 2006). This enables a closer parallel to

previous studies examining the necessity of cortical resources in the processing of

emotional facial expressions (Adolphs et al., 2000; Pitcher et al., 2008). In addition,

the findings demonstrate the importance of motor resources in auditory emotion

discrimination and show a functional dissociation for the role of sensorimotor

simulation in discriminating speaker emotion, but not speaker identity, from vocal

signals.

There is growing evidence that the ability to detect affect from voice relies

upon similar neural mechanisms which are recruited for visual social signals. For

example, in the visual domain, event related potential (ERP) studies have

demonstrated enhanced frontal positivity for emotional compared to neutral faces 150

msec after stimulus onset (Ashley, Vuilleumier, and Swick, 2004; Eimer and Holmes,

2002; Eimer and Holmes, 2007; Eimer, Holmes, and McGlone, 2003; Holmes,

Vuilleumier, and Eimer, 2003). This mechanism also extends to the auditory domain,

in which non-verbal auditory emotions compared with spectrally rotated neutral

sounds result in an early fronto-central positivity which is similar in timing, polarity

and scalp distribution to ERP markers of emotional face processing (Sauter and

Chapter 7

139

Eimer, in press). The findings presented here add to this by demonstrating that

activity in rSI is implicated in not only facial (Adolphs et al., 2000; Pitcher et al.,

2008), but also auditory emotion perception and imply that sensorimotor resources

may sub-serve an emotion-general processing mechanism in healthy adults (Adolphs,

2002; Adolphs, 2003; Damasio, 1990; Gallese, Keysers, and Rizzolatti, 2004;

Goldman and Sripada, 2005; Keysers and Gazzola, 2006).

In the current study I focussed on right hemisphere representations based on

previous fMRI, neuropsychological and TMS findings demonstrating the importance

of right hemisphere activity in affect recognition (Adolphs et al., 2000; Mitchell and

Crow, 2005; Pitcher et al., 2008; Pourtois et al., 2004; Van Lancker and Fromkin,

1973). There is some fMRI evidence that viewing static and dynamic facial

expressions evokes activity in bilateral primary somatosensory cortex and premotor

cortex (Montgomery and Haxby, 2008; van der Gaag et al., 2007). Further, in the

auditory domain, listening to non-vocal emotional expressions leads to bilateral

activations of the lateral premotor cortex (Warren et al., 2006). The lateralization of

these effects shall be addressed with further studies.

In sum, this study extends previous findings that rSI activity is important in

facial emotion recognition (Adolphs et al., 2000; Pitcher et al., 2008), by

demonstrating that neural activity in rSI is involved in emotion processing across

modalities. The findings also demonstrate that rPM activity reported in previous

fMRI studies is central to non-verbal auditory emotion discrimination. These

resources are not specifically required for discriminating the identity of others and

appear to play a specific role in facilitating emotion discrimination in healthy adults.

Chapter 8

140

CHAPTER 8: THE ROLE OF SENSORIMOTOR SIMULATION

IN FACIAL EXPRESSION RECOGNITION

The findings in chapter 7, demonstrated that the right primary somatosensory cortex

and right premotor cortex play a critical role in discriminating between the non-

verbal auditory emotions of others. Recent findings indicate that neural activity in

right primary somatosensory cortex is also necessary for the recognition of facial

expressions in healthy adults, but it remains unclear whether neural activity in

cortical regions involved in other aspects of sensorimotor simulation (e.g. simulation

of motor as opposed to somatic consequences of the perceived emotion) are also

central to the facial expression recognition abilities of healthy adults. Further, in the

face processing literature, whether neural activity in different components of the

sensorimotor simulation network are central for recognizing all expressions (i.e. an

expression-general mechanism) or for subsets of expressions remains a point of

debate (i.e. an expression-specific mechanism). Using continuous theta burst

transcranial magnetic stimulation (cTBS) in neurologically normal subjects, this

study sought to establish whether sensorimotor neural activity is critical for the facial

expression recognition abilities for some, or all, basic facial expressions. cTBS was

targeted at right primary somatosensory cortex (rSI), right inferior frontal gyrus

(rIFG) or right V5 / MT (control site) while participants completed a four-forced-

choice expression categorization task. cTBS to rSI, but not rIFG or right V5 / MT

(control site), significantly disrupted participants abilities to correctly categorize

happy and sad facial expressions, but not disgust or neutral facial expressions. These

findings are consistent with sensorimotor simulation models of expression recognition

which suggest that in order to understand another’s facial expressions individuals

must map the perceived expression onto the same sensorimotor representations which

are active during the experience of the perceived emotion.

8.1 Introduction

As noted previously, perceiving and correctly interpreting the expressions of

others is a vital component of social interaction. The processes which facilitate this

skill have often been described through mechanisms of simulation (Adolphs, 2002;

Adolphs, 2003; Damasio, 1990; Gallese, Keysers, and Rizzolatti, G, 2004; Goldman,

and Sripada, 2005; Keysers and Gazzola, 2006). These simulation-models of

expression recognition contend that in order to understand another’s expression one

must match the perceived state onto the sensorimotor responses associated with

experiencing the expression. Supporting this contention, in the visual domain,

Chapter 8

141

subliminal exposure to emotional facial expressions leads to increased responses in

expression relevant facial muscles of the observer (Dimberg, Thunberg, and Elmehed,

2000); blocking expression relevant facial muscles results in deficits in the observer’s

ability to correctly categorize the expressions of others (Oberman, Winkielman, and

Ramachandran, 2007); perceiving another person’s facial expressions correlates with

increased activity in similar motor (e.g. inferior frontal gyrus and premotor cortex of

the human mirror system) and somatosensory representations (e.g. primary and

secondary somatosensory cortex) as when the perceiver generates the same emotion

or expression (Hennenlotter et al., 2005; Leslie, Johnsen-Frey, and Grafton, 2004;

Montgomery and Haxby, 2008; van der Gaag, Minderaa, and Keysers, 2007);

transiently disrupting neural activity in the somatosensory cortex disrupts the

observer’s expression recognition abilities (Pitcher, Garrido, Walsh and Duchaine,

2008; Pourtois et al., 2004); and brain damage to somatosensory-related cortices

results in facial expression recognition deficits (Adolphs, Damasio, Tranel, Cooper,

and Damasio, 2000).

While these findings converge on a key role for sensorimotor simulation in

facial expression recognition, a number of unanswered questions remain. For

example, recent findings indicate that right somatosensory-related cortices play a

pivotal role in facial expression recognition (Adolphs et al., 2000; Pitcher et al., 2008;

Pourtois et al., 2004), but the extent to which neural activity in cortical regions

involved in other aspects of sensorimotor simulation (e.g. simulation of motor as

opposed to somatic consequences of the perceived emotion) are also critical for facial

expression recognition remains unclear. Functional brain imaging indicates a role of

neural activity in both somatosensory and motor regions of the cortex in expression

recognition (Carr, Iacoboni, Dubeau, Mazziotta, and Lenzi, 2003; Hennenlotter et al.,

Chapter 8

142

2005; Leslie, Johnsen-Frey, and Grafton, 2004; Montgomery and Haxby, 2008; van

der Gaag, Minderaa, and Keysers, 2007; Winston, O’Doherty, and Dolan, 2003), but

these data alone cannot imply causation about the role of motor resources for

expression recognition. Motor mirror system activation involving the inferior frontal

gyrus (IFG; BAs 44, 45) has been shown to occur in a number of studies investigating

facial expression recognition or evaluation (Carr et al., 2001; Dapretto et al., 2006;

Hennenlotter et al., 2006; Kesler-West et al., 2001; Seitz et al., 2008); emotional

empathy (Jabbi, Swart, and Keysers, 2007; Schulte-Ruther, Markowitsch, Fink and

Piefke, 2007); and emotion recognition more generally (Wildgruber et al., 2005).

Neuropsychological findings indicate that the IFG is necessary for recognizing

emotions from the eyes (Shamay-Tsoory, Aharon-Peretz, and Perry, 2009), but the

ability to recognize expressions from the whole-face was not tested. Adolphs and

colleagues (2002) also report deficits in facial emotion recognition following damage

to the frontal operculum (including BA44), but the lack of region specificity limits the

conclusions which one can draw on the role of the IFG in facial expression

recognition in healthy adults. Therefore whether the IFG is critical to facial

expression recognition in healthy adults remains unknown.

There is also a discrepancy in the literature on whether sensorimotor resources

are critical for recognizing all or only some distinct facial expressions. fMRI

indicates that changes in neural activity in both motor (including premotor cortex and

IFG) and somatosensory (including SI and SII) cortices are related to perceiving a

range of facial expressions (Montgomery and Haxby, 2008; van der Gaag, Minderaa,

and Keysers, 2007). Neuropsychological findings are consistent with a role for

somatosensory-related cortices in expression general processing (Adolphs et al.,

2000), but whether neural activity in the somatosensory cortex of healthy adults is

Chapter 8

143

necessary for all or only some distinct facial expressions is a matter of debate. To

date only two transcranial magnetic stimulation studies have addressed the necessity

of the right somatosensory cortex in healthy adults. In one study, two emotional

expressions (fear and happiness) and single pulse TMS over right somatosensory

cortex were used to investigate the necessity of this brain region for emotion

recognition in healthy adults. These authors observed an expression-selective TMS-

related interference following stimulation of right somatosensory cortex during the

recognition of fearful, but not happy expressions (Pourtois et al., 2004). In contrast, a

more recent repetitive TMS study (Pitcher et al., 2008), using six emotional

expressions (anger, disgust, fear, happiness, sadness and surprise), found an

expression-general impairment following stimulation of the right somatosensory

cortex. Further study is needed to clarify this discrepancy.

Furthermore, in both previous TMS studies a same-different matching task

was used to assess participants’ expression recognition abilities. The nature of these

tasks requires some degree of working memory in which the participant must not only

recognize a sample expression but store the information in memory to match it to a

sequential expression. Therefore it is difficult to disentangle whether TMS

impairment results from a disruption of fronto-parietal working memory networks (cf.

Harris, Harris, and Diamond, 2001; Mottaghy, Gangitano, Sparing, Krause, and

Pascual-Leone, 2002; Oliveri et al., 2001) or sensorimotor simulation mechanisms per

se4.

To address this, this study sought to establish: i) whether neural activity in the

right primary somatosensory cortex (rSI) and right inferior frontal gyrus (rIFG) is

central to recognizing the facial expressions of others and ii) whether, at the cortical

4 Note that this is controlled for in Chapter 7 because of the task-specific dissociation observed.

Chapter 8

144

level, sensorimotor simulation is expression-general or expression-selective. To do

so, participants performed a four-forced-choice expression categorization task

(disgust, happy, neutral, sadness) following continuous theta burst TMS (cTBS)

targeted at the rSI, rIFG, and right V5 / MT (visual control TMS site). Facial

expressions of disgust, happiness, sadness, and neutral (expression control) were used.

All expressions were selected from the Karolinska Directed Emotional Faces set

(Lundqvist, Flykt, and İhman, 1998). Based on previous TMS findings (Pitcher et

al., 2008) it was predicted that cTBS targeted at rSI would result in an impairment of

participants’ abilities to correctly categorize expression types, but not neutral (control

expression). If rIFG activity plays a causal role in the emotional expression

recognition abilities of healthy adults then an impairment for categorizing emotional

expression types (but not neutral) following cTBS targeted at the rIFG site was

expected. Whether this impairment would be expression specific or expression

selective remains to be determined. No cTBS disruption was expected following

stimulation at right V5 / MT, which acted as a visual control site.

8.2 Method

Participants

Eleven healthy adult participants, 3 female and 8 male (aged 24 to 45 years),

took part in the study. All participants were right handed and had normal or corrected

to normal vision. Each participant gave written informed consent in accordance with

the ethics committee of University College London and was naïve to the hypothesis of

the experiment.

Materials and Procedure

The experiment consisted of three testing sessions conducted over three non-

consecutive days. At each testing session one of the three brain regions were

Chapter 8

145

stimulated: rSI, rIFG and right V5 / MT (TBS control site; Figure 8.1). The order of

site of stimulation was randomized between subjects. Each session consisted of two

blocks, one run prior to TBS (baseline performance) and the other following TBS.

During each block, participants completed a four-forced-choice emotion

recognition task, comprised of 140 trials (preceded by 20 practice trials) in which

participants had to indicate the emotional expression of a target face. Each trial began

with the presentation of a fixation cross (1500 ms) followed by the presentation of the

target stimuli. Target stimuli were presented in the centre of the screen for 250 ms.

Following the offset of the target stimuli participants were asked to indicate the

emotion expressed in the target emotion (either happy, sad, neutral or disgust) using a

key press (Figure 8.1d). Each block lasted approximately ten minutes. Stimuli were

displayed on an SVGA 17 inch monitor with a refresh rate of 100Hz. Thirty-five

grey-scale standardised images per emotion were used. Stimuli were selected from the

Karolinska Directed Emotional Faces set (Lundqvist, Flykt, and İhman, 1998).

Chapter 8

146

Figure 8.1 Summary of TMS sites simulated, right SI (a), rIFG (b), V5 / MT (c), and single trial protocol (d). (a, b, c) Locations of TMS were determined using the Brainsight Coregistration System. In addition to coregistration, the location of area V5 / MT was confirmed functionally via phosphenes. Continuous TBS parameters were used to stimulate each site. (d) Participants completed pre- and post-TBS blocks. Within each block, trials began with a fixation cross followed by the presentation of a target face displaying emotional expressions of happy, sad, neutral or disgust. Participants were asked to indicate the expression of the target face using a key press.

Chapter 8

147

TMS Protocol and Site Localisation

TMS was delivered via a figure of eight coil with a 70mm diameter using a

Magstim Super Rapid Stimulator (Magstim, UK). An offline continuous theta burst

TMS paradigm was used to prevent any influence of proprioceptive effects of TMS on

reaction time performance (Terao et al., 1997). The parameters were identical to

those used in Chapter 7. TMS machine output was set to 80% of each participant’s

motor threshold with an upper limit of 50% of machine output. Motor threshold was

defined using visible motor twitch of the contralateral first dorsal interosseus

following single pulse TMS delivered to the best scalp position over motor cortex and

was calculated using a modified binary search paradigm (MOBS; Tyrell and Owens,

1988). For each subject, motor threshold was calculated following pre-TMS baseline

and prior to coregistration.

Following the pre-TMS block, locations for TMS were identified using

BrainSight TMS-magnetic resonance coregistration system (Rogue Research,

Montreal, Canada). FSL software (FMRIB, Oxford) was used to transform

coordinates for each site to each subject’s individual MRI scan (Figure 1a, b, c).

Talairach coordinates for rSI (27, -27, 69) were taken from Blakemore and colleagues

(2005) and were averages for twelve neurologically normal subjects in an fMRI study

following touch to their own face (the same rSI site as that used in Chapter 7). The

MNI coordinates for rIFG (60, 8, 6) were the averages of twelve neurologically

normal subjects in an fMRI study of facial emotion (Hennenlotter et al., 2005) and the

region broadly corresponding to BA 44 of the inferior frontal gyrus. Coordinates for

V5 / MT (44, -67, 0) were taken from Dumoulin and colleagues (2000). In addition to

BrainSight coregistration, V5 / MT was also confirmed functionally using

phosphenes.

Chapter 8

148

8.3 Results

To assess the influence of speed and accuracy, reaction times were corrected

for accuracy in each condition. This was achieved by dividing reaction time (± 3

standard deviations and all errors removed) by accuracy in each condition.

The role of rSI and rIFG in recognizing different facial expressions of emotion

Preliminary analysis confirmed that baseline performance for each expression-

type did not significantly differ across the sites stimulated [Disgust - F(2,20) = 1.4 ,

nsig; Happy – F(2,20) = .280, nsig; Neutral – F(2,20) = .913, nsig; Sad – F(2,20) =

1.01, nsig].

To assess the effects across expression-types and across sites, the difference

between the post cTBS and pre-cTBS baseline reaction times for each expression (i.e.

baseline RT corrected for accuracy minus post cTBS RT corrected for accuracy) was

compared for each site stimulated (as per Chapter 7). A 3 (TMS Site) x 4

(Expression-Type) repeated measures ANOVA showed that neither the main effect of

TMS Site [F(2,20) = 1.55, p = .237] nor the main effect of Expression-Type reached

significance [F(3,30) = 1.39, p = .265]. There was however a significant TMS Site x

Expression-Type interaction [F(6,60) = 2.82, p = .017]. This was because cTBS at rSI

resulted in a significantly different pattern of effects across expression types [F(3,30)

= 4.34, p = .012], whereby cTBS impaired performance on trials involving

expressions of happiness relative to neutral facial expressions (p = < .05), and on trials

involving sadness relative to neutral (p = < .01) and disgusted facial expressions (p =

< .05). This was not the case at rIFG [F(3,30) = .977, p = .417] or right V5 / MT

[F(1.863,18.629) = .489, p = .608], where the effects of cTBS did not significantly

differ between the expression-types (Figure 7.2).

Chapter 8

149

-250

-200

-150

-100

-50

0

50

100

150

200

250

Baseline minus cTBS RT (msec)

rSI

rIFG

rV5

*

*

Disgust Happy Neutral Sad

* *

*

Figure 8.2 Magnitude of disruption or facilitation across expression-types (mean ± s.e.m) following cTBS targeted at rSI, rIFG and right V5 / MT. In order to determine if the magnitude of impairment following cTBS stimulation differed across the sites and expression the difference between the post cTBS and pre-cTBS baseline reaction times (± 3 standard deviations and all errors removed; and corrected for accuracy) were calculated for each expression (i.e. baseline RT/Accuracy minus post cTBS RT/Accuracy for each site stimulated across tasks) and compared across conditions. A disruption in reaction times following stimulation is shown by a negative value and an improvement by a positive value. Stimulation to rSI disrupted performance at recognizing happy and sad facial expressions compared to rIFG and right V5 / MT. The disruption on happy and sad trials at rSI significantly differed from the improvement seen on neutral trials. The pattern of performance on sad trials at rSI also differed significantly from that seen on disgust trials. * = P < .05.

Comparisons for each expression-type across TMS sites revealed that the

disruption caused by cTBS to rSI on trials involving sadness was significantly

different to the effects of cTBS at rIFG and right V5 / MT – a one way repeated

measures ANOVA comparing performance on sadness recognition across the three

sites revealed dissociable effects of site stimulated [F(2,20) = 4.35, p = .027], and

pair-wise comparisons revealed that this was due to participants’ showing a

significant disruption at recognising sadness following stimulation at rSI compared to

rIFG (p = <.01) and right V5 / MT (p = < .05). Similarly, a comparison on happiness

Chapter 8

150

recognition across the three sites, revealed a trend towards significance [F(2,20) =

3.04, p = .07]. Paired-comparisons conducted on the basis of this strong trend,

revealed that this was due to a significantly larger disruption on participants’ abilities

to recognize happiness following stimulation at rSI compared to rIFG [t(10) = 2.30, p

= < .05] and right V5 / MT [t(10) = 2.41, p = < .05] (Figure 7.2).

8.4 Discussion

This study sought to establish whether i) neural activity in rSI and rIFG is

critical for recognizing the facial expressions of others and ii) whether these

mechanisms are expression-general or expression selective. The findings first

demonstrate that neural activity in rSI, but not rIFG, is critical for the ability to

recognize others’ facial expressions. They further indicate that neural activity in rSI

plays a more crucial role in the recognition of emotional compared with neutral facial

expressions.

Previous neuropsychological findings indicate that lesions to right

somatosensory-related cortices lead to expression-general face recognition

impairments (Adolphs et al., 2000). In healthy adults, focal transcranial magnetic

stimulation of rSI has been shown to a) impair fearful, but not happiness, facial

expression recognition, implying that rSI is expression selective (Pourtois et al., 2004

- note that only 2 expression types were used and differences only found when

participants discriminated between two fearful expressions), and b) to impair general

facial expression recognition abilities, including happiness – implying that rSI is

expression-general (Pitcher et al., 2008 – note only 12 trials per expression type used).

Using neuronavigation procedures, the findings from the current study show that

magnetic stimulation of rSI results in selective impairments of happiness and sadness

Chapter 8

151

recognition, but not disgust or neutral recognition, and indicate that neural activity in

rSI plays a crucial role in the recognition of facial expressions other than fear per se

(c.f. Pourtois et al., 2004). The study is also the first to show a facial expression

recognition deficit on an emotion categorisation task (as opposed to same-different

matching tasks used previously), which further clarifies that previously reported

impairments in expression recognition following stimulation to rSI are not linked to a

disruption of fronto-parietal working memory networks.

Models of face processing posit a number of expression-selective, expression-

general, face-selective and multimodal mechanisms which support our ability to

perceive another’s emotions (Bruce and Young, 1986; Calder and Young, 2005;

Haxby, Hoffman, and Gobbini, 2000). Sensorimotor simulation is likely to be one of

these mechanisms, with neural activity in rSI providing a possible candidate to

facilitate this process. While the evidence that cTBS stimulation resulted in a

disruption of participants’ abilities to recognise sadness and happiness, but not

disgust, may imply a degree of expression selectivity within this region, it remains

possible that rSI is involved in recognizing alternative expression types. Moreover,

findings from chapter 7, of a general impairment in auditory expression recognition

following cTBS to rSI, and from Pitcher et al. (2008), of general impairments in the

facial expression recognition following TMS to rSI would argue against rSI acting as

an expression-specific mechanism. Outside of rSI, there is evidence for emotion-

specific neuropsychological deficits for expressions of disgust (Calder, Keane, Manes,

Antoun, and Young, 2000; Sprengelmeyer et al., 1996, 2003), fear (Calder, Lawrence,

and Young, 2001) and anger (Calder, Keane, Lawrence, and Manes, 2004; Lawrence,

Calder, McGowan, and Grasby, 2002). This disproportionate role of different brain

regions in processing specific expressions indicates that facial expressions are not

Chapter 8

152

processed by a single “expression system” and that expression-general and

expression-specific representations may interact at varying levels of processing. The

lack of impairment on disgust recognition found here may reflect the fact that the

neural noise introduced in rSI by TMS is better compensated for by emotions which

have more alternative mechanisms / expression-specific representations elsewhere in

the brain. For example, there is a good degree of evidence from functional brain

imaging (Phillips et al., 1997, 1998; Sprengelmeyer, Rausch, Eysel, and Przuntek,

1998), intracerebral recording (Krolak-Salmon et al., 2003), and neuropsychological

studies (Calder et al., 2000; Kipps, Duggins, McCusker, and Calder, 2007) which

indicate that the anteroventral insula acts as an expression-specific mechanism for

disgust recognition. The anteroventral section of the insula is connected to number of

regions which are thought to be involved in emotion processing across modalities

(including the primary somatosensory cortex, basal ganglia, amygdala, orbitofrontal

cortex and superior temporal cortex; Augustine, 1996; Flynn, Benson, and Ardila,

1999; Mesulam and Mufson, 1982) and has been suggested to act as a point of

convergence for sources involved in the processing of disgust recognition to varying

degrees (Kipps et al., 2007). It is feasible that suppressing rSI with cTBS reduces

one, of the multiple sources, of information which contributes to disgust processing in

this section of insula, and therefore does not result in impairment. It is also possible

that with alternative paradigms (e.g. same-different expression matching paradigms as

opposed to a forced-choice paradigm which could bias responses of disgust and

thereby facilitate performance) and stimuli (e.g. dynamic stimuli as opposed to static),

stimulation to rSI may lead to a disruption of participants’ abilities to recognise

disgusted facial expressions (c.f. Chapter 7; Pitcher et al., 2008).

Chapter 8

153

The findings that cTBS targeted at rIFG did not result in an impairment of

participants’ abilities to correctly recognise the other’s facial emotions are also

intriguing, especially in light of evidence that lesions to the IFG have been linked to

deficits in self-reported emotional empathy and in the ability to recognise emotions

from the eyes (Shamay-Tsoory et al., 2009). These differences are likely to reflect

variations in the processes involved in recognising emotional expressions from the

whole-face compared with emotional empathy per se, and imply that the IFG may not

be critical for all emotion-general tasks. The IFG has been reported as a

cytoarchitectonic homologue to monkey F5 (Petrides, Cadoret, and Mackey, 2005)

and highlighted as a core component of the classical human mirror system (Rizzolatti

and Craighero, 2004). A number of authors have suggested that the human mirror

system (including IFG, IPL, and STS) may be pivotal to social cognition (Gallese,

Keysers, and Rizzolatti, G, 2004; Keysers and Gazzola, 2006; Oberman and

Ramachandran, 2007). While caution is urged in interpreting a null result, the

evidence that magnetic stimulation targeted at rIFG does not impair the ability to

recognise other’s facial emotions stands in contrast to this hypothesis. It is of note

that this may not imply that motor simulation play no role in emotion processing,

because while the human mirror system is one neurophysiological candidate to

facilitate this process it need not be the only mechanism and other regions of the

motor system may be crucial for facial expression recognition (e.g. the human

premotor cortex – c.f. Chapter 7).

In summary, the findings from the current study indicate that rSI is critical for

the recognition of emotional (across multiple expression types) compared with neutral

facial expressions. They also indicate the rIFG is not critical to facial expression

recognition. This adds to the evidence that somatosensory activity may provide a

Chapter 8

154

general expression recognition mechanism (Chapter 7; Adolphs et al., 2000; Pitcher et

al., 2008). However, the evidence that rSI stimulation disproportionately affected

happy and sad expressions but not disgust, indicates that while somatosensory cortex

activity may be involved in the processing of a variety of facial expressions, in some

cases (e.g. with disgust recognition) alternative facial expression recognition

mechanisms (e.g. expression selective) may be sufficient to support facial expression

recognition when rSI activity is suppressed.

Chapter 9

155

CHAPTER 9: CONCLUSIONS

In this chapter, the empirical findings from chapters 2-5 and 7-8 of this thesis are

discussed in a wider context. Chapters 2-5 addressed the prevalence, neurocognitive

mechanisms, and consequences of mirror-touch synaesthesia for perception and

social cognition. These findings are discussed in relation to previous research on

synaesthesia and future studies on the neurocognitive mechanisms of mirror-touch

synaesthesia are also described. Chapters 4 and 5 also used mirror-touch

synaesthesia to inform us about the role of sensorimotor resources in social

cognition. These findings were complemented by studies in chapters 7 and 8, which

investigated the impact of suppressing sensorimotor representations on the expression

recognition abilities of healthy adults. The findings from chapters 4, 5, 7 and 8 are

discussed in the context of research on social cognition and sensorimotor accounts of

social cognition.

9.1 Introduction

As noted in the introduction to this thesis, synaesthesia is a condition in which

one attribute of a stimulus (the inducer) triggers a conscious experience of another

attribute (the concurrent) not typically associated with the inducer. For example, in

grapheme-colour synaesthesia the letter ‘a’ may trigger synaesthetic experiences of

colours. A large body of synaesthesia research has focussed on grapheme-colour

synaesthesia, which is often reported as being one of the most common forms of the

condition (Baron-Cohen, Burt, Smith-Lailtan, Harrison, and Bolton, 1996; Rich,

Bradshaw, and Mattingley, 2005; Simner et al., 2006). More recently, a newly

documented form of synaesthesia has been described in which individuals experience

tactile sensations on their own body simply when observing touch to another’s body

(mirror-touch synaesthesia; Banissy and Ward, 2007; Blakemore, Bristow, Bird, Frith,

and Ward, 2005). The studies in the first five chapters of this thesis investigated the

neurocognitive and perceptual profiles of mirror-touch synaesthesia. In addition, the

role of sensorimotor simulation mechanisms in social cognition was examined by

using principles of neuropsychology (in the case of mirror-touch synaesthesia) and

Chapter 9

156

transcranial magnetic stimulation (TMS). Specifically the following questions were

addressed:

1. What is the prevalence of mirror-touch synaesthesia and what

characteristics identify the condition (Chapter 2)?

2. What neurocognitive mechanisms give rise to mirror-touch synaesthesia

(Chapter 2)?

3. Does the presence of synaesthesia have implications for perceptual

processing (Chapter 3)?

4. What are the implications of heightened sensorimotor simulation in mirror-

touch synaesthesia for social cognition (Chapters 4 and 5)?

5. What are the implications of suppressing sensorimotor resources on

expression recognition abilities in healthy adults (Chapters 7 and 8)?

9.2 What is the prevalence of mirror-touch synaesthesia and what

characteristics underpin the condition?

As noted previously, in mirror-touch synaesthesia individuals experience

tactile sensations on their own body simply when observing touch to another person

(Banissy and Ward, 2007; Banissy, Cohen Kadosh, Maus, Walsh, and Ward, 2009;

Blakemore et al., 2005). The mapping between synaesthetic experience (i.e. location

on the synaesthete’s body) and observed-touch (i.e. the location where touch is

perceived on another person’s body) has been shown to vary between mirror-touch

synaesthetes (Banissy and Ward, 2007), with some synaesthetes reporting

synaesthetic experiences as if looking in a mirror (e.g. observed-touch to the left face

elicits a synaesthetic experience on the left cheek of the mirror-touch synaesthete;

specular subtype) and others as if they share the same anatomical body space (e.g.

observed-touch to the left face elicits a synaesthetic experience on the left cheek of

Chapter 9

157

the mirror-touch synaesthete; anatomical subtype). The first reported case of the

condition was provided in a single case fMRI study, which linked the condition to

heightened neural activity in a network of brain regions which are also activated in

non-synaesthetic control subjects when observing touch to others (the mirror-touch

system, comprised of the primary and secondary somatosensory cortex, premotor

cortex, intraparietal sulcus and superior temporal sulcus; Blakemore et al., 2005).

Chapter 2 examined the prevalence and characteristics of the condition.

In the first experiment reported in chapter 2 the prevalence of mirror-touch

synaesthesia was assessed by screening a large population of undergraduate students

for the presence of mirror-touch synaesthesia and determining the validity of reported

cases with a synaesthetic stroop task (Banissy and Ward, 2007). In the task,

participants were asked to indicate the site of touch on their own body while

observing touch to another person. Participants were asked to report the site of

veridical touch and ignore any synaesthetic tactile experience induced. For

synaesthetes, but not for controls, veridical touch could be in the same (congruent) or

different (incongruent) location to observed / synaesthetic touch (congruency was

determined according to each synaesthete’s self reports). Synaesthetes performed

slower in the incongruent condition and produced more errors linked to their

synaesthesia. Nine mirror-touch synaesthetes (from 567 participants screened) were

confirmed, which provides an estimated prevalence rate of 1.6%. This places mirror-

touch synaesthesia as one of the most common variants of synaesthesia, alongside

grapheme-colour synaesthesia (estimated prevalence rate of 2%; Simner et al., 2006)

and day-colour synaesthesia (estimated prevalence rate of 2.8%; Simner et al., 2006).

By combining cases of mirror-touch synaesthesia from the prevalence study

with cases of mirror-touch synaesthesia from self-referrals, the findings from chapter

Chapter 9

158

2 also indicate a number of features linked to the characteristics of the condition. The

findings from experiment 2 indicate that the inducer for synaesthetic experience is not

linked to spatial cueing, but is related to bodily touch (and in some cases touch to

objects). The findings also indicate the specular subtype is the more common frame

of reference adopted by mirror-touch synaesthetes and the relative frequencies

(approximately 81% show a specular frame of reference) are similar to those reported

in studies investigating the preferred spatial frame adopted when imitating another’s

behaviour - both adults and children tend to imitate in a specular mode (Schofield,

1976; Franz, Ford and Werner, 2007). Further characteristics indicate commonalities

between mirror-touch synaesthesia and other variants of synaesthesia. For example, a

general characteristic of synaesthesia is that different variants of synaesthesia tend to

co-occur (Simner et al., 2006). This also appears to be the case in mirror-touch

synaesthesia. Synaesthetic experiences also tend to be consistent over time (Baron-

Cohen, Wyke, and Binnie, 1987) and the mirror-touch synaesthete’s spatial sub-type

(i.e. whether they belong to the specular or anatomical category) appears to be

consistent across time (Chapter 2) and across different body parts (Banissy and Ward,

2007).

While these findings indicate that mirror-touch synaesthesia shares common

ground with other types of synaesthesia, possible similarities in the neural basis of the

condition are less apparent. A point of dispute in the synaesthesia literature is

whether synaesthetic experience is due to cross-activation between brain regions

(either through increased structural connectivity or malfunctions in cortical inhibition)

or disinhibition of the same cortical networks found in non-synaesthetes (Bargary and

Mitchell, 2008; Cohen Kadosh, Henik, Catena, Walsh, and Fuetnes, 2009; Cohen

Kadosh and Walsh, 2008; Grossenbacher and Lovelace, 2001; Hubbard and

Chapter 9

159

Ramachandran, 2005; Rouw and Scholte, 2007). Cross-activation accounts have

tended to focus on adjacent brain regions (e.g. in the case of grapheme-colour

synaesthesia - between visual grapheme and colour processing areas in the fusiform

gyrus) and suggest that activation in the region responsible for processing the

synaesthetic inducer (e.g. the grapheme in grapheme-colour synaesthesia) leads to

activation in the adjacent region for processing the synaesthetic concurrent (e.g.

colour in grapheme-colour synaesthesia). It is not entirely clear how the principle of

adjacency can be applied to mirror-touch synaesthesia, and an alternative mechanism

which may bias individuals to this type of synaesthesia is the normal architecture for

multi-sensory interactions (Sagiv and Ward, 2006). For example, there is good

evidence for an observed-touch mirror system in non-synaesthetes (Keysers, Wicker,

Gazzola, Anton, Fogassi, and Gallese, 2004; Blakemore et al., 2005; Ebisch, Perrucci,

Ferretti, Del Gratta, Romani, and Gallese, 2008) and mirror-touch synaesthesia has

been suggested to reflect over-activity within this network (Blakemore et al., 2005).

Future studies will aim to address the similarities and differences in the neural

basis of different subtypes of synaesthesia by investigating structural and functional

correlates of different variants of synaesthesia (e.g. grapheme-colour, tone-colour,

mirror-touch, and number-space synaesthesia). For example, previous DTI findings

indicate that grapheme-colour synaesthesia is linked with increased structural

connectivity in right inferior-temporal, right parietal, and bilateral frontal regions

(Rouw and Scholte, 2007), and research in progress indicates that tone-colour

synaesthesia is linked to increased cortical thickness (a marker of cortical morphology

and neurodevelopment; MacDonald, Kabani, Avis, and Evans, 2000; Shaw et al.,

2006) in similar right inferior temporal regions (Banissy, Stewart, Ward, Walsh, and

Kanai, in prep). I am also starting a combined fMRI-DTI study of mirror-touch

Chapter 9

160

synaesthesia, which will investigate functional and structural correlates of mirror-

touch synaesthesia at the group level. This will permit further assessment of

similarities and differences in the neural substrates of different variants of

synaesthesia and across subtypes of mirror-touch synaesthetes.

9.3 What neurocognitive mechanisms may underpin mirror-touch

synaesthesia?

In addition to the studies reported in chapter 2, I proposed a neurocognitive

model to account for the processes which may underpin mirror-touch synaesthesia

(Figure 9.1). Three key mechanisms were highlighted: identifying the type of visual

stimulus touched (‘What’ mechanism), discriminating between self and other (‘Who’

mechanism), and locating where on the body and in space observed touch occurs

(‘Where’ mechanism).

The ‘What’ mechanism is considered to be involved in several

discriminations, including: ‘is this a human or object?’ ‘Is this a face or body?’ One

intriguing characteristic shown by some mirror-touch synaesthetes is that observing

touch to objects can elicit synaesthetic interactions in some, but not all, synaesthetes

(Chapter 2; Banissy and Ward, 2007). One brain region of the observed-touch

network (Blakemore et al., 2005) which may be central to this is the intraparietal

sulcus (IPS). Recent findings indicate that visual object information is processed

along the dorsal stream to areas along the medial bank of the intraparietal sulcus (IPS;

including IPS1 and IPS2, Konen and Kaster, 2008). For mirror-touch synaesthetes,

this pathway may be particularly important when considered in relationship to visual-

tactile body maps within the intraparietal cortex. Single-cell recording in primates has

identified bimodal neurons in the intraparietal cortex which fire in response to not

only passive somatosensory stimulation, but also to a visual stimulus presented in

Chapter 9

161

close proximity to the touched body part (Duhamel, Colby and Goldberg, 1998).

Intriguingly, the visual spatial reference frames of such bimodal neurons are dynamic,

such that if the monkey is trained to use a tool the visual receptive field extends to

incorporate the tool into the representation of the body - potentially as an extension of

the body schema (Iriki, Tanaka and Iwamura., 1996). Similar evidence of dynamic

multisensory body representations in the parietal cortex have been reported in human

subjects (Bremmer et al., 2001; Macaluso and Driver, 2003; also see Colby, 1998;

Maravita and Iriki, 2002; Berlucchi and Aglioti, 1997 for review). One hypothesis

generated by the model is that the degree to which observing touch to an object is able

to elicit visual-tactile synaesthetic interactions depends upon the extent to which the

object is incorporated into visual-tactile representations of the body, potentially within

the intraparietal cortex. A potential approach to investigate this would be to

investigate if extending the body-schema of a mirror-touch synaesthete through tool

use can result in a synaesthete who does not normally experience synaesthetic touch

for objects showing synaesthetic interactions for observed object-touch.

The key process instigated by the ‘Who’ mechanism is to distinguish between

the self and other. I suggest that mirror-touch synaesthesia reflects a breakdown in

the mechanisms which normally distinguish between self and other (i.e. processes

involved in linking visual representations with internal representations of bodies).

One prediction of this is that mirror-touch synaesthetes will show a tendency to over-

incorporate viewed bodies onto their own body schema. In accordance with this,

research in progress indicates that mirror-touch synaesthetes show a greater degree of

the rubber hand illusion (RHI) compared to matched non-synaesthete controls

(Banissy, MacDonald, Ward, Walsh, Haggard and Longo, in prep). The RHI is a

body schema illusion in which an observer is touched on their own hand while

Chapter 9

162

observing a rubber hand being touched. When observed and veridical touch are

synchronous the perceived location of the observer’s hand drifts towards the location

of the rubber hand (Longo, Schuur, Kammers, Tsakiris, and Haggard, 2008; Tsakiris

and Haggard, 2005) and participants begin to attribute the rubber hand to their own

body representation (Botvinick and Cohen, 1998). This effect is abolished with

asynchronous stroking and when participants view a non-hand object rather than a

rubber hand (Tsakiris and Haggard, 2005). A comparison of the size of the perceived

drift towards the rubber hand (in centimetres) indicates the mirror-touch synaesthetes

show a greater incidence of the illusion compared to matched non-synaesthetes when

observed and veridical touch are synchronous (i.e. under normal RHI conditions), but

no differences are found in conditions in which the illusion is not expected to take

place (e.g. non-hand object control conditions; Banissy et al., in prep). Future studies

will investigate whether modulations of the RHI differ between mirror-touch

synaesthetes and non-synaesthetes. For example, in non-synaesthetes rotating the

RHI 180° (i.e. as if from another’s perspective) abolishes the illusion (Tsakiris and

Haggard, 2005), however given that mirror-touch synaesthetes experience tactile

sensations when viewing touch from another’s perspective it will be interesting to

determine whether they show a similar or different pattern to non-synaesthetes.

The final class of mechanism described in the model involves linking visual

representations of body with tactile representations based on spatial frames of

reference. For this I draw a distinction between embodied (the sense of being

localised within one’s own body) and disembodied representations of the body (e.g.

autoscopic phenomena in which individuals experience the location of the self outside

of one’s own body – Brugger 2002; Blanke, Landis, Spinelli and Seeck, 2004; Blanke,

Ortigue, Landis and Seeck, 2002). It is postulated that a similar division can be made

Chapter 9

163

for the specular-anatomical division in mirror-touch synaesthetes. The specular

subtype appear to process the visual representation of the other body in an embodied

manner (i.e. as a mirror-image of oneself), while for the anatomical subtype the

spatial mapping between self and other could be considered disembodied in that the

synaesthete’s own body appears to share the same bodily template as the others

person (i.e. the synaesthete’s body is placed in the perspective of the other person).

This difference may suggest that anatomical mirror-touch synaesthesia can be

considered to be similar to types of autoscopic phenomena in which individuals

experience the location of the self outside of one’s own body (Brugger 2002; Blanke,

Landis, Spinelli and Seeck, 2004; Blanke, Ortigue, Landis and Seeck, 2002),

however, no mirror-touch synaesthetes report typical phenomena of autoscopy – that

of seeing one’s own body and the world from a location outside of their own physical

body5 (Bünning and Blanke, 2005). Therefore rather than classifying anatomical

mirror-touch synaesthesia within the bracket of autoscopic phenomena I would

suggest that some mechanisms which give rise to the spatial frames of adopted by

mirror-touch synaesthetes are modulated by similar mechanisms as those observed in

autoscopy. For example, disembodied experiences have been suggested to arise from

functional disintegration of low-level multisensory processing mechanisms (Bünning

and Blanke, 2005; Blanke and Mohr, 2005) and abnormal activity at the temporal

parietal junction (TPJ; Arzy, Thut, Mohr, Michel and Blanke, 2006; Blanke, Landis,

Spinelli and Seeck, 2004; Blanke, Ortigue, Landis and Seeck, 2002) and one may

suggest the anatomical sub-type will be associated with these neurocognitive

mechanisms.

5 It is of note that while synaesthetes may not have overtly reported autoscopy they may still experience this if tested systematically (e.g. Terhune, 2009).

Chapter 9

164

Figure 9.1 The ‘What, Who, Where Model of Mirror-Touch Synaesthesia’. ‘What’ mechanisms are shown in red boxes and are involved in defining the stimulus touched. ‘Who’ mechanisms implement discriminations between self and other, and are shown in blue boxes. ‘Where’ mechanisms are shown in green boxes and are involved in locating where on the body and in space observed touch occurs. Processes necessary for all subjects are shown with black arrows, necessary for specular mirror-touch synaesthetes with orange arrows, and for anatomical mirror-touch synaesthetes with purple arrows. Brain regions represented are considered with regard to importance for mirror-touch synaesthesia. AI = Anterior Insula; EBA = Extrastriate Body Area; FBA = Fusiform Body Area; FFA = Fusiform face area; IFG = Inferior Frontal Gyrus; IPL = Inferior Parietal Lobule; IPS = Intraparietal Sulcus; LO = Lateral Occipital Cortex; SI = Primary Somatosensory Cortex; SII = Secondary Somatosensory Cortex; STS = Superior Temporal Sulcus; TPJ = Temporoparietal Junction.

9.4 Does the presence of synaesthesia have implications for perceptual

processing?

Chapter 3 investigated whether the presence of synaesthesia has implications

for perceptual processing. Previous ERP findings indicated that the presence of

synaesthesia may exert a wider influence over sensory processing and impact on the

veridical sensory processing of synaesthetes (Barnett et al., 2008; Goller, Otten, and

Ward, 2009). For example, Barnett and colleagues (2008) report that, compared to

Chapter 9

165

non-synaesthetes, linguistic-colour synaesthetes show differences in early components

of the visual evoked potential (VEP) when presented with simple visual stimuli which

do not evoke synaesthesia. Further to this, Yaro and Ward (2007) report that

synaesthetes who experience colour show superior colour discrimination abilities

compared to non-synaesthetic control subjects. To assess whether enhanced

perceptual processing was a core property of synaesthesia, colour and tactile

sensitivity was contrasted between mirror-touch synaesthetes, synaesthetes who

experience colour as evoked sensations (colour synaesthetes), synaesthetes who

experience mirror-touch synaesthesia and colour synaesthesia (dual synaesthetes), and

a group of non-synaesthetic controls. The findings indicate a relationship between the

modality of synaesthetic experience and the modality of sensory enhancement. On a

test of tactile discrimination, mirror-touch synaesthetes showed superior tactile

discrimination compared to colour synaesthetes and non-synaesthetes. On a test of

colour perception, colour synaesthetes outperformed non-synaesthetes. Dual

synaesthetes (synaesthetes who experience both touch and colour as evoked

sensations) outperformed the non-synaesthetic control group on both tasks, and

outperformed colour synaesthetes on the tactile perception task. These findings imply

that sensory enhancement in the concurrent perceptual system may be a general

property of synaesthesia and show that the presence of synaesthesia exerts a wider

influence over sensory processing.

The mechanisms which underpin sensory enhancement in synaesthesia are

likely to reflect differences in brain development as a function of synaesthesia (which

may be either a cause or consequence of synaesthesia). As noted previously, the

neural mechanisms which underpin synaesthesia are a subject of uncertainty, with

some authors suggesting that the condition may be due to additional structural

Chapter 9

166

connectivity (i.e. structural differences; Bargary and Mitchell, 2008; Rouw and

Scholte, 2007), others in favour of malfunctions in cortical inhibition (i.e. functional

but not structural differences; Cohen Kadosh and Henik, 2007; Cohen Kadosh and

Walsh, 2008; Grossenbacher and Lovelace, 2001), and others for a combination of

both (Smilek, Dixon, Cudahy, and Merikle, 2001). In principle, the findings of

enhanced sensory perception in synaesthesia could be accounted for by any of these

approaches (e.g. mechanisms of inhibition may unmask local anatomical pathways,

while altered connectivity may result in alternative local and widespread anatomical

pathways which could facilitate performance). Moreover, compatible findings of

sensory-enhancement in the deprived brain would suggest that both aberrant

connectivity and malfunctions in cortical inhibition could play a role in the sensory-

enhancement found in synaesthetes. For example, temporary enhancements in tactile

acuity can occur following blindfolding, and are thought to be due to fast-acting

unmasking of existing connections to maintain functional behaviour. In comparison,

tactile acuity can also be enhanced in blindness, which is though to reflect sustained

unmasking of existing connections leading to new local and widespread anatomical

pathways (a slow-acting mechanism; Pascual-Leone, Amedi, Fregni, and Merabet,

2005). Future studies should investigate how and whether mechanisms of cortical

inhibition and connectivity interact in synaesthesia, and assess the possibility that

increased structural connectivity in synaesthesia (Rouw and Scholte, 2007) may

reflect sustained unmasking of existing connections (Cohen Kadosh et al., 2009). A

further intriguing possibility would be to examine the interaction between fast-acting

cortical unmasking mechanisms in sensory-enhancement following deprivation and

synaesthesia. For example, if temporary enhancements in tactile acuity following

blindfolding are linked to perceptual unmasking, and synaesthesia is linked to reduced

Chapter 9

167

cortical inhibition (and therefore increased unmasking; c.f. Cohen Kadosh and Walsh,

2006), then one may suspect that grapheme-colour synaesthetes will show more rapid

enhancements in tactile acuity following sensory deprivation compared to non-

synaesthetes.

9.5 What are the implications of heightened sensorimotor simulation in

mirror-touch synaesthesia for social cognition?

In addition to studies investigating the neurocognitive basis of synaesthesia,

there is a growing interest in using the condition to inform us about normal models of

cognitive processing (Mattingley and Ward, 2006). This approach rests on the

assumption of neuropsychology, where one is able to use a symptom affecting the

normal system to inform us about the function of the normal system. In the case of

synaesthesia the symptom is a positive one and in the case of mirror-touch

synaesthesia one is assessing the impact of facilitated sensorimotor simulation

mechanisms on cognition. Moreover, functional brain imaging has linked mirror-

touch synaesthesia to heightened neural activity in a network of brain regions which

are also activated in non-synaesthetic control subjects when observing touch to others

(the mirror-touch system; Blakemore et al., 2005). The mirror-touch system is

comprised of brain areas active during both the observation and passive experience of

touch (including primary and secondary somatosensory cortices, and premotor cortex;

Blakemore et al., 2005; Ebisch et al., 2008; Keyers et al., 2004) and has been

suggested to be a candidate neural mechanism to aid social cognition through

sensorimotor simulation (Gallese, 2006; Gallese and Goldman, 1998; Keysers and

Gazzola, 2006; Oberman and Ramachandran, 2007). Accounts of social cognition

involving sensorimotor simulation contend that, in order to understand another’s

emotions and physical states, the perceiver must map the bodily state of the observer

Chapter 9

168

onto the same representations involved in experiencing the perceived state (Adolphs,

2002; Adolphs, 2003; Damasio, 1990; Gallese, Keysers, and Rizzolatti, G, 2004;

Gallese, 2006; Gallese and Goldman, 1998; Goldman, and Sripada, 2005; Keysers and

Gazzola, 2006; Oberman and Ramachandran, 2007). There is now a good degree of

evidence from functional brain imaging, neuropsychological, transcranial magnetic

stimulation, and electrophysiological studies to indicate a role for sensorimotor

resources in this process (Adolphs, Damasio, Tranel, Cooper, and Damasio, 2000;

Carr, Iacoboni, Dubeau, Mazziotta, and Lenzi, 2003; Dapretto et al., 2006;

Hennenlotter et al., 2006; Jabbi, Swart, and Keysers, 2007; Kesler-West et al., 2001;

Leslie, Johnsen-Frey, and Grafton, 2004; Montgomery and Haxby, 2008;

Nummenmaa, Hirvonen, Parkkola, and Hietanen, 2008; Oberman, Winkielman, and

Ramachandran, 2007; Pitcher, Garrido, Walsh, and Duchaine., 2008; Schulte-Ruther,

Markowitsch, Fink, and Piefke, 2007; Seitz et al., 2008; van der Gaag, Minderaa, and

Keysers, 2007; Warren et al., 2006; Wildgruber et al., 2005; Winston, O’Doherty, and

Dolan, 2003). The studies reported in chapters 4 and 5 attempted to use mirror-touch

synaesthesia as a model to inform us about the impact of facilitated sensorimotor

simulation on empathy and facial expression recognition.

In chapter 4, the empathic abilities of mirror-touch synaesthetes were

compared to control participants in two studies. The first study used a self-report

empathy questionnaire (the empathy quotient – Baron-Cohen and Wheelwright, 2004)

and showed that mirror-touch synaesthetes have higher levels of emotional reactive

empathy (i.e. instinctive responses to others emotions), but do not have higher levels

of cognitive or social components of empathy compared with control participants.

Higher levels of empathy were not observed in other-variants of synaesthesia,

indicating that it relates specifically to mirror-touch synaesthesia (and the mechanisms

Chapter 9

169

which underpin it). In the second study, these findings were replicated using the same

measure of empathy and on another measure of empathy (Davis, 1980). The findings

from the second study also confirmed that differences observed in empathy between

mirror-touch synaesthetes and non-synaesthetes were linked to ‘other’ rather than

‘self’ orientated reactions, and found a significant relationship between the openness

to experience personality trait and emotional empathy. Mirror-touch synaesthetes

were also shown to significantly differ from non-synaesthetes on the openness to

experience personality trait. These findings appear consistent with accounts of

empathy that posit a role for sensorimotor simulation mechanisms (Gallese, 2006;

Gallese and Goldman, 1998; Keysers and Gazzola, 2006; Oberman and

Ramachandran, 2007) and with functional brain imaging (Nummenmaa et al., 2008)

and neuropsychological findings (Shamay-Tsoory, Aharon-Peretz, and Perry, 2009)

which indicate that emotional empathy is linked more closely to sensorimotor

simulation of another’s state than cognitive empathy.

In addition to the differences in emotional empathy reported in chapter 4, the

study presented in chapter 5 sought to establish whether mirror-touch synaesthetes

differed in another aspect of social perception, namely facial expression recognition.

The findings from this study showed that mirror-touch synaesthetes outperformed

non-synaesthete control participants on tasks of facial expression recognition, but not

control tasks involving identity recognition and identity perception. These findings

are consistent with functional brain imaging (Carr et al., 2001; Hennenlotter et al.,

2005; Leslie, Johnsen-Frey, and Grafton, 2004; Montgomery and Haxby, 2008; van

der Gaag, Minderaa, and Keysers, 2007; Winston, O’Doherty, and Dolan, 2003),

neuropsychological (Adolphs et al., 2000; Adolphs, Baron-Cohen, and Tranel, 2002)

and TMS findings (Pitcher et al., 2008; Pourtois et al., 2004) which indicate a central

Chapter 9

170

role for sensorimotor resources in facial expression recognition, and suggest that

facilitated sensorimotor simulation appears to be linked with heightened facial

expression sensitivity and emotional empathy (Chapter 4).

A number of predictions can also be drawn from the evidence of heightened

emotion sensitivity in mirror-touch synaesthesia. For example, given the evidence

that mirror-touch synaesthesia has been reported to be linked to heightened activity in

the mirror-touch system activated by us all when observing touch to others, one may

predict that the extent of activity in this system may correlate with levels of emotional

reactive empathy (but not cognitive empathy – where mirror-touch synaesthetes did

not significantly differ from controls). Further, one may suspect that the level of

activity in sensorimotor cortices when perceiving touch to others should also correlate

with an individual subject’s facial expression recognition abilities. These possibilities

are to be addressed with future studies.

Another interesting point for consideration is whether mirror-touch

synaesthesia represents a distinct population or the tail-end of a distribution of how

much we simulate / empathize with of others6. As noted previously, mirror-touch

synaesthesia shares a number of similarities with other types of developmental

synaesthesia. For example, there is a tendency for mirror-touch synaesthetes to have

other family members with synaesthesia and multiple types of synaesthesia, indicating

a genetic component to the condition. Yet, the principles which bias what type of

synaesthesia will or will not be developed are largely unclear. One could envisage

that if individual differences in emotional sensitivity are in someway heritable /

6 I would like to thank Prof. Christian Keysers for raising and corresponding with me on this issue.

Chapter 9

171

separate from a ‘specific allele’ for synaesthesia7 then heightened emotional

sensitivity may bias these individuals to a form of interpersonal synaesthesia.

9.6 What are the implications of suppressing sensorimotor resources on

expression recognition abilities in healthy adults?

While the findings from chapter 5 assessed the influence of facilitated

sensorimotor simulation on expression recognition, the studies presented in chapters 7

and 8 assessed what impact suppressing sensorimotor resources has on the expression

recognition abilities of healthy adults.

In chapter 7, a continuous theta burst (cTBS) TMS paradigm was used to

suppress cortical activity in the right primary somatosensory cortex (rSI), right lateral

premotor cortex (rPM), and the vertex. Participants completed two tasks. In

experiment 1, participants were asked to complete a same-different auditory

expression recognition task. In experiment 2, a new group of participants were asked

to complete a same-different auditory identity task. Stimuli were non-verbal auditory

emotions (amusement, disgust, fear and sadness), adapted from a previous study

documenting the role of sensorimotor resources in non-verbal auditory emotion

recognition (Warren et al., 2006), and were identical in each task. A comparison

across tasks and sites, revealed that cTBS targeted at rSI and rPM impaired

participants’ abilities to discriminate between the auditory emotions, but not

identities, of others. This was not found to be the case following cTBS to the vertex

(cTBS control site). Therefore consistent with simulation accounts of expression

recognition (Adolphs, 2002; Adolphs, 2003; Damasio, 1990; Gallese, Keysers, and

Rizzolatti, G, 2004; Goldman, and Sripada, 2005; Keysers and Gazzola, 2006),

7 The genetic mechanisms for synaesthesia are of course likely to be more complex than this (e.g. see Asher et al., 2009).

Chapter 9

172

suppressing sensorimotor activity resulted in a task-specific impairment on

participants’ abilities to discriminate emotions but not identities from vocal signals.

These findings add to previous studies documenting a pivotal role for sensorimotor

cortices in facial expression recognition (Adolphs et al., 2000; Pitcher et al., 2008;

Pourtois et al., 2004) and suggest that they are central to the discrimination of emotion

across modalities.

In chapter 8, I again used a cTBS paradigm, but investigated the role of neural

activity in rSI and the right inferior frontal gyrus (rIFG) on facial expression

recognition. Previous findings had indicated that rSI activity is central to facial

expression discrimination (Adolphs et al., 2000; Pitcher et al., 2008; Pourtois et al.,

2004; Winston et al. 2003), but whether this was expression-general or expression-

specific remained a point of dispute. Further, rIFG activity had been reported in a

number of brain imaging experiments investigating the neural correlates of facial

expression recognition or evaluation (Carr et al., 2001; Dapretto et al., 2006;

Hennenlotter et al., 2006; Kesler-West et al., 2001; Seitz et al., 2008), but whether the

region is critical for the facial expression recognition abilities of healthy adults

remained to be clarified. Using a four-forced-choice expression recognition task, the

findings indicated that cTBS to rSI impaired the recognition of emotional facial

expressions (happy and sad) relative to neutral expressions. This is consistent with

previous findings documenting a central role for somatosensory cortices in facial

expression discrimination (Adolphs et al., 2000; Pitcher et al., 2008; Pourtois et al.,

2004) and compliments the findings of chapter 7 by indicating that rSI activity is

involved in discriminating emotional expressions across modalities. No cTBS effect

was observed following stimulation to rIFG or right V5 / MT (visual control site).

This is interesting given that rIFG is considered to be a part of the human mirror

Chapter 9

173

system (Rizzolatti and Craighero, 2004) and has been reported in a number of fMRI

studies on facial expression evaluation (Carr et al., 2001; Dapretto et al., 2006;

Hennenlotter et al., 2006; Kesler-West et al., 2001; Seitz et al., 2008). The lack of

impairment following cTBS to rIFG would suggest that although this region may be

involved in facial expression recognition (Carr et al., 2001; Dapretto et al., 2006;

Hennenlotter et al., 2006; Kesler-West et al., 2001; Seitz et al., 2008), it may not be

critical to the process. Other components of motor simulation may play a more

critical in facial expression discrimination (e.g. premotor cortex as studied in chapter

7) and future studies should address this.

Further possibilities for future research include combining TMS with other

methodologies to consider the role of cortico-cortical interactions play in

discriminating another’s expressions. Moreover, while the effects of online TMS are

spatially discrete, the effects of offline stimulation will spread to other cortical areas

along the greatest lines of conductivity from the stimulated area. By combining cTBS

with fMRI paradigms one should be able to assess any secondary effects of cTBS on

other regions involved in expression recognition.

9.7 General Summary

In summary, this thesis has investigated the neurocognitive and perceptual

profiles of mirror-touch synaesthesia (Chapters 2-5). I have provided a

neurocognitive model of the condition (which provides testable predictions for future

studies) and used mirror-touch synaesthesia as a tool to inform us about the

neurocognitive mechanisms of synaesthesia more generally. The studies presented

have also used mirror-touch synaesthesia as a model to inform us about the impact

that heightened sensorimotor activity has on social cognition (Chapters 4 and 5), and

the findings from these studies are compatible with research presented in chapters 7

Chapter 9

174

and 8 which investigated the impact of suppression of sensorimotor activity on

expression recognition. This has resulted in a number of interesting possibilities for

further studies, some of which are currently in progress and others open to be

conducted.

References

175

REFERENCES

Adolphs, R. (2002). Neural systems for recognizing emotion. Current Opinion in

Neurobiology, 12, 169-77.

Adolphs, R. (2003). Cognitive neuroscience of human social behaviour. Nature

Reviews Neuroscience, 4, 165-178.

Adolphs, R., Baron-Cohen, S., and Tranel, D. (2002). Impaired recognition of social

emotions following amygdala damage. Journal of Cognitive Neuroscience,

14, 1264-1274.

Adolphs, R., Damasio, H., Tranel, D., Cooper, G., and Damasio, A. R. (2000). A role

for somatosensory cortices in the visual recognition of emotion as revealed by

three-dimensional lesion mapping. Journal of Neuroscience, 20, 2683-2690.

Aghajanian, G. K., and Marek, G. J. (1999). Serotonin and hallucinogens.

Neuropsychopharmacology, 21, 16S-23S.

Aleman, A., Rutten, G. J., Sitskoorn, M. M., Dautzenberg, G., and Ramsey, N. F.

(2001). Activation of striate cortex in the absence of visual stimulation: An

fMRI study of synesthesia. Neuroreport, 12, 2827-2830.

Aluja, A., Garcia, O., and Garcia, L. F. (2002). A comparative study of Zuckerman’s

three structural models for personality through the NEO-PI-R, ZKPQ-III-R,

EPQ-RS and Goldberg’s 50-bipolar adjectives. Personality and Individual

Differences, 33, 713-725.

Amassian, V.E., Cracco, R.Q., Maccabee, P.J., Cracco, J.B., Rudell, A.P., and Eberle,

L. (1993). Unmasking human visual perception with the magnetic coil and its

relationship to hemispheric asymmetry. Brain Res, 605, 312-316.

References

176

Amassian, V.E., Eberle, L., Maccabee, P., and Cracco, R.Q. (1992). Modeling

magnetic coil excitation of human cerebral cortex with a peripheral nerve

immersed in a brain-shaped volume conductor: The significance of fiber

bending in excitation. Electroencephalography and Clinical Neurophysiology,

85, 291–301.

Anderson, A. K., Spencer, D. D., Fullbright, R. K., and Phelps, E. A. (2000).

Contribution of the anteromedial temporal lobes to the evaluation of facial

emotion. Neuropsychology, 14, 526-536.

Armel, K. C., and Ramachandran, V. S. (2001). Acquired synethesia in retinitis

pigmentosa. Neurocase, 5, 293-296.

Arzy, S., Thut, G., Mohr, C., Michel, C. M., and Blanke, O. (2006). Neural basis of

embodiment: Distinct contributions of the temporoparietal junction and the

extrastriate body area. Journal of Neuroscience, 26, 8074-8081.

Ashbridge,E., Walsh,V., and Cowey, A. (1997). Temporal aspects of visual search

studied by transcranial magnetic stimulation. Neuropsychologia, 35, 1121-

1131.

Asher, J. E., Lamb, J. A., Brocklebank, D., Cazier, J. B., Maestrini, E., Addis, L., et

al. (2009). A whole-genome scan and fine-mapping linkage study of auditory-

visual synesthesia reveals evidence of linkage to chromosomes 2q24, 5p33,

6p12, and 12p12. Am J Hum Genet, 84, 279-285.

Ashley, V., Vuilleumier, P., and Swick, D. (2004). Time course and specificity of

event-related potentials to emotional expressions. NeuroReport, 15, 211-216.

Augstine, J. R. (1996). Circuitry and functional aspects of the insular lobe in primates

including humans. Brain Res Brain Res Rev., 22, 229-244.

References

177

Avenanti, A., Bueti, D., Galati, G., and Aglioti, S. M. (2005). Transcranial magnetic

stimulation highlights the sensorimotor side of empathy for pain. Nature

Neuroscience, 8, 955-960.

Avenanti, A., Minio-Paluello, I., Bufalari, I., and Aglioti, S. M. (2009). The pain of a

model in the personality of an onlooker: Influence of state-reactivity and

personality traits on embodied empathy for pain. Neuroimage, 44, 275-283.

Banissy, M. J., Cohen Kadosh, R., Maus, G. W., Walsh, V., and Ward, J. (2009).

Prevalence, characteristics, and a neurocognitive model of mirror-touch

synaesthesia. Experimental Brain Research, 198, 261-272.

Banissy, M. J., MacDonald, L., Ward, J., Walsh, V., Haggard, P., and Longo, M. R.

(in preparation). The rubber hand illusion in mirror-touch synaesthesia.

Banissy, M. J., Stewart, L., Ward, J., Walsh, V., and Kanai, R. (in prepartion).

Neuroanatomical correlates of tone-colour synaesthesia.

Banissy, M. J., and Ward, J. (2007). Mirror-touch synaesthesia is linked with

empathy. Nature Neuroscience, 10, 815-816.

Bargary, G., and Mitchell, K. J. (2008). Synaesthesia and cortical connectivity.

Trends in Neurosciences, 31, 335-342.

Barker, A.T., Freeston, I.L., Jalinous, R., Merton, P.A., and Morton, H.B. (1985).

Magnetic stimulation of the human brain. J. Physiol. (Lond.), 369, 3P.

Barnett, K. J., Feeney, J., Gormley, M., and Newell, F. N. (2009). An exploratory

study of linguistic-colour assocaitions across languages in multilingual

synaesthetes. Q J Exp Psychol, 62, 1343-1354.

Barnett, K. J., Finucane, C., Asher, J. E., Bargary, G., Corvin, A. P., Newell, F. N., et

al. (2008). Familial patterns and the origins of individual differences in

synaesthesia. Cognition, 106, 871-893.

References

178

Barnett, K. J., Fox, J. J., Molholm, S., Kelly, S. P., Shalgi, S., Mitchell, K. J., and

Newell, F. N. (2008). Differences in early sensory-perceptual processing in

synaesthesia: A visual evoked potential study. Neuroimage, 43, 605-613.

Barnett, K. J., and Newell, F. N. (2008). Synaesthesia is associated with enhanced,

self rated imagery. Consciousness and Cognition, 17, 1032-1039.

Baron-Cohen, S., Burt, L., Smith-Laittan, F., Harrison, J., and Bolton, P. (1996).

Synaesthesia: Prevalence and familiality. Perception, 25, 1073-1079.

Baron-Cohen, S., Harrison, J., Goldstein, L. H., and Wyke, M. (1993). Coloured

speech perception: Is synaesthesia what happens when modularity breaks

down. Perception, 22, 419-426.

Baron-Cohen, S., Richler, J., Biarya, D., Gurunathan, N., and Wheelwright, S. (2003).

The systemizing quotient: An investigation of adults with Aspergers syndrome

or high-functioning autism, and normal sex differences. Philos Trans R Soc

Lond B Biol Sci., 358, 361-374.

Baron-Cohen, S. and Wheelwright, S. (2004). The empathy quotient: An

investigation of adults with asperger syndrome or high functioning autism, and

normal sex differences. J. Aut. and Dev. Disorders, 34, 163-175.

Baron-Cohen, S., Wyke, M. A., and Binnie, C. (1987). Hearing words and seeing

colours: An experimental investigation of synaesthesia. Perception, 16, 761-

767.

Barrick, M. R., and Mount, R. K. (1991). The Big Five personality dimensions and

job performance: A meta-analysis. Personnel Psychology, 44, 1-26.

Basser, P. J. (1995). Inferring microstructural features and the physiological state of

tissues from diffusion-weighted images. NMR Biomed., 8, 333-344.

References

179

Batson, C. D. (1991). The altruism question: Towards a social-psychology answer.

Erlbaum, Hillside, NJ.

Beck, D., Muggleton, N., Walsh, V., and Lavie, N. (2006). The right parietal cortex

plays a critical role in change blindness. Cereb. Cortex, 16, 712-717.

Beeli, G., Esslen, M., and Jäncke, L. (2005). Synaesthesia: When coloured sounds

taste sweet. Nature, 434, 38.

Beeli, G., Esslen, M., and Jäncke, L. (2008). Time course of neural activity correlated

with colored-hearing synesthesia. Cereb Cortex, 18, 379-385.

Berlucchi, G., and Aglioti, S. (1997). The body in the brain: Neural basis of corporeal

awareness. Trends Neurosci., 20, 560-564.

Bestmann, S., Baudewig, J., Siebner, H.R., Rothwell, J.C., and Frahm, J. (2004).

Functional MRI of the immediate impact of transcranial magnetic stimulation

on cortical and subcortical motor circuits. European Journal of Neuroscience,

19, 1950-1962.

Bjoertomt, O., Cowey, A., and Walsh, V. (2002). Spatial neglect in near and far space

investigated by repetitive transcranial magnetic stimulation. Brain, 125, 2012-

2022.

Blake, R, Palmeri, T.J., Marois, R., and Kim, C. Y. (2004). On the perceptual reality

of synesthetic color. In: L. Robertson and N. Sagiv, Editors, Synesthesia:

Perspectives from Cognitive Neuroscience, Oxford University Press, Oxford

(2004), pp. 47–73.

Blakemore, S. J., Bristow, D., Bird, G., Frith, C., and Ward, J. (2005). Somatosensory

activations during the observation of touch and a case of vision-touch

synaesthesia. Brain, 128, 1571-1583.

References

180

Blanke, O., Landis, T., Spinelli, L., and Seeck, M. (2004). Out-of-body experiences

and autoscopy of neurological origin. Brain, 127, 243-258.

Blanke, O., and Mohr, C. (2005). Out-of-body experience, heautoscopy, and

autoscopic hallucinations of neurological origin: Implications for

neurocognitive mechanisms of corporeal awareness and self-consciousness.

Brain Res Brain Res Rev., 50, 184-199.

Blanke, O., Ortigue, S., Landis, T., and Seeck, M. (2002). Stimulating illusory own

body perceptions. Nature, 419, 269-270.

Bohning, D.E., Shastri, A., McConnell, K.A., Nahas, Z., Lorberbaum, J., and Roberts,

D. (1999). A combined TMS/fMRIstudy of intensity-dependent TMS over

motor cortex. Biol Psychiatry, 45, 385–394.

Botvinick, M., and Cohen, J. (1998). Rubber hands feel touch that I see. Nature, 391,

756.

Bradshaw, J. L., and Mattingley, J. B. (2001). Allodynia: A sensory analogue of

motor mirror neurons in a hyperaesthetic patient reporting instantaneous

discomfort to another’s perceived sudden minor injury? J Neurol Neurosurg

Psychiatry, 70, 135-136.

Bremmer, F., Schlak, A., Shah, N. J., Zafiris, O., Kubishik, M., Hoffmann, K., et al.

(2001). Polymodal motion processing in posterior parietal and premotor

cortex: A human fMRI study strongly implies equivalences between humans

and monkeys. Neuron, 29, 287-296.

Bruce, V., and Young, A. W. (1986). Understanding face recognition. Br. J. Psychol.,

77, 305-327.

Brugger, P. (2002). Reflective mirrors: Persepective-taking in autoscopic phenomena.

Cognitive Neuropsychiatry, 7, 179-194.

References

181

Bryant, G., and Barrett, H. C. (2007). Recognizing intentions in infant-directed

speech: Evidence for universals. Psychological Science, 18, 746-751.

Buccino, G., Binkofski, F., Fink, G. R., Fadiga, L., Fogassi, L., Gallese, V., et al.

(2001). Action observation activates premotor and parietal areas in a

somatotopic manner. European Journal of Neuroscience, 13, 400-404.

Bufalari, I., Aprile, T., Avenanti, A., Di Russo, F., and Aglioti, S. M. (2007).

Empathy for pain and touch in the human somatosensory cortex. Cereb

Cortex, 17, 2553-2661.

Bünning, S., and Blanke, O. (2005). The out-of body experience: Precipitating factors

and neural correlates. Prog Brain Res., 150, 331-350.

Burrack, A., Knoch, D., and Brugger, P. (2006). Mitempfindung in synaesthetes: Co-

incidence or meaningful association? Cortex, 42, 151-154.

Bushara, K. O., Grafman, J., and Hallett, M. (2001). Neural correlates of auditory-

visual stimulus onset asynchrony detection. J Neurosci, 21, 300-304.

Calder, A. J., Keane, J., Lawrence, A. D., and Manes, F. (2004). Impaired recognition

of anger following damage to the ventral striatum. Brain, 127, 1958-1969.

Calder, A. J., Keane, J., Manes, F., Antoun, N., and Young, A. W. (2000). Impaired

recognition and experience of disgust following brain injury. Nature

Neuroscience, 3, 1077-1078.

Calder, A. J., Lawrence, A. D., and Young, A. W. (2001). The neuropsychology of

fear and loathing. Nature Reviews Neuroscience, 2, 352-363.

Calder, A. J., and Young, A. W. (2005). Understanding the recognition of facial

identity and facial expression. Nature Reviews Neuroscience, 6, 641-651.

Campana, G., Cowey, A., and Walsh, V. (2002). Priming of motion direction and

area V5/MT: A test of perceptual memory. Cereb Cortex, 12, 663-669.

References

182

Carr, L., Iacoboni, M., Dubeau, M. C., Mazziotta, J. C., and Lenzi, G. L. (2003).

Neural mechanisms of empathy in humans: A relay from neural systems for

imitation to limbic areas. Proc. Natl. Acad. Sci., 100, 5497-5502.

Cohen, L., and Dehaene, S. (2004). Specialization within the ventral stream: The case

for the visual word form area. Neuroimage, 22, 466-476.

Cohen Kadosh, R., Cohen Kadosh, K., and Henik, A. (2007). The neural correlate of

bidirectional synaesthesia: A combined ERP and fMRI study. Journal of

Cognitive Neuroscience, 19, 2050-2059.

Cohen Kadosh, R., and Henik, A. (2006). When a line is a number: Colour yields

magnitude information in a digit-colour synaesthete. Neuroscience, 137, 3-5.

Cohen Kadosh, R., and Henik, A. (2007). Can synaesthesia research inform cognitive

science? Trends in Cognitive Sciences, 11, 177-184.

Cohen Kadosh, R., Henik, A., and Walsh, V. (2007). Small is bright and big is dark

in synaesthesia. Current Biology, 17, R834-R835.

Cohen Kadosh, R., Henik, A., Catena, A., and Walsh, V. (2009). Induced Cross-

Modal Synesthetic Experience without Abnormal Neuronal Connections.

Psychological Science, 20, 258-265.

Cohen Kadosh, R., Tzelgov, J., and Henik, A. (2008). A synethetic walk on the

mental number line: The size effect. Cognition, 106, 548-557.

Cohen Kadosh, R., and Walsh, V. (2006). Cognitive neuroscience: Rewired or

crosswired brains. Current Biology, 16, 962-963.

Cohen Kadosh, R., and Walsh, V. (2008). Synaesthesia and Cortical Connections:

Cause or Correlation? Trends in Neurosciences, 31, 549-550.

Colby, C. L. (1998). Action-orientated spatial reference frames in cortex. Neuron,

20, 15-24.

References

183

Coslett, B. (1998). Evidence for a disturbance of the body schema in neglect. Brain

and Cognition, 37, 527-544.

Costa, P. T., and McCrae, R. R. (1985). The NEO Personality Inventory manual.

Odessa, FL: Psychological Assessment Resources.

Cowey, A., and Gross, C. G. (1970). Effects of foveal prestriate and inferotemporal

lesions on visual discrimination by rhesus monkey. Experimental Brain

Research, 11, 128-144.

Craig, A. D. (2002). How do you feel? Interoception: The sense of the physiological

condition of the body. Nature Reviews Neuroscience, 3, 655-666.

Crawford, J. R., and Garthwaite, P. H. (2002). Investigation of the single case in

neuropsychology: Confidence limits on the abnormality of test scores and test

scores differences. Neuropsyhcologia, 40, 1196-1208.

Cytowic, R. E., and Wood, F. B. (1982). Synaesthesia II: Psychophysical relations in

the synaesthesia of geometrically shaped taste and colored hearing. Brain and

Cognition, 1, 36-49.

Damasio, A. R. (1990). Time-locked multiregional retroactivation: A systems-level

proposal for the neural substrates of recall and recognition. In Neurobiology

of Cognition, Edited by Eimas, P. D., and Galaburda, A. M. MIT Press,

Cambridge, 25-62.

Dapretto, M., Davies, M. S., Pfeifer, J. H., Scott, A. A., Sigman, M., Bookheimer, S.

Y., et al. (2006). Understanding emotions in others: Mirror neuron dysfunction

in children with autism spectrum disorders. Nature Neuroscience, 9, 28-30.

Davidoff, J. (2001). Language and perceptual categorisation. Trends in Cognitive

Sciences, 5, 382-387.

References

184

Davis, M. H. (1980). A multidimensional approach to individual differences in

empathy. Catal. Sel. Doc. Psychol., 10, 85.

Davis, M. H. (1994). Empathy: A social psychology approach. Colorado: Westview

Press.

Day, S. (2005). In: L. Robertson and N. Sagiv, Editors, Synesthesia: Perspectives

from Cognitive Neuroscience, Oxford University Press, Oxford (2004),

Decety, J., and Jackson, P. L. (2004). The functional architecture of human empathy.

Behav Cogn Neurosci Rev., 3, 71-100.

del Barrio, V., Aluja, A., and Garcia, L. F. (2004). Relationship between empathy and

the Big Five personality traits in a sample of Spanish adolescents. Social

Behavior and Personality, 32, 677-681.

De Waal, F. B. (2007). Putting the altruism back into altruism: The evolution of

empathy. Ann Rev Psychol, 59, 1-22.

Di Lazzaro, V., Pilato, F., Saturno, A., Oliviero, M., Dileone, M., Mazzone, P., et al.

(2005). Theta-burst repetitive transcranial magnetic stimulation suppresses

specific excitatory circuits in the human motor cortex. J. Physiology, 565 945-

950.

Dietrich, S., Szameitat, D., Ackermann, H., and Alter, K. (2006). How to disentangle

lexical and prosodic information? Psychoacoustic studies on the processing of

vocal interjections. Prog Brain Res. 156, 295-302.

Dimberg, U., Thunberg, M., Elmehed, K. (2000). Unconcious facial reactions to

emotional facial expressions. Psychological Science, 11, 86-89.

Dixon, M. J., Smilek, D., and Merikle, P. M. (2004). Not all synaesthetes are created

equal: Projector versus associator synaesthetes. Cogn Affect Behav Neurosci,

4, 335-343.

References

185

Donner, T. H., Kettermann, A., Diesch, E., Ostendorf, F., Villringer, A., and Brandt,

S. A. (2002). Visual feature and conjunction searches of equal difficulty

engage only partially overlapping frontoparietal networks. Neuroimage, 15,

16-25.

Downing, P.E., Jiang, Y., Shuman, M., and Kanwisher, N. A. (2001). Cortical area

selective for visual processing of the human body. Science, 293, 2470-2473.

Duchaine, B., and Nakayama, K. (2006). The Cambridge Face Memory Test: Results

from neurologically intact individuals and an investigation of its validity using

inverted performance and prosopagnosic subjects. Neuropsychologia, 16,

166-173.

Duchaine, B., Germine, L., and Nakayama, K. (2007). Family resemblance: Ten

family members with prosopagnosia and within-class object agnosia.

Cognitive Neuropsychology, 24, 419-430.

Duhamel, J. R., Colby, C. L., and Goldberg, M. E. (1998). Ventral intraparietal area

of the macaque: Congruent visual and somatic response properties. Journal

of Neurophysiology, 79. 126-136.

Dumoulin, S.O., Bittar, R. G., Kabani, N. J., Baker, C. L. Jr, Le Goualher, G., Bruce

Pike, G., et al. (2000). A new anatomical landmark for reliable identification

of human area V5 / MT: A quantative analysis of sulcal patterning. Cereb

Cortex, 10, 454-463.

Eagleman, D. M., Kagan, A. D., Nelson, S. S., Sagaram, D., and Sarma, A. K. (2007).

A standardized test battery for the study of synaesthesia. Journal of

Neurosciences Methods, 159, 139-145.

References

186

Ebisch, S. J. H., Perucci, M. G., Ferretti, A., Del Gratta, C., Luca Romani, G., and

Gallese, V. (2008). The sense of touch: Embodied simulation in a visuo-

tactile mirroring mechanism for observed animate or inanimate touch. Journal

of Cognitive Neuroscience, 20, 1611-1623.

Eimer, M., and Holmes, A. (2002). An ERP study on the time course of emotional

face processing. NeuroReport, 13, 427-431.

Eimer, M., and Holmes, A. (2007). Event-related brain potential correlates of

emotional face processing. Neuropsychologia, 45, 15-31.

Eimer, M., Holmes, A., and McGlone, F. (2003). The role of spatial attention in the

processing of facial expression: An ERP study of rapid brain responses to six

basic emotions. Cognitive, Affective and Behavioural Neuroscience, 3, 97-110.

Eisenberg, N. (2000). Emotion, regulation, and moral development. Annu Rev

Psychol., 51, 665-697.

Ekman, P., and Friesen, W. V. (1976). Pictures of facial affect. Palo Alto, CA:

Consulting Psychologists.

Elias, L. J., Saucier, D. M., Hardie, C., and Sart, C. E. (2003). Dissocaiting semantic

and perceptual components of synaesthesia: Behavioural and functional

neuroanatomical investigations. Cognitive Brain Research, 16, 232-237.

Esterman, M., Verstynen, T., Ivry, R. B., and Robertson, L. C. (2006). Coming

unbound: Disrupting automatic integration of synesthetic color and graphemes

by transcranial magnetic stimulation of the right parietal lobe. J Cogn

Neuroci, 18, 1570-1576.

Fadiga, L., Fogassi, L., Pavesi, G., and Rizzolatti, G. (1995). Motor facilitation during

action observation: A magnetic stimulation study. J. Neurophsyiology, 73,

2608-2611.

References

187

Fink, G. R., Markowitsch, H. J., Reinkemeier, M., Bruckbauer, T., Kessler, J., and

Heiss, W. D. (1996). Cerebral representations of one’s own past: Neural

networks involved in autobiographical memory. Journal of Neuroscience, 16,

4275-4282.

Flynn, F. G., Benson, D. F., and Ardila, A. (1999). Anatomy of the insula –

Functional and clinical correlates. Aphasiology, 13, 55-78.

Franz, E. A., Ford, S., and Werner, S. (2007). Brain and cognitive processes of

imitation in bimanual situations: Making inference about mirror neuron

systems. Brain research, 1145, 138-149.

Friedman-Hill, S. R., Robertson, L. C., and Treisman, A. (1995). Parietal

contributions to visual feature binding: Evidence from a patient with bilateral

lesions. Science, 269, 853-855.

Gallagher, S. (1995). Body schema and intentionality. In J. L. Bermudez, A. Marcel,

and N. Eilan (Eds.), The body and the self, Cambridge, MA: MIT Press.

Gallese, V. (2006). Intentional atonement: A neurophysiological perspective on

social cognition and its disruption in autism. Brain Res Cog Brain Res., 1079,

15-24.

Gallese, V., Fadiga, L., Fogassi, L., and Rizzolatti, G. (1996). Action recognition in

the premotor cortex. Brain, 119, 593-609.

Gallese, V., and Goldman, A. (1998). Mirror neurons and simulation theory of mind

reading. Trends in Cognitive Sciences, 12, 493-501.

Gallese, V., Keysers, C., and Rizzolatti, G. (2004). A unifying view of the basis of

social cognition. Trends in Cognitive Sciences, 8, 396-403.

Galton, F. (1880). Visualised numerals. Nature (1880), 252-256.

References

188

Garrido, L., Furl, N., Draganski, B., Weiskopf, N., Stevens, J., Tan, G. C-Y., et al. (in

press) VBM reveals reduced grey matter volume in the temporal cortex of

developmental prosopagnosics. Brain.

Gazzola, V., Aziz-Zadeh, L., and Keysers, C. (2006). Empathy and the somatotopic

auditory mirror system in humans. Current Biology, 16, 1824-1829.

Giummarra, M. J., Gibson, S. J., Georgiou-Karistianis, N., and Bradshaw, J. L.

(2008). Mechanisms underlying embodiment, disembodiment and loss of

embodiment. Neuroscience and Biobehavioural Reviews, 32, 143-160.

Goldberg, L. R. (1992). The development of markers for the Big-Five factor

structure. Psychological Assessment, 4, 26-42.

Goldman, A., and Sripada, C. S. (2005). Simulationist models of face-based emotion

recognition. Cognition, 94, 193-213.

Goldreich, D., and Kanics, I. M. (2003). Tactile acuity is enhanced in blindness.

Journal of Neuroscience, 23, 3439-3445.

Goller, A., Otten, L., and Ward, J. (2009). Seeing sounds and hearing colours: An

event-related potential study of audio-visual synaesthesia. Journal of

Cognitive Neuroscience, 21, 1869-1881.

Gray, J. A., Chopping, S., Nunn, J., Parslow, D., Gregory, L., Williams, S., et al.

(2002). Implications of synaesthesia for functionalism. Journal of

Consciousness Studies, 9, 5-31.

Gray, J. A., Parslow, D. M., Brammer, M. J., Chopping, S., Vythelingum, G. N., and

ffytche, D. H. (2006). Evidence against functionalism from neuroimaging of

the alien colour effect in synaesthesia. Cortex, 42, 309-318.

References

189

Grossenbacher, P.G., and Lovelace, C.T. (2001). Mechanisms of synaesthesia:

Cognitive and physiological constraints. Trends in Cognitive Sciences, 5, 36-

41.

Hadjikhani, N., and Roland, P. E. (1998). Cross-modal transfer of information

between the tactile and the visual representations in the human brain: A

positron emission tomographic study. Journal of Neuroscience, 18, 1072-

1084.

Haggard, P. (2006). Just seeing you makes me feel better: Interpersonal enhancements

of touch. Social Neuroscience, 1, 104-110.

Halligan, P. W., Hunt, M., Marshall, J. C., and Wade, D. T. (1996). When seeing is

feeling: Acquired synaesthesia or phantom touch? Neurocase, 2, 21-29.

Harris, J. A., Harris, I. M., Diamond, M. E., (2001). The topography of tactile

working memory. J Neurosci, 21, 8262-8269.

Haxby, J. V., Hoffman, E. A., and Gobbini, M. I. (2000). The distributed human

neural system for face processing. Trends in Cognitive Sciences, 4, 223-233.

Head, H., and Holmes, G. (1911-1912). Sensory disturbances from cerebral lesions.

Brain, 34, 102-254.

Hennenlotter, A., Schroeder, U, Erhard, P., Castrop, F., Haslinger, B., Stoecker, D.,

et al. (2005). A common neural basis for receptive and expressive

communication of pleasant facial affect. Neuroimage, 26, 581-591.

Hoekert, M., Bais, L., Kahn, R. S., and Aleman, A. (2008). Time course of the

involvement of the right anterior superior temporal gyrus and the right fronto-

parietal operculum in emotional prosody perception. PlOS One 3, doi:

10.1371/journal.pone.0002244.

References

190

Holmes, A., Vuilleumier, P., and Eimer, M. (2003). The processing of emotional

facial expressions is gated by spatial attention: Evidence from event-related

brain potentials. Cognitive Brain Research, 16, 174-184.

Hong, S. W. and Blake, R. (2008). Early visual mechanisms do not contribute to

synaesthetic colour experience. Vision Research, 48, 1018-1026.

Howell, D. C. (2002). Statistical Methods for Psychology. Thomson Learning: USA.

Huang, Y. Z., Edwards, M. J., Rounis, E., Bhatia, K. P., and Rothwell, J. C. (2005).

Theta burst stimulation of the human motor cortex. Neuron, 45, 201-206.

Huang, Y. Z., Rothwell, J. C., Lu, C. S., Wang, J., Weng, Y. H., Lai, S. C., et al.

(2009). The effect of continuous theta burst stimulation over premotor cortex

on circuits in primary motor cortex and spinal cord. Clin Neurophysiol., 120,

796-801.

Hubbard, E. M. (2007). A real red-letter day. Nat Neurosci, 10, 671-672.

Hubbard, E. M., Arman, A. C., Ramachandran V. S., and Boynton, G. M. (2005).

Individual differences among grapheme-color synesthetes: brain-behavior

correlations. Neuron, 45, 975–985.

Hubbard, E. M., Manohar, S., and Ramachandran, V. S. (2006). Contrast affects the

strength of synesthetic colors. Cortex, 42, 184-194.

Hubbard, E.M., and Ramachandran, V. S. (2003). Refining the experimental lever: A

reply to Shannnon and Pribram. J Consciousness Stud, 9, 77-84.

Hubbard, E.M., and Ramachandran, V. S. (2005). Neurocognitive mechanisms of

synaesthesia. Neuron, 48, 509-520.

Iacoboni, M., and Dapretto, M. (2006). The mirror neuron system and the

consequences of its dysfunction. Nat Rev Neurosci, 7, 942-951.

References

191

Iriki, A., Tanaka, M., and Iwamura, Y. (1996). Coding of modified body schema

during tool use by macaque postcentral neurones. Neuroreport, 7, 2325-2330.

Jabbi, M., Swart, M., and Keysers, C. (2007). Empathy for positive and negative

emotions in the gustatory cortex. Neuroimage, 34, 1744-1753.

Jäncke, L., Beeli, G., Eulig, C., and Hänggi, J. (2009). The neuroanatomy of

grapheme-color synethesia. Eur J Neurosci, 29, 1287-1293.

Jewanski, J., Day, S. A., and Ward, J. (2009). A colourful albino: The first

documented case of synaesthesia, by George Tobias Ludwig Sachs in 1812.

Journal of the History of Neurosciences, 18, 1-11.

John, O. P., Donahue, E. M., and Kentle, R. L. (1991). The Big Five Inventory –

Versions 4a and 54. Berkeley: University of California, Berkeley, Institute of

Personality and Social Research.

John, O. P. and Srivastava, S. (1999). The Big Five trait taxonomy: History,

measurement and theoretical perspectives. In Handbook of personality:

Theory and research (2nd ed.), Edited by Pervin, L., and John, O. P. New

York: Guilford.

Johnson, R., Burton, P., and Ro, T. (2006). Visually induced feelings of touch. Brain

Research, 1073-1074, 398-406.

Johnsen, K. O., and Philips, J. R. (1981). Tactile spatial resolution. I. Two-point

discrimination, gap detection, grating resolution, and letter recognition.

Journal of Neurophysiology, 46, 1177-1191.

Juan, C. H., and Walsh, V. (2003). Feedback to V1: A reverse hierarchy in vision. Exp

Brain Res., 150, 259-263.

References

192

Kalla, R., Muggleton, N., Cowey, G., and Walsh, V. (2009). Human dorsolateral

prefrontal cortex is involved in visual search for conjunctions but not features:

A theta TMS study. Cortex 45, 1085-1090.

Kammer. T. (1999). Phosphenes and transient scotomas induced by magnetic

stimulation of the occipital pole: their topographic relationship.

Neuropsychologia, 37, 191–198.

Kaplan, E. (1991). The receptive field of retinal ganglion cells in cat and monkey. In:

Leventhal, A. G. (Ed.), Vision and Visual Dysfunction. CRC Press: Boston. pp

10-40.

Kauffman, T., Théoret, H., Pascual-Leone, A. (2002). Braille character discrimination

in blindfolded human subjects. Neuroreport, 13, 571-574.

Keane, J., Calder, A. J., Hodges, J. R., and Young, A. W. (2002). Face and emotion

processing in frontal variant frontotemporal dementia. Neuropsychologia, 40,

655-665.

Kesler-West, M. L., Andersen, A. H., Smith, C. D., Avison, M. J., Davis, C. E.,

Kryscio, R. J., et al. (2001). Neural substrates of facial emotion processing

using fMRI. Brain Res. Cogn. Brain Res., 11, 213-226.

Keysers, C., and Gazzola, V. (2006). Towards a unifying theory of social cognition.

Progress in Brain Research, 156, 379-401.

Keyers, C., Wicker, B., Gazzola, V., Anton, J. L., Fogassi, L., and Gallese, V. (2004).

A touching sight: SII/PV activation during the observation and experience of

touch. Neuron, 42, 335-346.

Kim, C. Y., Blake, R., and Palmeri, T. J. (2006). Perceptual interaction between real

and synesthetic colors. Cortex, 42, 195–203.

References

193

Kipps, C. M., Duggins, A. J., McCusker, E. A., and Calder, A. J. (2007). Disgust and

happiness recognition correlate with anteroventral insula and amygdala

volume respectively in preclinical Huntigton’s disease. Journal of Cognitive

Neuroscience, 19, 1206-1217.

Kircher, T. T., Senior, C., Phillips, M. L., Rabe-Hesketh, S., Benson, P. J., Bullmore,

E. T., et al. (2001). Recognizing one’s own face. Cognition, 78, B1-B15.

Konen, C. S., and Kaster, S. (2008). Two hierarchically organized neural systems for

object information in human visual cortex. Nature Neuroscience, 11, 224-231.

Krolak-Salmon, P., Henaff, M. A., Isnard, J., Tallon-Baudry, C., Guenot, M.,

Vighetto, A., et al. (2003). A specific response to disgust moudlated by

attention in human ventral anterior insula. Annals of Neurology, 53, 446-453.

Lamm, C., Batson, C. D., and Decety, J. (2007). The neural substrate for human

empathy: Effects of perspective-taking and cognitive appraisal. Journal of

Cognitive Neuroscience, 19, 42-58.

Lavidor, M., and Walsh, V. (2003). A magnetic stimulation examination of

orthographic neighborhood effects in visual word recognition. Journal of

Cognitive Neuroscience, 15, 354-363.

Lawrence, A. D., Calder, A. J., McGowan, S. W., and Grasby, P. M. (2002). Selective

disruption of the recognition of facial expressions of anger. Neuroreport, 13,

881-884.

Lawrence, E. J., Shaw, P., Baker, D., Baron-Cohen, S., and David, A. S. (2004).

Measuring empathy: Reliability and validity of the Empathy Quotient.

Psychological Medicine, 34, 911-924.

References

194

Leslie, K. R., Johnson-Frey, S. H., and Grafton, S. T. (2004). Functional imaging of

face and hand imitation: Towards a motor theory of empathy. Neuroimage,

21, 601-607.

Lewis, S., Thoma, R. J., Lanoue, M. D., Miller, G. A., Heller, W., Edgar, C., et al.

(2003). Visual processing of facial affect. Neuroreport, 14, 1841-1845.

Lomber, S. (1999). The advantages and limitations of permanent or reversible

deactivation techniques in the assessment of neural function. Journal of

Neuroscience Methods, 86, 109-118.

Longo, M. R., Schüür, F., Kammers, M. P., Tsakiris, M., and Haggard, P. (in press).

Self awareness and the body image. Acta Psychologica.

Lundqvist, D., Flykt, A., and İhman, A. (1998). The Karolinska Directed Emotional

Faces (KDEF). Stockholm: Department of Neurosciences Karolinska

Hospital.

Macaluso, E., and Driver, J. (2003). Multimodal spatial representations in the human

parietal cortex: Evidence from functional neuroimaging. Adv Neurology, 93,

219-233.

MacDonald, D., Kabani, N., Avis, D., and Evans, A. C. (2000). Automated 3-D

extraction of inner and outer surfaces of cerebral cortex from MRI.

Neuroimage, 12, 340-356.

MacLeod, C. M., and Dunbar, K. (1988). Training and stroop-like interference:

Evidence for a continuum of automaticity. J Exp Pscyhol Learn Mem Cogn,

14, 126-135.

Maravita, A., and Iriki, A. (2002). Tools for the body (schema). Trends in Cognitive

Sciences, 8, 79-86.

References

195

Marks, L. E. (1975). On coloured hearing synaesthesia: Cross-modal translations of #

sensory dimensions. Psychological Bulletin, 82, 303-331.

Marzi, C. A., Miniussi, C., Maravita, A., Bertolasi, L., Zanette, G., Rothwell, J. C.,

and Sanes, J. N. (1998). Transcranial magnetic stimulation selectively impairs

interhemispheric transfer of visuo-motor information in humans. Exp. Brain.

Res., 118, 435-438.

McCabe, C., Rolls, E. T., Bilderbeck, A., and McGlone, F. (2008). Cognitive

influences on the affective representations of touch and the sight of touch in

the human brain. Social Cognitive and Affective Neuroscience, 3, 97-108.

Mesulam, M. M., and Mufson, E. (1982). Insula of the old world monkey, III:

Efferent cortical output and comments on function. Journal of Comparative

Neurology, 212, 38-52.

Meyer, M., Zysset, S., von Cramon, D. Y., and Alter, K. (2005). Distinct fMRI

responses to laughter, speech, and sounds along the human perisylvian cortex.

Cog Brain Res. 24, 291-306.

Milner, B. (1966). Amnesia following operation on the temporal lobes. In Amnesia,

C.W.M. Whitty and O.L. Zangwill (eds.). London: Botterworths, pp 109-133.

Mills, C. B., Boteler, E. H., and Oliver, G. K. (1999). Digit synaesthesia: A case

study using a stroop-type test. Cognitive Neuropsychology, 16, 181-191.

Mitchell, R. L., and Crow, T. J. (2005). Right hemisphere language functions and

schizophrenia: The forgotten hemisphere? Brain, 128, 963-978.

Montgomery, K. J., and Haxby, J. (2008). Mirror neuron system differentially

activated by facial expressions and social hand gestures: A functional

magnetic resonance imaging study. Journal of Cognitive Neuroscience, 20,

1866-1877.

References

196

Morrison, I., Lloyd, D., di Pellegrino, G., and Roberts, N. (2004). Vicarious responses

to pain anterior cingulated cortex: Is empathy a multisensory issue? Cogn

Affect Behav Neurosci, 4, 270-278.

Mottaghey, F. M., Gangitano, M., Sparing, R., Krause, B. J., Pascual-Leone, A.

(2002). Segregation of areas related to visual working memory in the

prefrontal cortex revealed by rTMS. Cereb Cortex, 12, 369-375.

Mufson, E., and Mesulam, M. M. (1982). Insula of the old world monkey, II:

Afferent cortical input and comments on the claustrum. Journal of

Comparative Neurology, 212, 23-37.

Muggleton, N., Tsakanikos, E., Walsh, V., and Ward, J. (2007). Disruption of

synaesthesia following TMS of the right posterior parietal cortex.

Neuropsychologia, 45, 1582-1585.

Muncer, S., and Ling, J. (2006). Psychometric analysis of the empathy quotient (EQ)

scale. Personality and Individual Differences, 40, 1111-1119.

Nagarajan, S.S., Durand, D., and Warman, E.N. (1993). Effects of induced electric

fields on finite neuronal structures: a simulation study. IEEE Trans. in

Biomedical Engineering, 40, 1175-1188.

Nikolić, D., Lichti, P., and Singer, W. (2007). Color opponency in synaesthetic

experiences. Psychological Science, 18, 481-486.

Nummenmaa, L., Hirvonen, J., Parkkola, R., and Hietanen, J. K. (2008). Is emotional

contagion special? An fMRI study on neural systems for affective and

cognitive empathy. Neuroimage, 43, 571-580.

Nunn. J. A., Gregory, L. J., Brammer, M., Williams, S. C., Parslow, D. M., Morgan,

M .J., et al. (2002). Functional magnetic resonance imaging of synesthesia:

Activation of V4 / V8 by spoken words. Nature Neuroscience, 5, 371-375.

References

197

Oberman, L. M., Hubbard, E. M., McCleery, J. P., Altschuler, E. L., Ramachandran,

V. S., and Pineda, J.A. (2005). EEG evidence of mirror neuron dysfunction in

autism spectrum disorders. Brain Res Cogn Brain Res., 24, 190-198.

Oberman, L. M., and Ramachandran, V. S. (2007). The simulating social mind: The

role of the mirror neuron system and simulation in the social and

communicative deficits of autism spectrum disorders. Psychol Bull., 133, 310-

327.

Oberman, L.M. , Winkielman, P., and Ramachandran, V. S. (2007). Face to face:

Blocking facial mimicry can selectively impair recognition of emotional

expressions. Social Neuroscience, 2, 167-178.

Olausson, H., Lamarre, Y., Backlund, H., Morin, C., Wallin, B. G., Starck, G.,

Ekholm, S., Strigo, I., Worsley, K., Valbo, Ǻ. B., and Bushnell, M. C. (2002).

Unmyelinated tactile afferents signal touch and project to insular cortex.

Nature Neuroscience, 5, 900-904.

Oliveri, M., Turriziani, P., Carlesimo, G. A., Koch, G., Tomaiuolo, F., Panella, M., et

al. (2001). Parieto-frontal interactions in visual-object and visual-spatial

working memory. Cereb Cortex, 11, 606-618.

Olson, I. R., Gatenby, J. C., and Gore, J. C. (2002). A comparison of bound and

unbound audio-visual information processing in the human cerebral cortex.

Cognitive Brain Research, 14, 129-138.

O'Shea, J., Muggleton, N. G., Cowey, A., and Walsh, V. (2004). Timing of target

discrimination in human frontal eye fields. Journal of Cognitive Neuroscience,

16, 1060-1067.

References

198

Parise, C. V., and Spence, C. (2009). ‘When birds of a feather flock together’:

Synesthetic correspondances modulate audiovisual integration in non-

synesthetes. PloS One, 4, e5664.

Pascual-Leone, A., Amedi, A., Fregni, F., and Merabet, L. B. (2005). The plastic

human brain cortex. Ann. Rev. Neuroscience, 28, 377-401.

Paulesu, E., Harrison, J., Baron-Cohen, S., Watson, J. D., Goldstein, L., Heather, J., et

al. (1995). The physiology of coloured hearing. A PET activation study of

colour-word synaesthesia. Brain, 118, 661-676.

Peelen, M. V., and Downing, P. E. (2005). Selectivity for the human body in

fusiform gyrus. Journal of Neurophysiology, 93, 603-608.

Petrides, M., Cadoret, G., and Mackey, S. (2005). Orofacial somatomotor responses in

the macaque monkey homolouge of Broca’s area. Nature, 435, 1235-1238.

Philips, M. L., Young, A. W., Scott, S. K., Calder, A. J., Andrew, C., Giampietro, V.,

et al. (1998). Neural responses to facial and vocal expressions of fear and

disgust. Proc. Biol. Sci., 265, 1809-1817.

Philips, M. L., Young, A. W., Senior, C., Brammer, M., Andrew, C., Calder, A. J., et

al. (1997). A specific substrate for perceiving facial expressions of disgust.

Nature, 389, 495-498.

Pitcher, D., Charles, L., Devlin, J. T., Walsh, V., and Duchaine, B. C. (2009). Triple

dissociation of faces, bodies, and objects in extrastraite cortex. Current

Biology, 19, 319-324.

Pitcher, D., Garrido, L., Walsh, V., and Duchaine, B. C. (2008). Transcranial

magnetic stimulation disrupts the perception and embodiment of facial

expressions. Journal of Neuroscience, 28, 8929-8933.

References

199

Pitcher, D., Walsh, V., Yovel, G., and Duchaine, B. C. (2008). TMS evidence for the

involvement of the right occipital face area in early face processing. Current

Biology, 17, 1568-1573.

Pourtois, G., Sander, D., Andres, M., Grandjean, D., Reveret, L., Olivier, E., et al.

(2004). Dissociable roles of the human somatosensory and superior temporal

cortices for processing social face signals. Eur. J. Neurosci., 20, 3507-3515.

Preston, S. D., and de Waal, F. B. (2002). Empathy: It’s ultimate and proximal basis.

Behav Brain Sci., 25, 1-20.

Ramachandran, V. S., and Hubbard, E. (2001). Synaesthesia – A window into

perception, thought and language. Journal of Consciousness Studies, 8, 3-34.

Rao, A., Nobre, A. C., Alexander, I., and Cowey, A. (2007). Auditory evoked visual

awareness following sudden ocular blindness: An EEG and TMS

investigation. Exp Brain Res, 176, 288-298.

Rich, A. N., and Mattingley, J. B. (2002). Anomalous perception in synaesthesia: A

cognitive neuroscience. Nature Reviews Neuroscience, 3, 43-52.

Rich, A. N., Bradshaw, J. L., and Mattingley, J. B. (2005). A systematic large-scale

study of synaesthesia: Implications for the role of early experience in lexical-

colour assocaitions. Cognition, 98, 53-84.

Rich, A. N., Williams, M. A., Pruce, A., Syngeniotis, A., Howard, M. A., McGlone,

F., et al. (2006). Neural correlates of imagined and synaesthetic colours.

Neuropsychologia, 44, 2918-2925.

Rizzo, M., and Eslinger, P. J. (1989). Colored hearing synesthesia: An investigation

of neural factors. Neurology, 39, 781-784.

Rizzolatti, G., and Craighero, L. (2004). The mirror-neuron system. Annual Review

of Neuroscience, 27, 169-192.

References

200

Ro, T., Farnè, A., Johnson, R. M., Wedeen, V., Chu, Z., Wang, Z. J., et al. (2007).

Feeling sounds after a thalamic lesion. Ann Neurol., 62, 433-441.

Robertson, L. C. (2003). Binding, spatial attention, and perceptual awarness. Nat Rev

Neurosci., 4, 93-102.

Robertson, D., Davies, I., and Davidoff, J. (2000). Colour categories are not

universal: Replications and new evidence from a stone-age culture. Journal of

Experimental Psychology, 129, 369-398.

Robertson, I. H., and Murre, J. M. J. (1999). Rehabilitation of brain damage: Brain

plasticity and principles of guided recovery. Psychological Bulletin, 125, 544-

575.

Rorden, C., Heutink, J., Greenfield, E., and Robertson, I. H. (1999). When a rubber

hand ‘feels’ what the real hand cannot. NeuroReport, 10, 135-138.

Rouw, R., and Scholte, H. S. (2007). Increased structural connectivity in grapheme-

colour synesthesia. Nat Neurosci., 10, 792-797.

Ruby, P., and Decety, J. (2001). Effect of subjective perspective taking during

simulation of action: A PET investigation of agency. Nature Neuroscience, 4,

546-550.

Ruff, C.C., Blankenburg, F., Bjoertomt, O., Bestmann, S., Freeman, E., Haynes, J-D.,

Rees, G., Josephs, O., Deichmann, R., and Driver, J. (2006). Concurrent TMS-

fMRI and psychophysics reveal frontal influences on human retinotopic visual

cortex. Current Biology, 16, 1479-1488

Rushworth, M., Ellison, A., and Walsh, V. (2001). Complementary localization and

lateralization of orienting and motor attention. Nature Neuroscience, 4, 656-

661.

References

201

Russell, R., Duchaine, B., and Nakayama, K. (2009). Super-recognizers: People with

extraordinary face recognition ability. Psychonomic Bulletin and Review, 16,

252-257.

Saarela, M. V., Hlushchuk, Y., Williams, A. C., Schürmann, M., Kalso, E., and Hari,

R. (2007). The compassionate brain: Humans detect intensity of pain from

another’s face. Cereb Cortex, 17, 230-237.

Sachs, G. T. L. (1812). Historiae naturalis duorum leucaetiopum: Auctoris ipsius et

sororis eius. Erlangen.

Saenz, M., and Koch, C. (2008). The sound of change: Visually-induced auditory

synesthesia. Current Biology, 18, 650-651.

Sagiv, N. (2004). In: L. Robertson and N. Sagiv, Editors, Synesthesia: Perspectives

from Cognitive Neuroscience, Oxford University Press, Oxford (2004),

Sagiv, N., Simner, J., Collins, J., Butterworth, B., and Ward, J. (2006). What is the

relationship between synaesthesia and visuo-spatial number forms?

Cognition, 101, 114-128.

Sagiv, N., and Ward, J. (2006). Cross-modal interactions: Lessons from Synaesthesia.

Progress in Brain Research, 155, 263-275.

Sauter, D. A., and Eimer, M. (in press). Rapid detection of emotion from human

vocalizations. Journal of Cognitive Neuroscience.

Sauter, D. A., and Scott, S. K. (2007). More than one kind of happiness: Can we

recognise vocal expressions of different positive states? Motivation and

Emotion, 31, 192-199.

Saxe, R., Jamal, N., and Powell, L. (2006). My body or yours? The effect of visual

perspective on cortical body representations. Cerebral Cortex, 16, 178-182.

References

202

Schiltz, K., Trocha, K., Wieringa, B. M., Emirch, H. M., Johannes, S., and Münte, T.

F. (1999). Neurophysiological aspecrs of synesthetic experience. J

Neuropsychiatry Clin Neurosci., 11, 58-65.

Schofield, W. N. (1976). Do children find movements which cross the body midline

difficult? Q. J. Exp. Psychol. 28, 571-582.

Schröder, M. (2003). Experimental study of affect bursts. Speech communication, 40,

99-116.

Schulte-Rüther, M., Markowitsch, H. J., Fink, G. R., and Piefke, M. (2008). Miror

neuron and theory of mind mechanisms involed in face-to-face interactions: A

functional magnetic resonance imaging approach to empathy. Journal of

Cognitive Neuroscience, 19, 1354-1372.

Scott, S. K., Sauter, D. A., and McGettigan, C. (In press). Brain mechanisms for

processing perceived emotional vocalizations in humans. To appear in

Handbook of Mammalian Vocalizations: A Neuroscience Approach, S

Brudzynski ed. (Elsevier).

Scott, S. K., Young, A. W., Calder, A. J., Hellawell, D. J., Aggleton, J. P., and

Johnsen, M. (1997). Impaired auditory recognition of fear and anger following

bilateral amygdala lesions. Nature, 385, 254-257.

Seitz, R. J., Schäfer, R., Scherfield, D., Friederichs, S., Popp, K., Wittsack, H. J., et al.

(2008). Valuating other people’s emotional face expression: A combined

functional magnetic resonance imaging and electroencephalography study.

Neuroscience, 152, 713-722.

Serino, A., Pizzoferrato, F., and Làdavas, E. (2008). Viewing a face (especially one’s

own face) being touched enhances tactile perception on the face.

Psychological Science, 19, 434-438.

References

203

Seron, X., Pesenti, M., Nöel, M. P., Deloche, G., and Cornet, J. A. (1992). Images of

numbers, or ‘When 98 is upper left and 6 is sky blue”. Cognition, 44, 159-

196.

Shallice, T. (1988). The neuropsychology of mental structure. Cambridge: Cambridge

University Press.

Shamay-Tsoory, S. G., Aharon-Peretz, J., and Perry, D. (2009). Two systems for

empathy: A double dissociation between emotional and cognitive empathy in

inferior frontal gyrus versus ventromedial prefrontal lesions. Brain, 132, 617-

627.

Shaw, P., Greenstein, D., Lerch, J., Clasen, L., Lenroot, R., Gogtay, N., et al. (2006).

Intellectual ability and cortical development in children and adolescents.

Nature, 440, 676-679.

Siebner HR, Willoch F, Peller M, Auer C, Boecker H, Conrad B, and Bartenstein P.

1998). Imaging brain activation induced by long trains of repetitive

transcranial magnetic stimulation. NeuroReport, 9, 943-948.

Simner, J. (2007). Beyond perception: Synaesthesia as a psycholingustic

phenomenon. Trends in Cognitive Sciences, 11, 23-29.

Simner, J., and Holenstein, E. (2007). Ordinal linguistic personification as a variant

of synaesthesia. Journal of Cognitive Neuroscience, 19, 694-703.

Simner, J., Mulvenna, C., Sagiv, N., Tsakanikos, E., Witherby, S., Fraser, C., Scott,

K., and Ward, J. (2006). Synaesthesia: The prevalence of atypical cross-modal

experiences. Perception, 35, 1024-1033.

Simner, J., Ward, J., Lanz, M., Jansari, A., Noonan, K., Glover, L., and Oakley, D.

(2005). Non-random associations of graphemes and colour in the synaesthetic

and normal populations. Cognitive Neuropsychology, 22, 1069-1085.

References

204

Singer, T., Seymour, B., O’Doherty, J., Kaube, H., Dolan, R. J., and Frith, C. D.

(2004). Empathy for pain involves the affective but not sensory components of

pain. Science, 303, 1157-1162.

Singh, K.D., Hamdy, S., Aziz, Q., and Thompson, D.G. (1997). Topographic mapping

of trans-cranial magnetic stimulation data on surface rendered MR images of

the brain. EEG Clin Neurophysiol, 105, 345-351.

Sirigu, A., Grafman, J., Bressler, K., and Sunderland, T. (1991). Multiple

representations contribute to body knowledge processing. Brain, 114, 629-

642.

Smilek, D., Dixon, M. J., Cudahy, C., and Merikle, P. M. (2001). Synaesthetic

photisms influence visual perception. Journal of Cognitive Neuroscience, 13,

930-936.

Smilek, D., Malcolmson, K. A., Carriere, J., Eller, M., Kwan, D., and Reynolds, M.

(2007). When “3” is a jerk and “E” is king: Personifying inanimate objects in

synaesthesia. Journal of Cognitive Neuroscience, 19, 981-992.

Smilek, D., Moffatt, B. A., Pasternak, J., White, B. N., Dixon, M. J., and Merikle, P.

M. (2002). Synaesthesia: A case study of discordant monozygotic twins.

Neurocase, 8, 338-342.

Smith, H. L. (1905). Synesthesia. John Hopkins Hospital Bulletin, 16, 258-263.

Southgate, V., and Hamilton, A. F. (2008). Unbroken mirrors: Challenging a theory

of autism. Trends in Cognitive Sciences, 12, 225-229.

Sperling. J. M., Prvulovic, D., Linden, D. E., Singer, W., and Stirn, A. (2006).

Neuronal correlates of colour-graphemic synaesthesia: A fMRI study. Cortex,

42, 295-303.

References

205

Spiller, M. J., and Jansari, A. S. (2008). Mental imagery and synaesthesia: Is

synaesthesia from internally-generated stimuli possible. Cognition, 109, 143-

151.

Sprengelmeyer, R., Raush, M., Eysel, U. T., and Przuntek, H. (1998). Neural

substrates associated with recognition of facial expressions of basic emotions.

Proc. Biol. Sci., 265, 1927-1931.

Sprengelmeyer, R., Young, A. W., Calder, A. J., Karnat, A., Lange, H., Hömberg, V.,

et al. (1996). Loss of disgust: Perception of faces and emotions in

Huntington’s disease. Brain, 119, 1647-1665.

Sprengelmeyer, R., Young, A. W., Mahn, K., Schroeder, U., Woitalla, D., Buttner, T.,

et al. (2003). Facial expression recognition in people with medicated and

unmedicated Parkinson’s disease. Neuropsychologia, 41, 1047-1057.

Stefan, K., Gentner, R., Zeller, D., Dang, S., and Classen, J. (2008). Theta-burst

stimulation: Remote physiological and local behavioural after-effects.

Neuroimage, 40, 265-274.

Steven, M. S., and Blakemore, C. (2004). Visual synaesthesia in the blind.

Perception, 33, 855-868.

Steven, M. S., Hansen, P. C., and Blakemore, C. (2006). Activation of color-selective

areas of the visual cortex in a blind synesthete. Cortex, 42, 304-308.

Stewart, L., Ellison, A., Walsh, V., and Cowey, A. (2001). The role of transcranial

magnetic stimulation in studies of vision, attention and language. Acta

Psychologica, 107, 275-291.

Stewart, L., Mulvenna, C., Griffiths, T., and Ward, J. (in prep). Neural correlates of

audio-visual synaesthesia induced from tone sequences.

References

206

Stewart, L., Walsh, V., Frith, U., and Rothwell, J. C. (2001). TMS produces two

dissociable types of speech arrest. Neuroimage, 13, 472-478.

Sugiura, M., Watanabe, J., Maeda, Y., Matsue, Y., Fukuda, H., and Kawashima, R.

(2005). Cortical mechanisms of visual self-recognition. Neuroimage, 24,

143-149.

Tang, J., Ward, J., and Butterworth, B. (2008). Number forms in the brain. Journal

of Cognitive Neuroscience, 20, 1547-1556.

Taylor, J. C., Wiggett, A. J., and Downing, P. E. (2007). Functional MRI analysis of

body and body part representations in the extrastriate and fusiform body areas.

Journal of Neurophysiology, 98, 1626-1633.

Taylor-Clarke, M., Kennett, S., and Haggard, P. (2004). Persistence of visual-tactile

enhancement in humans. Neuroscience Letters, 354, 22-25.

Terao, Y., Fukuda, H., Ugawa, Y., Hikosaka, O., Furubayashi, T., Hanajima, R.,

Sakai, K., and Kanazawa, I. (1998). Visualization of the information through

human oculomotor cortical regions by transcranial magnetic stimulation. J

Neurophysiol, 80, 936-946.

Terao, Y., Ugawa, Y., Suzuki, M., Sakai, K., Hanajima, R., Gemba-Shimizu, K., and

Kanazawa, I. (1997). Shortening of simple reaction time by peripheral

electrical and submotor-threshold magnetic cortical stimulation. Experimental

Brain Research, 115, 541-545.

Terhune, D. B. (2009). The incidence and determinants of visual phenomenology

during out-of-body experiences. Cortex, 45, 236-242.

Thilo, K. V., Santoro, L., Walsh, V., and Blakemore, C. (2004). The site of saccadic

suppression. Nature Neuroscience, 7, 13-14.

References

207

Treisman, A. (2004). In: L. Robertson and N. Sagiv, Editors, Synesthesia:

Perspectives from Cognitive Neuroscience, Oxford University Press, Oxford

(2004).

Tsakiris, M., and Haggard, P. (2005). The rubber hand illusion revisited: Visuotactile

integration and self-attribution. J Exp Psychol Hum Percept Perform., 31, 80-

91.

Tyrell, R. A., and Owens, D. A. (1988). A rapid technique to assess the resting states

of eyes and other threshold phenomena: The modified binary search (MOBS).

Behav. Res. Methods Instrum. Comput., 20, 137-141.

Uddin, L. Q., Iacoboni, M., Lange, C., and Keenan, J. P. (2007). The self and social

cognition: The role of cortical midline structures and mirror neurons. Trends

in Cognitive Sciences, 11, 153-157.

Uddin, L. Q., Kaplan, J. T., Molnar-Szakacs, I., Zaidel, E., and Iacoboni, M. (2005).

Self-face recognition activates a frontoparietal “mirror” network in the right

hemisphere: An event –related fMRI study. Neuroimage, 25, 926-935.

Uddin, L. Q., Molnar-Szakacs, I., Zaidel, E., and Iacoboni, M. (2006). rTMS to the

right inferior parietal lobule disrupts self-other recognition. Soc. Cogn. Affect.

Neurosci., 1, 65-71.

Ueno, T., Tashiro, T., and Harada, K. (1988). Localized stimulation of neural tissues

in the brain by means of a paired configuration of time-varying magnetic

fields. J. Appl. Phys, 64, 5862–5864.

Vallesi, A., Shallice, T., and Walsh, V. (2007). Role of the prefrontal cortex in the

foreperiod effect: TMS evidence for dual mechanisms in temporal preparation.

Cerebral Cortex, 17, 466-474.

References

208

Van Boven, R. W., and Johnson, K. (1994). A psychophysical study of the

mechanisms of sensory recovery following nerve injury in humans. Brain,

117, 149-167.

Van Boven, R. W., Hamilton, R. H., Kauffman, T., Keenan, P., and Pascual-Leone, A.

(2000). Tactile spatial resolution in blind Braille readers. Neurology, 54, 2230-

2236.

van der Gaag, C., Minderaa, R. B., and Keysers, C. (2007). Facial expressions: What

the mirror neuron system can and cannot tell us. Social Neuroscience, 2, 179-

222.

Van Lacker, D., and Fromkin, V. A. (1973). Hemispheric specialization for pitch and

tone: Evidence from Thai. Journal of Phonetics, 1, 101-109.

van Rijn, S., Aleman, A., van Diessen, E., Berckmoes, C., Vingerhoets, G., Kahn, R.

S. (2005). What is said or how it is said makes a difference: Role of the right

fronto-parietal operculum in emotional prosody as revealed by repetitive TMS.

Eur J. Neurosci. 21, 3195-3200.

Vega-Bermudez, F., and Johnsen, K. O. (1999). SA1 and RA receptive fields,

response variability, and population response mapped with a probe array.

Journal of Neurophysiology, 81, 2701-2710.

Wade, A. R., Brewer, A. A., Rieger, A. W., and Wandell, B. A. (2002). Functional

measurements of human ventral occipital cortex: Retinotopy and colour.

Philos. Trans. R. Soc. Lond. B. Biol. Sci., 357, 963-973.

Walsh, V., and Butler, S. R. (1996). Different ways of looking at seeing. Behavioural

Brain Research, 76, 1-3.

Walsh, V., and Cowey, A. (2000). Transcranial magnetic stimulation and cognitive

neuroscience. Nature Reviews Neuroscience, 1, 73-79.

References

209

Walsh, V., and Pascual-Leone, A. (2003). Transcranial Magnetic Stimulation: A

neurochromometrics of mind, (MIT Press).

Walsh, V., and Rushworth, M. (1999). A primer of magnetic stimulation as a tool for

neuropsychology. Neuropsychologia, 37, 125-135.

Ward, J. (2004). Emotionally mediated synaesthesia. Cognitive Neuropsychology,

71, 761-772.

Ward, J., Banissy, M. J., and Jonas, C. (2008). Haptic perception in synaesthesia. In

Human Haptic Perception: Basics and Applications, Edited by

M.Grunwald. Birkhäuser, Basel.

Ward, J., Huckstep, B., Tsakanikos, E. (2006). Sound-colour synaesthesia: To what

extent does it use cross-modal mechanisms common to us all. Cortex, 42,

264-280.

Ward, J., Li, R., Salih, S., and Sagiv, N. (2007). Varieties of grapheme-colour

synaesthesia: A new theory of phenomenological and behavioural differences.

Conscious Cogn., 16, 913-931.

Ward, J., and Mattingley, J. B. (2006). Synaesthesia: An overview of contemporary

findings and controversies. Cortex, 42, 129-136.

Ward, J., and Simner, J. (2003). Lexical-gustatory synaesthesia: Linguistic and

conceptual factors. Cognition, 89, 237-261.

Ward, J., and Simner, J. (2005). Is synaesthesia an X-linked trait lethality in males?

Perception, 34, 611-623.

Ward, J., Simner, J., and Auyeung, V. (2005). A comparison of lexical-gustatory and

grapheme-colour synaesthesia. Cognitive Neuropsychology, 22, 28-41.

References

210

Warren, J. E., Sauter, D. A., Eisner, F., Wiland, J., Dresner, M. A., Wise, R. J. S., et

al. (2006). Positive emotions preferentially engage an auditory-motor “mirror”

system. Journal of Neuroscience, 26, 13067-13075.

Wassermann, E. M. (1998). Risk and safety of repetitive transcranial magnetic

stimulation: Report and suggested guidelines from the International Workshop

on the Safety of Repetitive Transcranial Magnetic Stimulation.

Electroencephalogr Clin Neurophysiol., 108, 1-16.

Wassermann, E.M., McShane, L.M., Hallett, M., and Cohen, L.G. (1992).

Noninvasive mapping of muscle representations in human motor cortex.

Electroencephalogr Clin Neurophysiol, 85, 1-8.

Weiss, P. H., Shah, N. J., Toni, I., Zilles, K., and Fink, G. R. (2001). Associating

colours with people: A case of chromatic-lexical synaesthesia. Cortex, 37,

750-753.

Weiss, P. H., Zilles, K., and Fink, G. R. (2005). When visual perception causes

feelings: Enhanced cross-modal processing in grapheme-color synesthesia.

Neuroimage, 28, 859-868.

Wheeler, R. H., and Cutsforth, T. D. (1921). The role of synaesthesia in learning. J

Exp Psychology, 4, 448-468.

Wicker, B., Keysers, C., Plailly, J., Royet, J. P., Gallese, V., and Rizzolatti, G. (2003).

Both of us disgusted in my insula: The common neural basis of seeing and

feeling disgust. Neuron, 40, 655-664.

Wig, G., Grafton, S., Demos, K., and Kelley, W. (2005). Specificity of priming: a

cognitive neuroscience perspective. Nat. Neurosci. 8, 1228 – 1234.

References

211

Wildgruber, D., Riecker, A., Hertrich, I., Erb, M., Grodd, W., Ethofer, T., et al.

(2005). Identification of emotional intonation evaluated by fMRI.

Neuroimage, 24, 1233-1241.

Winston, J., O’Doherty, J., and Dolan, R. (2003). Common and distinct neural during

direct and incidental processing of multiple facial emotions. Neuroimage, 20,

84-97.

Wittenberg, G. F., Werhahn, K. J., Wassermann, E. M., Herscovitch, P., and Cohen,

L. G. (2004). Functional connectivity between somatosensory and visual

cortex in early blind humans. European Journal of Neuroscience, 20, 1923-

1927.

Witthoft, N., and Winawer, J. (2006). Synesthetic colors determined by having

refrigerator magents in childhood. Cortex, 42, 175-183.

Yaro, C., and Ward, J. (2007). Searching for Shereshevski: What is superior about

the memory of synaesthetes? Quarterly Journal of Experimental Psychology,

60, 681-695.

Appendix

212

Appendix 1: Questionnaire used to recruit potential synaesthetes in Experiment 2.1

(Chapter 2).

PLEASE READ THIS FIRST People with synaesthesia experience certain things (e.g. colours, tastes) when engaged in activities (e.g. reading) that would not elicit such a response in non-synaesthetic people. For instance, colours may be experienced in response to music or words, or shapes may be experienced in response to tastes. Synaesthesia is quite rare, but the questionnaire below asks whether you regularly have these types of experiences.

Everything you write will be treated in confidence, in accordance with the data protection act. You do not have to answer any questions if you feel uncomfortable about them. We may wish to contact a small number of people (by phone/e-mail/letter) to invite them to take part in a further study of memory and perception. None of the tasks are harmful or stressful. It would be helpful then, if you included contact details below, in case you are one of the people we would like to contact. You are in no way obliged to part in any further experiments. Your personal details (name, email, etc.) will not be passed on to anybody else.

Name: _________________________________________________________ Age: ______________ Profession/Degree Course: ______________________________________ Year: ________________ Telephone number: __________________________ E-mail: ___________________________

(1) Do you think about the letters of the alphabet (and/or words and numbers) as having specific colours (i.e. the letter A is experienced as red)? Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree If SO, Which ones? Letters Words Numbers

Other? ______________

(2) Do you think about the letters of the alphabet (and/or days of the week/months of the

year/numbers) as being arranged in a specific pattern in space? Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree If SO, Which ones? Letters Days Months Numbers

Other? ______________

(3) Do you experience taste sensations when you observe another person eating or

drinking something (i.e. observing someone eating strawberries and experiencing a sweet taste in your own mouth)?

Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree

Appendix

213

(4) Do you experience touch sensations on your own body when you see them on another person’s body?

Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree

(5) When experiencing touch to your own body do you experience visual sensations (i.e.

colour)?

Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree (6) Do these experiences have specific locations (e.g. on your body, on words or objects

in the environment, in front of your eyes) or not (e.g. they feel as if they are in ‘your minds eye’)? Please describe. _____________________________________________________________________ _______________________________________________________________________________________________________________________________________________________________________________________________________________

(7) To the best of your knowledge have you always had these sensations?

YES NO DON’T KNOW If YES – at what age did you realise that other people did not have the same sensations as you? _____________________________________________________________________

If NO – at what age did they arise and was there a triggering incident?

_____________________________________________________________________

(8) Do the sensations that you have to particular things change over time or are they fixed

(e.g. if the word ‘book’ is green then is it always green and always has been)?

FIXED VARIABLE DON’T KNOW

(9) On the next page please match the triggers on the left with synaesthetic experiences on the right. For instance, if you experience colours in response to numbers then draw a line in between ‘numbers’ (left) and ‘colours’ (right).

IMPORTANT: Please do not connect the same things (e.g. colours–colours) as this is assumed true of everyone. We also assume (without you having to indicate) that letters/ words etc. as

experienced as shapes on a written page, and musical instruments/ voices/ spoken words as noise.

Moreover, if you have no synaesthetic experiences then there is no need to connect any triggers

with experiences.

Appendix

214

TRIGGERS EXPERIENCES Letters of alphabet Colours English words Shapes/Patterns Foreign words Tastes People’s names Smells Addresses/places Pains/touches Numbers Noises Days of week Flashes Months of year Music Pains/touches Movements Music (instrumental) Noises Smells Tastes Colours Shapes/Patterns

Thanks for your time.