Accesing Embodied Object Representations1

Psychological BulletinAccessing Embodied Object Representations From Vision:A ReviewHeath Matheson, Nicole White, and Patricia McMullenOnline First Publication, October 13, 2014. http://dx.doi.org/10.1037/bul0000001

CITATIONMatheson, H., White, N., & McMullen, P. (2014, October 13). Accessing Embodied ObjectRepresentations From Vision: A Review. Psychological Bulletin. Advance online publication.http://dx.doi.org/10.1037/bul0000001

Accessing Embodied Object Representations From Vision: A ReviewHeath MathesonDalhousie University

Nicole WhiteUniversity of Toronto

Patricia McMullenDalhousie University

Theories of embodied cognition (EC) propose that object concepts are represented by reactivations ofsensorimotor experiences of different objects. Abundant research from linguistic paradigms providessupport for the notion that sensorimotor simulations are involved in cognitive tasks like comprehension.However, it is unclear whether object concepts, as accessed from the visual presentation of objects, areembodied. In the present article we review a large body of visual cognitive research that addresses 5 mainpredictions of the theory of EC. First, EC accounts predict that visual presentation of manipulable objects,but not nonmanipulable objects, should activate motor representations. Second, EC predicts that senso-rimotor activity is necessary to perform visualcognitive tasks such as object naming. Third, EC positsthe existence of distinct neural ensembles that integrate information from action and vision. Fourth, ECpredicts that relationships between visual and motor activity change throughout development. Fifth, ECpredicts that the visual presentation of objects or actions should prime performance cross-modally. Wesummarize findings from neuroimaging, neuropsychology, neurophysiology, development, and behav-ioral paradigms. We show that while much of the research published so far demonstrates that there is arelationship between visual and motoric representations, there is no evidence supporting a strong formof EC. We conclude that sensorimotor simulations may not be required to perform visual cognitive tasksand highlight a number of directions for future research that could provide strong support for EC in visualcognitive paradigms.

Keywords: visual object recognition, vision and action, theory of embodied cognition

Since the cognitive revolution of the 1960s, information-processing theories have dominated experimental psychology, pro-viding models for understanding human cognition and shaping thedisciplines ideas about what constitutes mental processes. Theseideas have their roots in computational metaphors (e.g., the mindas software executed in the brains hardware). Whether explicitlyor implicitly, most theories in cognitive psychology assume thatthe brain is like a computer, processing sensory input with sym-bolic algorithms and producing a meaningful output (i.e., behav-ior). These ideas are perhaps most strongly reflected in the seminalworks of Fodor (1983) and Marr (2010). Classic applications ofthis framework have resulted in models of cognitive processingthat are largely modular, with different processes specialized forparticular types of information and transforming it in particularways. An important feature of these theories is the notion that thesensorimotor systems used to initially perceive the world are notused to represent the world; that is, the cognitive processes carried

out by the brain can be understood as algorithms that operate onamodal representations. Therefore, the cognitive systems repre-sentations are stored as abstract symbols. This property allowsthem to be implemented in any mediumincluding, of course, thevery thing from which they were inspired: computers.

However, these models tend to view the production of action asthe outcome of cognitive processes, not central to them. Recogni-tion of the influence of action on cognition has led to dissatisfac-tion with amodal theories. Importantly, theories of embodied cog-nition (EC) provide a powerful alternative for understanding thefunctional relationships between vision and action. Though there isno single unified theory of embodied cognition, current proposalsassume that sensorimotor experiences constrain and shape cogni-tive processes (see Garbarini & Adenzato, 2004). As such, abstractsymbol manipulation in the brains computational centers does notdefine cognition; rather, cognition depends on the human bodiesthat constrain it, the brain that implements it, and the environmentin which it takes place. At their roots, theories of embodiedcognition argue that cognition is better understood as a biologicalsystem constrained by experience.

The Embodied Cognition HypothesisMany philosophers and experimental researchers have used the

term embodied cognition to frame their questions, though there isgreat diversity in how this term is used. In a comprehensive reviewof the concept, Shapiro (2011) defined three distinct categories of

Heath Matheson, Department of Psychology and Neuroscience, Dalhou-sie University; Nicole White, Department of Psychology, University ofToronto; Patricia McMullen, Department of Psychology and Neuroscience,Dalhousie University.

Correspondence concerning this article should be addressed to HeathMatheson, Department of Psychology, Dalhousie University, Life SciencesCenter, Halifax, Nova Scotia, B3H 4J1. E-mail: [email protected]

This

docu

men

tis

copy

right

edby

the

Am

eric

anPs

ycho

logi

calA

ssoc

iatio

nor

one

ofi

tsal

lied

publ

isher

s.Th

isar

ticle

isin

tend

edso

lely

fort

hepe

rson

aluse

oft

hein

divi

dual

use

ran

dis

not

tobe

diss

emin

ated

broa

dly.

Psychological Bulletin 2014 American Psychological Association2014, Vol. 141, No. 1, 000 0033-2909/14/$12.00 http://dx.doi.org/10.1037/bul0000001

1

EC hypotheses.1 The category that we focus on here is the con-ceptualization hypothesis (Shapiro, 2011). According to research-ers endorsing this hypothesis, cognitive processes are not onlyconstrained by the body and its modal sensory systems, but cog-nitionidentifying objects, reasoning about people and places,solving emotional problems and planning for the futurealsocomprises reactivating the same sensory and motor regions in-volved in perception. According to this view of embodied cogni-tion, to know that an object lying on the table is a hammer, wemust simulate (or reactivate) our experiences with a hammerwhat it looks like, the sounds it makes, how to use it. Additionally,in the future, when thinking about a hammer, we use these simu-lations as the basis of cognitive processing. Indeed, all of ourthoughts about hammers will rely on these modality-specific sim-ulations in some way. By extension (and perhaps this is the mostradical consequence of the conceptualization hypothesis), even ourabstract thoughts rely on these simulationsthoughts about ham-mering our argument home or what it means to hammer away ata task (e.g., Lakoff & Johnson, 1999). It is the conceptualizationhypothesis that has attracted the most attention in experimentalcognitive psychology and cognitive neuroscience and underlies thepresent review.

Why Theories of Embodied Cognition?Before discussing models of EC, it is worth considering why

they hold such appeal for cognitive psychologists. There are atleast two advantages to the conceptualization hypothesis whencompared to more traditional, amodal hypotheses. First, EC isneurologically constrained. Cognitive neuroscientific methods in-cluding functional magnetic resonance imaging (fMRI), event-related potentials (ERPs), and single cell recordings have providedevidence that brain activity is not strictly modular in a Fodoriansense, and research has shown that many cognitive tasks activatethe entire brain, even in simple visual attention tasks (e.g.,Gonzalez-Castillo et al., 2012). Most researchers argue that thebrain is best understood as a system of parallel networks (e.g.,McClelland & Rumelhart, 1981; Sporns, 2011) or a dynamicalsystem (e.g., see Shapiro, 2011). Though it is certainly possible foran amodal, neuroagnostic theory to be internally consistent,highly predictive, and useful in generating new research questions,by its very nature such a model is hardware-independent and thusis not specific to the neuroarchitecture of the human brain. Hom-mel and colleagues (e.g., Hommel, Msseler, Aschersleben, &Prinz, 2001) theory of event coding provides an excellent exampleof such an neuroagnostic theory. However, we argue that, in theend, the EC approach is a more neurologically plausible account ofhuman cognition because its predictions arise directly from knowl-edge of extant neural mechanisms.

The second advantage of the conceptualization hypothesis isthat it is robust against serious shortcomings of classical amodalmodels. For instance, amodal theories essentially posit that con-cepts are duplicated in the brain, represented independently atsensory and semantic levels. This redundancy is absent in EC,which posits a single system for processing and representingincoming information. More important, it has been argued that anyamodal account of cognition is inherently unfalsifiable (see Bar-salou, Simmons, Barbey, & Wilson, 2003), as any result can beexplained by an additional module or algorithm. On the other

hand, the conceptualization hypothesis makes specific a prioripredictions about the relationship between neural and cognitiveprocesses. For these reasons, the conceptualization hypothesisprovides a strong framework to understand the relationship be-tween brain and behavior.

Models of EmbodimentThough there is no single agreed-upon neuropsychological

model of EC, there are at least two that provide a strong theoreticalfoundation.2 Fundamentally, both of these models share the as-sumption that Hebbian processes of association (see Hebb, 1949,p. 62) underlie the functional organization of the brain. That is, theparticular collections of neurons activated by the experience of aparticular event (e.g., the visual, tactile, and auditory experiencesof hammering) become functionally connected by virtue of beingactive at the same time and are thus more likely to be activatedtogether at a later time (see also Wennekers, Garagnani, & Pul-vermller, 2006). There is now compelling neuroscientific evi-dence for such associative processes. Research in molecular neu-roscience supports the notion that neural connections changedynamically as a result of functional coactivation with other neu-ronal populations (see Seung, 2012, for a discussion).

Meyer and Damasio (2009) developed one model they call theconvergencedivergence zone (CDZ) framework. This model isorganized into functional hierarchies. At the lowest level are neuralensembles that process basic sensory information in the differentmodalities. For instance, distinct neural ensembles have evolved to

1 The first is what Shapiro (2011) calls the replacement hypothesis. Forresearchers endorsing this hypothesis, mental representations (i.e., amodalsymbol manipulation) are not needed for cognitive processes; conse-quently, cognition is not modeled as symbolic and algorithmic. For in-stance, robots that have an embodied neural architecture, in whichcognition is modeled as a dynamic, online interaction with the environmentwithout any explicit representational system, can navigate complex envi-ronments much more effectively than robots based on traditional informa-tion processing architectures (see Brooks, 1991). The second hypothesisdescribed by Shapiro is the constitution hypothesis. Here, researchersbelieve that while mental representations exist and are necessary, cognitiveprocesses carried out on representations comprise both the body and theenvironment; thus the body and environment are both critical componentsof knowledge (e.g., Clark & Chalmers, 1998). For instance, when a personuses a grocery list to remember which items to pick up on the way home,the constitution hypothesis would argue that the grocery list is a form of thecognitive process. Although these two embodied cognitive hypotheses areintriguing, and they have obvious consequences for understanding cogni-tive science in the broader sense, they have had relatively little impact onexperimental cognitive psychology and cognitive neuroscience; such linesof thought are more concerned with artificial intelligence and basic ques-tions about the ontology of the concept of cognition.

2 Within cognitive psychology, ideas about embodiment have appearedthroughout its history. Allport (1985), was one of the first cognitivepsychologists to posit that the same neural elements that are involved incoding the sensory attributes of a (possibly unknown) object presented toeye or hand or ear also make up the elements of the auto-associatedactivity-patterns that represent familiar object-concepts in semantic mem-ory (p. 53). Further, Gibsons (1986) affordance hypothesis is oftendescribed as an early form of embodiment. Indeed, ideas of embodimentcan be traced to James (1890), who suggested that never is the body feltall alone, but always together with other things (Chapter 10, para. 16) andalso speculated about the role of the body in generating emotional reac-tions. Finally, Hommel, Msseler, Aschersleben, & Prinzs (2001) theoryof event coding shares some strong conceptual similarities to the modelsdiscussed here, though it is neuroagnostic and not modality-specific.

This

docu

men

tis

copy

right

edby

the

Am

eric

anPs

ycho

logi

calA

ssoc

iatio

nor

one

ofi

tsal

lied

publ

isher

s.Th

isar

ticle

isin

tend

edso

lely

fort

hepe

rson

aluse

oft

hein

divi

dual

use

ran

dis

not

tobe

diss

emin

ated

broa

dly.

2 MATHESON, WHITE, AND MCMULLEN

detect edges of different orientations in the visual system anddifferent pitches in the auditory system. These are best viewed asfeature maps, as they explicitly map the features of the environ-ment by signaling the presence of various stimulus configurations.These sensory ensembles then converge on other neural networksbeyond primary cortices (i.e., higher association cortices). Thesenetworks converge on other secondary networks, and so on (up andup to what are typically considered the most evolved structureswithin the anterior cortices, including the prefrontal cortex andtemporal poles). By definition, convergence zones process infor-mation from multiple sources; given this pattern of connectivity,convergence zones receive information about the relative timing ofactivations in different modalities. In this way, higher order con-vergence zones hold records of different collections of sensorimo-tor experiences. According to Damasio (Meyer & Damasio, 2009;see also Damasio, 2012), these highest order zones are bestthought of as dispositional ensembles; they are disposed to reac-tivate the modality specific cortices that were active during expe-rience. The critical feature of this model is that these zonesthemselves cannot be said to possess any knowledge per se (e.g.,they are not the locus of semantic memory); rather, they possessinstructions for reactivating components of experience. That is,each convergence zone sends divergent back-projections towardthe neural ensembles they receive information from, allowing themto retroactivate different modality-specific ensembles in a simula-tion of a previous experience.

Proponents of this model argue that these simulations form thebasis of all cognitive processes, including object recognition. Sen-sorimotor systems map stimulus-specific sensory information ofobjects (e.g., the feeling of the hammers handle, the sound itmakes on wood, the sight of its motion, its shape). These featuremaps are constantly updated as experience unfolds. By virtue ofthe functional relationships between sensorimotor regions andassociation cortices, experiences are stored as dispositions inhigher level zones and can be reactivated during recognition. Forinstance, the visual presentation of objects will activate object-related experiences in other modalities. Importantly, while therepresentation of both a nonmanipulable object like a bear and amanipulable object like a hammer will rely on retroactivations ofvisual and auditory experience, only the hammer should retroac-tivate motor experiences (assuming that one has little experiencemanually manipulating bears). In cognitive tasks, the specificity ofthese retroactivations (i.e., what types of experiences are retroac-tivated) will form the basis of naming or categorizing the stimulus.

A second model of embodiment extends Damasios CDZ frame-work. Barsalou et al. (2003) have developed the perceptual sym-bols systems theory. As in the CDZ framework, concepts arerepresented through retroactivations of modality-specific informa-tion. However, Barsalou et al. (2003) extends the CDZ frameworkby suggesting that attentional processes are important in shapingsimulations of sensorimotor activity. Attentional processes prior-itize components of experience, increasing the chance that thosecomponents will become part of the objects representation. In thisway, the representation of different stimuli may be weightedtoward different types of information. That is, simulations becomesituated; the activation of the hammer simulation might not com-prise all possible combinations of sensorimotor components butwill depend on the goal and experiences of the observer.

Together, the CDZ (Meyer & Damasio, 2009) and perceptual-symbol-systems (Barsalou et al., 2003) models serve as a generalframework for understanding how the brain might represent aconcept like hammer. They are useful in generating specific pre-dictions stemming from the conceptualization hypothesis andguide much of the review of evidence presented here.

The Present ReviewIn general, much evidence has accumulated in favor of an EC

perspective from tasks involving action production and under-standing, memory, social cognition, problem solving and reason-ing, and child development (see Barsalou, 2008, for a review ofstudies in these areas; see also M. Wilson, 2002).3 The mostcompelling support for EC comes from linguistic paradigms, fromwhich there are abundant behavioral and neuroscientific findingssupportive of motor activity in embodied object representations(see Fischer & Zwaan, 2008; Kiefer & Pulvermller, 2012; Pul-vermller, 2005; Pulvermller & Fadiga, 2010).4 In contrast, thereis little discussion about whether object concepts, as accessed fromthe visual presentation of the objects themselves, are embodied.Research has yet to address the question of whether visuallyencountering an object induces embodied simulations of activity inmodal regions associated with knowledge about the object. This isimportant because it is unclear whether embodied simulations aremore important in certain modalities (e.g., language vs. vision vs.audition, etc.) than others. To address this, we review researchrelevant to the question of whether the visual presentation ofobjects activates embodied object concepts.

In this review, we consider the broad visual-cognitive literaturerelevant to examining an EC account of object representation. Wefocus on five main predictions, each addressed by a differentdomain of psychology and cognitive neuroscience. First, EC ac-counts predict that visual presentation of manipulable objects, butnot nonmanipulable objects, should activate motor representations,since experience with manipulable objects should result in asso-ciated motoric representations. Neuroimaging studies are wellsuited to address the prediction that there is a relationship betweenvisual representations (visual-specific activity) and action repre-sentations (motor-specific activity) for objects that we can manip-ulate, but not for nonmanipulable objects. Second, EC predicts thatsensorimotor activity is necessary to perform visual-cognitivetasks such as object naming. Neuropsychological studies of pa-tients with selective lesions to visual or motor regions provide anideal means of addressing this question, since damage to motorregions (and therefore damage to essential simulation machinery)

3 However, embodied theories are not undisputed. As discussed later,Caramazza and Mahon (2003) provide an introduction to specific, alter-native accounts of conceptual organization (see also Capitani, Laiacona,Mahon, & Caramazza, 2003; Dove, 2011). According to this line ofthought, although there is much evidence consistent with the theory of ECin this literature, the interpretation of it is disputable. Indeed, Mahon andCaramazza (2008) suggested that most of these studies show, at best, thatconcepts in semantic memory are enriched by (rather than constituted by)retroactivations of sensorimotor activity.

4 Discussion of the psycholinguistic literature and the relationship be-tween embodied studies of language and embodied studies of visualcognition are beyond the scope of the present review. Because there isabundant discussion of embodiment in language, we restrict our discussionto visual paradigms here.

This

docu

men

tis

copy

right

edby

the

Am

eric

anPs

ycho

logi

calA

ssoc

iatio

nor

one

ofi

tsal

lied

publ

isher

s.Th

isar

ticle

isin

tend

edso

lely

fort

hepe

rson

aluse

oft

hein

divi

dual

use

ran

dis

not

tobe

diss

emin

ated

broa

dly.

3A REVIEW OF EMBODIED OBJECTS

should impair object naming of manipulable objects. Third, ECposits the existence of distinct neural assemblies that integrateinformation from action and vision and that activity in eithermodality can activate these assemblies. The neurophysiologicalliterature exploring single cell responses allows us to assess thisclaim. Fourth, EC predicts that relationships between visual andmotor association areas change throughout development, a ques-tion that can be addressed via research in child development.Finally, the vast literature of behavioral paradigms examiningvisual-cognitive abilities is well suited to address the EC predic-tion that the visual presentation of objects or actions should primeperformance in visual tasks and tasks of action generation.

To investigate these hypotheses, we began our literature searchusing Google Scholar to search for the exact phrases embodiedcognition (which revealed 399,000 results) and grounded cogni-tion (which revealed 184,000 results). This initial search was usedprimarily to identify extant reviews on the topic. The search forembodied cognition resulted in M. Wilson (2002), Mahon andCaramazza (2008), and Shapiro (2011) as the top hits; the top hitsfor grounded cognition were Barsalou (2008) and Barsalou (2010).Problematically for the present review, most explicit discussion onthe embodied cognition hypothesis occurs within the domain oflinguistic processing, so a further keyword search was not useful inidentifying empirical articles pertinent to our review. Therefore,using these extant reviews, we identified behavioral and fMRIstudies from the reference lists that pertain uniquely to the domainof visual processing of objects. From there, we used the cited byfunction of Google Scholar and, through author agreement, iden-tified potential relevant neuroimaging, neuropsychological, neuro-physiological, developmental, and behavioral studies. All studiesincluded in our review were published in English in establishedempirical, peer-reviewed journals. Because most of the publishedresearch was not explicitly discussed as a test of the theory ofembodied cognition, we selected studies on the basis of threegeneral criteria: The studies must (a) use exclusively (or primarily)the visual presentation of objects as stimuli (either real objects liketools or other visually presented shapes); (b) record a dependentmeasure from explicit motor behavior (including verbal respond-ing) or neural activity from motor systems (or their indices); and(c) be (by author agreement) cited in the literature as evidence infavor of some form of embodied hypothesis. Additionally, wewere particularly interested in studies that included a task that wassemantic in nature (e.g., object naming, picture word matching,categorization), though this was not exclusively the case (e.g., wereview studies with passive viewing). With this approach we wereable to identify and review some of the main predictions of thetheory of embodied cognition; we do not intend to exhaustivelyreview research in sensorimotor vision and action. We includestudies from the modern period of psychological investigation only(i.e., since 1900) and focus on studies that were published since theintroduction of functional neuroimaging; though there is a longphilosophical and potentially empirical tradition of thought thatcould be considered embodied (see No, 2004), we sought toreview only those studies that were peer reviewed in the currentpsychological tradition. We included studies from any age groupof participants (though, with the exception of the few developmen-tal studies we discovered, most were conducted in young adults).We excluded studies and reports that focused on computationalmodeling, computer simulations, or robotics, as these areas are

beyond the scope of the review. Our literature search was con-ducted between October 2012 and March 2014. In the final review,we discuss four studies from neuroimaging, six from neuropsy-chology, two from neurophysiology, two from development, andthree from behavioral studies (summarized in Table 1) and drawon additional sources to highlight our main criticisms of theinterpretation of these studies.

On the basis of our review of the current evidence, we argue thatfindings in diverse areas of psychology and cognitive neurosciencedo not provide evidence for a strong form of EC in visual objectprocessing. In the next section, we examine studies relevant to ECand attempt to identify the main problems with interpreting theirresults as evidence for embodied object representations. While thisreview does not speak against the general utility of EC, it doesprovide a comprehensive integration of research examining ECand object representation and helps to constrain the developmentof the theory of embodied cognition as a way of understanding themind and brain. We conclude by discussing the types of evidencethat will be needed to support an EC account of object conceptsfrom vision.

NeuroimagingOne of the most obvious predictions of the EC account of object

representations is that viewing manipulable objects should elicitactivity in both visual and motor cortices, while viewing nonma-nipulable objects should elicit activity in visual cortices only. It iswell established that, after reaching primary visual cortex, visualinformation is processed in separate streams; Information aboutobject shape and color is processed in the ventral stream, extendingfrom V1 into the inferior temporal lobes, while information aboutan objects movement, its location in space, and the metrics neededto grasp it are processed dorsally, in the posterior middle temporalgyrus and regions of the anterior inferior parietal lobe (see Milner& Goodale, 1995). Despite interactions between these two streamsduring higher visual processes (Matheson & McMullen, 2010), thenotion that there are separate ventral and dorsal cortical streams isgenerally taken as evidence that different object features are en-coded in distributed neural networks (e.g., form vs. motion oraction; see Chao, Haxby, & Martin, 1999; see also Thompson-Schill, 2003). This claim was supported by PET research (Martin,Wiggs, Ungerleider, & Haxby, 1996) showing that although nam-ing images of animals and tools activated overlapping ventralvisual cortical structures, there were category-specific activationsin the occipital lobe in response to animals and unique premotoractivations in response to tools. An EC account of object repre-sentation is consistent with this pattern of brain activity.

Additional research has shown that visual presentation of ma-nipulable objects activates motor regions. PET studies have shownactivation in premotor cortex in response to tools during passiveviewing, naming, and describing a tools function, suggestingmotor involvement in representing objects (e.g., Grafton, Fadiga,Arbib, & Rizzolatti, 1997). fMRI results have further revealedfrontoparietal activity during passive viewing of manipulable ob-jects (e.g., Chao & Martin, 2000). Dorsal regions are activatedwhen participants imagine manipulating visually presented ob-jects, including the inferior parietal cortex, the prefrontal cortex,the motor cortex (in some cases), and the supplementary motorarea (see Grzes & Decety, 2001). Similar findings have been

This

docu

men

tis

copy

right

edby

the

Am

eric

anPs

ycho

logi

calA

ssoc

iatio

nor

one

ofi

tsal

lied

publ

isher

s.Th

isar

ticle

isin

tend

edso

lely

fort

hepe

rson

aluse

oft

hein

divi

dual

use

ran

dis

not

tobe

diss

emin

ated

broa

dly.


Tabl

e1

Sum

mar

yofS

tudi

esofI

nter

estR

elev

antt

oth

eM

ain

Pred

ictio

nsofE

CRe

view

ed

Stud

yM

ain

task

ofi

nter

est

Mai

nfin

ding

Rel

evan

ceto

ECCo

nclu

sion

Neu

roim

agin

gM

artin

etal

.,19

96O

bject

nam

ing

Uni

que

occ

ipita

lact

ivat

ion

toan

imal

s;uniq

uepr

emot

orac

tivat

ion

toto

ols

The

visu

alpr

esen

tatio

nofm

anip

ulab

leobje

ctsel

icits

prem

otor

activ

ity.

Sugg

ests

aco

rrel

atio

nalr

elat

ions

hip,

butn

ot

aca

usa

lone.

Gra

fton

etal

.,19

97Pa

ssiv

evie

win

g,nam

ing,

desc

ribin

gfu

nctio

nTo

olpr

esen

tatio

nan

dde

scrib

ing

tool

use

activ

ated

front

opar

ieta

lreg

ions

.Ch

ao&

Mar

tin,2

000

Pass

ive

vie

win

gTo

olpr

esen

tatio

nac

tivat

edfro

ntop

arie

talr

egio

ns.

Gr

zes

&D

ecet

y,20

02O

rient

atio

njud

gmen

tTo

olpr

esen

tatio

nac

tivat

edfro

ntop

arie

talr

egio

ns.

Neu

rops

ycho

logy

Neg

riet

al.,

2007

;Pap

eoet

al.,

2010

Obje

ctnam

ing;

dem

onstr

atin

gobje

ctuse

Patie

nts

show

edde

ficits

inobje

ctnam

ing

butp

rese

rved

obje

ctuse

;oth

erpa

tient

ssh

owed

the

opp

osite

patte

rn.

ECpr

edic

tsobje

ctuse

and

nam

ing

(i.e.

sem

antic

info

rmat

ion)

shou

ldbe

impa

ired

toge

ther

.

Stro

ngfo

rmofe

mbo

dim

entc

annot

bem

aint

aine

d.

Cube

lliet

al.,

2000

Obje

ctnam

ing;

Pict

ure-

wor

dm

atch

ing

Patie

ntw

ithim

paire

dobje

ctuse

buti

ntac

tide

ntifi

catio

nH

alsb

and

etal

.,20

01D

escr

ibin

gfu

nctio

nPa

tient

with

impa

ired

obje

ctuse

buti

ntac

ttoo

luse

iden

tific

atio

nR

osci

etal

.200

3O

bject

nam

ing;

pict

ure-

wor

dm

atch

ing

Patie

ntw

ithap

raxi

cde

ficts

did

not

show

wors

epe

rform

ance

with

man

ipul

able

vs.

nonm

anip

ulab

leobje

ctsLh

erm

itte,

1983

Patie

nts

with

front

opar

ieta

lles

ions

spon

tane

ously

reac

hed

fora

nd

use

dvisu

ally

pres

ente

dobje

ctsSu

gges

tstig

htco

upl

ing

ofv

isual

and

moto

rpr

oces

ses.

Doe

snot

show

that

moto

rpr

oces

ses

are

nec

essa

ryfo

rcogn

itive

task

s.N

euro

phys

iolo

gy(no

nhum

anpr

imat

e)M

urat

aet

al.,

1997

Obje

ctre

achi

ngan

dgr

aspi

ngSi

ngle

neu

rons

that

are

activ

ew

hen

reac

hing

foro

bject

are

also

activ

edu

ring

the

visu

alpr

esen

tatio

noft

heobje

ct.

May

refle

ctac

tivity

ofh

ighe

rord

erco

nver

genc

ezo

ne

inte

grat

ing

visu

alan

dm

oto

rac

tivity

.

Unc

lear

whe

ther

thes

eneu

rons

are

use

din

cogn

itive

task

s.

Ferr

arie

tal

.,20

05To

oluse

obs

erva

tion

and

goal

dire

cted

reac

hing

Sing

leneu

ron

resp

onds

toth

eobs

erva

tion

oft

oolu

se(bu

tnot

sam

eac

tion

with

outt

ool)

and

tope

rform

ing

the

sam

eac

tion.

Dev

elop

men

tK

aln

ine

&B

onth

oux,

2008

Obje

ctre

latio

nshi

pca

tego

rizat

ion

5-ye

ar-o

lds

dete

ctth

emat

icre

latio

nshi

ps(sc

rewdri

ver

scre

w)f

aste

rfor

man

ipul

able

obje

ctsth

annonm

anip

ulab

leobje

cts.

Act

ion

expe

rienc

eaf

fect

sobje

ctca

tego

rizat

ion

asyo

ung

as5

year

s.U

ncle

arw

heth

ervisi

ona

ctio

nre

latio

nshi

pssh

ould

incr

ease

or

decr

ease

with

age;

uncl

ear

whe

ther

actio

nex

perie

nce

isca

usa

llynec

essa

ryin

obje

ctre

cogn

ition

.M

ouno

udet

al.,

2007

Obje

ctnam

ing

and

cate

goriz

atio

n(pr

imed

byac

tion

pant

omim

es)

Com

patib

leac

tion

pant

omim

espr

ime

reco

gniti

onas

youn

gas

5ye

ars

old

.B

ehav

ior

Crai

gher

oet

al.,

1996

Rea

chin

gan

dgr

aspi

ngto

bars

(prim

edby

pict

ures

ofo

rient

edba

rs)

Gra

spin

gw

asfa

sterw

hen

prim

ew

asco

ngr

uent

with

the

prod

uced

actio

n,su

gges

ting

that

obje

ctpe

rcep

tion

faci

litat

esco

ngr

uent

actio

ns.

Sugg

ests

that

the

pres

enta

tion

ofa

visu

alobje

ctau

tom

atic

ally

activ

ates

actio

nre

pres

enta

tions

,fac

ilita

ting

man

ual

actio

ns.

Mou

ntin

gev

iden

cesu

gges

tsth

atth

ese

type

soff

acili

tatio

nef

fect

sha

veth

eirl

ocus

inat

tent

iona

lbi

ases

.M

asso

net

al.,

2011

;see

also

Bub

&M

asso

n,20

10;B

ubet

al.,

2008

;Tip

per,

Paul

,&H

ayes

,20

06;T

ucke

r&El

lis,1

998;

Tuck

er&

Ellis

,200

1

Rea

chin

gan

dgr

aspi

ng(pr

imed

bypi

ctur

esofh

andl

edobje

cts)

Gra

spin

gw

asfa

sterw

hen

prim

es

hand

lew

asco

ngr

uent

with

the

prod

uced

actio

n,su

gges

ting

that

obje

ctpe

rcep

tion

faci

litat

esco

ngr

uent

actio

ns.

Sugg

ests

that

the

pres

enta

tion

ofa

visu

alobje

ctau

tom

atic

ally

activ

ates

actio

nre

pres

enta

tions

,fac

ilita

ting

spec

ific

man

ual

actio

ns.

Unc

lear

whe

ther

thes

ere

sults

are

casu

ally

rela

ted

toco

gniti

veta

skor

refle

ctm

ore

gene

rally

reac

hing

and

gras

ping

.

Witt

etal

.,20

10O

bject

nam

ing

Perfo

rmin

ga

concu

rren

tm

oto

rta

skim

pairs

nam

ing

of

man

ipul

able

butn

ot

nonm

anip

ulab

leobje

cts.

This

sugg

ests

that

moto

rsim

ulat

ions

are

causa

llyre

late

dto

obje

ctnam

ing.

How

ever

,thi

sm

ain

findi

ngdi

dnot

repl

icat

ew

ithoth

erobje

ctse

t;ad

ditio

nally

,litt

leev

iden

cefo

rm

oto

rin

terfe

renc

eef

fect

sfro

mw

ork

ing

mem

ory

.

Note

.EC

embo

died

cogn

ition

.

This

docu

men

tis

copy

right

edby

the

Am

eric

anPs

ycho

logi

calA

ssoc

iatio

nor

one

ofi

tsal

lied

publ

isher

s.Th

isar

ticle

isin

tend

edso

lely

fort

hepe

rson

aluse

oft

hein

divi

dual

use

ran

dis

not

tobe

diss

emin

ated

broa

dly.


reported in an upright/inverted orientation judgment task of ma-nipulable objects (Grzes & Decety, 2002; see also Devlin et al.,2002, for a relevant meta-analysis). Overall, these neuroimagingstudies are consistent with the theory of ECs prediction of motorinvolvement for manipulable objects but not nonmanipulable ob-jects.

However, the neuroimaging findings are subject to an importantlimitation in that they are strictly correlational. Findings of motoractivation during the visual presentation of manipulable objectscannot be used to argue that such activity is necessary for objectperception; activity in motor regions may be inconsequential forperforming cognitive tasks like object naming (see also Mahon &Caramazza, 2008). Alternatively, observed frontoparietal activitymight simply reflect a covert, automatic preparation for action onan object (e.g., grasping) that has nothing to do with the cognitivetask (i.e., object naming). Thus, the neuroimaging evidence doesnot allow us to make strong claims about the role of motor activityin cognitive tasks and provides only correlational evidence.

NeuropsychologyResults from studies of the cognitive performance of brain-

damaged individuals are relevant to the predictions of embodiedobject representations. Specifically, damage to modality specificcortices used during simulations should impair cognitive perfor-mance (e.g., object naming). It is well documented that lesions todifferent regions of the cortex can result in category-specificdeficits in object recognition. Warrington and Shallice (1984), forexample, described patients who had deficits in identifying ani-mals but relatively intact identification of tools. The reverse pat-tern has also been shown (e.g., Gonnerman, Andersen, Devlin,Kempler, & Seidenberg, 1997; see also Hillis & Caramazza, 1991,for a double dissociation between manipulable objects and non-manipulable objects; see Capitani, Laiacona, Mahon, & Cara-mazza, 2003, for a discussion of the interpretation of these disso-ciations). A double dissociation can suggest that objectrepresentations are distributed over different cortical regions andprovides evidence that different experiences with manipulable andnonmanipulable objects result in distinct neural representations(see also Masullo et al., 2012). However, given the nature andextent of the lesions reported in these studies (i.e., clearly definedlesions to motor regions are not always present), double dissoci-ations do not provide strong support for the notion that motorregions are necessary for processing manipulable objects.

The importance of sensorimotor activity in processing manipu-lable objects is more comprehensively addressed in studies ofpatients with visual apraxia. Apraxia is a multifaceted disorderresulting in impaired object use (i.e., a deficit in action production)after lesions to different cortical regions (see Petreska, Adriani,Blanke, & Billard, 2007, for a detailed review; see De Renzi,Faglioni, & Sorgato, 1982). Importantly, the EC account of objectrepresentation makes a clear prediction about the effects of apraxiclesions on object processing. Specifically, if motor simulationsplay a role in representing knowledge about an object, then apraxicpatientsunable to use intact simulations due to cortical dam-ageshould show deficits in visual cognitive tasks (e.g., naming,categorizing, etc.) pertaining to manipulable objects. This predic-tion has been supported by Negri et al. (2007), who reported fourpatients who showed deficits in a simple object-naming task while

retaining the ability to demonstrate the objects use; however,these authors also report four patients who showed the oppositepattern, with impaired object use but retained object naming (seealso Papeo, Negri, Zadini, & Ida Rumiati, 2010). Additionally,Cubelli, Marchetti, Boscolo, and Della Sala (2000) described anapraxic patient with intact picture naming and picturewordmatching, while Halsband et al. (2001) reported a small number ofapraxic patients who could still provide verbal semantic informa-tion about tool functions. Finally, using a standard set of pictures,Rosci, Chiesa, Laiacona, and Capitani (2003) reported that apraxicpatients did not show worse naming performance of manipulablevs. nonmanipulable objects.

Overall, it is clear that the neuropsychological dissociationwithin apraxia is at odds with a strong form of the EC account ofobject representations; patients can perform visual cognitive tasks(e.g., naming, describing functional knowledge) despite impairedmotor execution or planning (due to parietal or frontal damage).5This is evidence that the strong form of EC cannot be maintained.

There are other neuropsychological conditions that are impor-tant to the EC account of object representations. For instance, in awell-cited report, Lhermitte (1983) described what he calls utili-zation behavior, in which neuropsychological patients spontane-ously act upon objects (i.e., reaching for them, grasping them, andsometimes putting the objects to use) upon the visual presentationof the object. A full description is useful:

While seated, the patient took a glass, gave it to the examiner and thenpicked up a jug. He poured water into the glass and, having put downthe jug, took the glass from the hand of the examiner and drank thewater. Taking a pack of cigarettes, he hesitated a moment, thenopened it and drew out a cigarette. He looked puzzled at it, being anonsmoker. A few seconds later, he held it to the mouth of theexaminer who accepted it and taking the lighter which was in theexaminers hand, near his knees, the patient lit the cigarette. Ques-tioned on this behaviour, he simply said You held out objects to me;I thought I had to use them. (Lhermitte, 1983, pp. 245246)

An important feature of this behavior is that the examiner doesnot instruct the patient to do anything, nor does the patient reportintending to act on it. In fact, patients often observe their ownbehavior with confusion. One patient was shown different pairs ofsunglasses during examination and was wearing all three simulta-neously by the end of the session. In these patients, neurologicaldamage was restricted to the frontal lobes, leading Lhermitte(1983) to suggest that utilization behavior reflected a lack ofinhibition of automatically activated motor behaviors toward ob-jects. In other words, it is possible that utilization behavior reflectsthe involuntary nature of motor activation in response to a visualobject; this would suggest that motor activations simply reflect theassociation between action and vision, but does not allow us tomake strong claims about the necessity of motor activity in cog-nitive tasks as put forth by the theory of EC.

5 Note that a number of apraxic patients do show deficits in both namingand using objects, demonstrating that the deficits may commonly co-occur.Again, however, this only suggests that these two functions are correlatedin some patients (see Buxbaum, Kyle, & Menon, 2005 for a similarreasoning with regard to the relationship between action production andaction understanding).

This

docu

men

tis

copy

right

edby

the

Am

eric

anPs

ycho

logi

calA

ssoc

iatio

nor

one

ofi

tsal

lied

publ

isher

s.Th

isar

ticle

isin

tend

edso

lely

fort

hepe

rson

aluse

oft

hein

divi

dual

use

ran

dis

not

tobe

diss

emin

ated

broa

dly.


Taken together, neuropsychological results certainly suggest arelationship between the visual presentation of objects and theactivation of motor representations, but again fail to show that theintact sensorimotor retroactivations are necessary for cognitivetasks. The double dissociation technique has been criticized be-cause simulations have shown that double dissociations can arisewith distributed damage to a single network in addition to focaldamage to a small number of unique networks (Devlin, Gonner-man, Andersen, & Seidenberg, 1998). For the theory of EC, thismay limit the interpretation of any neuropsychological result.6However, over and above this limitation, there are a number ofadditional caveats to the conclusion that the neuropsychologicalliterature does not favor EC, which is likely why the neuropsy-chological literature has not limited the development of EC. First,research has shown that substantial alterations to brain organiza-tion and function may occur after brain damage (i.e., plasticity;e.g., Chollet et al., 1991) and/or patients may develop compensa-tory strategies that make it seem as if certain abilities are intactwhen instead the patient has learned to perform a task in a newway. Indeed, some models of EC (e.g., Barsalou et al., 2003)predict a differential weighting of information in each modalitydepending on the goals of the observer. It is possible that afterbrain damage to motor regions, representations become moreheavily weighted to other intact modalities (auditory and visual) asa way of dealing with unavailable simulations in the damagedmodality. Second, few studies have attempted to identify moresubtle consequences of sensorimotor system damage, for instanceby investigating reaction times or three-dimensional movementtrajectories (e.g., Till, Masson, Bub, & Driessen, 2014). Suchmeasurements may reveal specific deficits that are overlookedwhen looking at overall performance accuracy. Regardless, thereview presented here shows that the strongest form of EC, inwhich sensorimotor simulations in modality specific cortices areabsolutely necessary for cognitive performance, cannot be main-tained (see also Mahon & Caramazza, 2008) and suggests that atthe very least the predictions made by EC must be qualified.

NeurophysiologyThe discovery of mirror neurons in the monkey frontal lobe (i.e.,

neurons that respond during both the performance and observationof action) has prompted much speculation about the brains ca-pacity to simulate other peoples actions, and it is often argued thatmirror neurons are the basis of action understanding (see Rizzolatti& Craighero, 2004). Indeed, mirror neurons appear to validatemany assumptions made by EC and serve as a neurophysiologicalbasis for simulation-based models such as CDZ and perceptualsymbols systems (see Gallese & Sinigaglia, 2011).

However, the role of mirror neurons in representing objectconcepts (rather than action concepts) is unclear. There is a sub-class of visuomotor neurons called canonical neurons in area F5 ofthe macaque frontal lobe. These neurons preferentially respond tothe presentation of objects or the performance of specific actionson those objects (see Murata et al., 1997). For instance, cells thatfire when the animal performs precision grip movements also fireduring the visual presentation of small objects that afford precisiongrips; similarly, those that fire during power grasps also respondpreferentially to the visual presentation of larger objects. Thesefindings suggest that canonical neurons code movements for in-

teracting with specific objects (Rizzolatti et al., 1988) and reflectthe activation of motor simulations during the visual presentationof an object (see Garbarini & Adenzato, 2004). Similarly, Ferrari,Rozzi, and Fogassi (2005) discovered a unique class of mirrorneurons that respond preferentially when a monkey observes an-other using a tool, but not to the same action without the tool or tothe presentation of the tool alone. Together, the different types ofmirror neurons may be a neurophysiological basis of a higher orderCDZ convergence zone that can initiate retroactivations acrossmultiple modalities (i.e., vision and sensorimotor). However,whether these neurons play a role in cognitive tasks remainsunknown. Given the lack of language in nonhuman primates,empirically investigating the direct role of mirror neurons, canon-ical neurons, or tool-responding mirror neurons during cognitivetasks remains a challenge. However, this does not rule out thepossibility of their involvement and remains an opportunity forfunctional imaging work to be done during human performance.

Development

One prediction of the EC is that experience with manipulableand nonmanipulable objects shapes embodied representations overa lifetime. According to one group, as a child accumulates expe-rience with objects, reliance on multiple modalities for represen-tation may actually decrease as more specific representations form(see Mounoud, Duscherer, Moy, & Perraudin, 2007). Lexicalresearch has shown that manipulability is important in conceptualprocessing throughout development (e.g., Borghi & Caramelli,2003), and research with physical objects shows that children asyoung as 4 years prefer to categorize novel objects on the basis offunction rather than visual similarity (e.g., Kemler Nelson, Fran-kenfield, Morris, & Blair, 2000). However, very little developmen-tal research has explored the nature of visual object representationsover time. Kalnine and Bonthoux (2008) have shown that chil-dren as young as 5 years old are able to detect that a picture of ascrewdriver matches with a picture of a screw (i.e., thematicrelationships) faster for manipulable objects than for nonmanipu-lable objects (e.g., castle matches with knight), a finding theysuggest reflects differential contributions of action experience toconcept representation. Mounoud et al. (2007) showed that inchildren as young as 5 years old the presentation of an action canprime object categorization (e.g., the presentation of a personsawing results in faster categorization of a saw as a tool), but thispriming decreases as children age. According to Mounoud et al.,this suggests that the role of experience in grounding conceptualrepresentations of objects is actually greater in early development,when interactions with the world dominate experience (see alsoPerraudin & Mounoud, 2009).

The developmental literature indirectly supports the notion thataction and object representations are associated early in develop-ment. However, given the dearth of research in this area, the exactdevelopmental trajectory of purported embodied object represen-tations remains unknown. Further, we argue that the assumption ofstronger embodiment in early development is flawed. That is, asone accumulates experience and correlated sensorimotor activityoccurs more frequently for manipulable objects (i.e., correlationsbetween actions and visual presentation of objects), models based

6 We thank an anonymous reviewer for this suggestion.

This

docu

men

tis

copy

right

edby

the

Am

eric

anPs

ycho

logi

calA

ssoc

iatio

nor

one

ofi

tsal

lied

publ

isher

s.Th

isar

ticle

isin

tend

edso

lely

fort

hepe

rson

aluse

oft

hein

divi

dual

use

ran

dis

not

tobe

diss

emin

ated

broa

dly.


on Hebbian cell assemblies should predict stronger relationshipsbetween them. Thus, it is conceivable that embodied effects shouldgrow stronger in development, not weaker; this has yet to be testedin any paradigm. Finally, as with the research reviewed in preced-ing sections, there is little evidence that action simulations are anecessary component of manipulable object processing in earlydevelopment. Future developmental research should explore thestrength of the vision-action relationship through infancy.

Behavioral ParadigmsIncreasingly, EC is influencing behavioral investigations of

visual cognition. It is now well established that the visual presen-tation of an object can affect both action production (e.g., grasp-ing) and object recognition (e.g., naming). For instance, Craighero,Fadiga, Umilta`, & Rizzolatti (1996) had participants grasp barsthat were oriented 45 from upright. On some of the trials, a visualprime of a bar oriented in the same way (i.e., a congruent visualprime) was shown before the grasp, and on others a visual primeof a bar oriented in the opposite way (i.e., an incongruent visualprime) was shown. The authors reported that grasping was fasteron congruent trials and interpreted this as evidence that the visualpresentation of a graspable object automatically primes relatedmotor actions. This type of result is consistent with a body ofresearch that suggests a direct coupling between object perceptionand action. In a more specific demonstration of the couplingbetween object perception and action, Masson, Bub, and Breuer(2011) showed that manual actions toward objects are facilitatedby the presentation of visual objects whose handles are orientatedcongruently with the targets. For example, visually presenting apicture of a beer mug with its handle oriented vertically facilitatedvertical grasps to a grasping apparatus. These results suggestfurther that the presentation of a visual object automatically acti-vates action representations (see also Bub & Masson, 2010; Bub,Masson, & Cree, 2008; Tipper, Paul, & Hayes, 2006; Tucker &Ellis, 1998; Tucker & Ellis, 2001).

The behavioral literature suggests that the visual presentation ofan object can automatically activate actions, influencing motorresponses in cognitive tasks. These findings are consistent with ECand support one of its main predictions: namely, that visual pre-sentation of an object will activate motor activity associated withthat object. However, a growing body of behavioral researchsuggests that an EC interpretation of these results may not becorrect. Specifically, stimulusresponse compatibility effects (e.g.,facilitated responding with the hand that is compatible with theorientation of an objects handle) may be better explained by thelocation of visual attention. For instance, Matheson, White, andMcMullen (2014a) had participants make upright/inverted or cat-egory judgments about left- and right-oriented tools and animals.They showed that participants were faster at responding with thehand that was compatible with the orientation of the tools handlein the upright/inverted judgment task. However, responses werealso faster with the hand that was compatible with the animalheads. Facilitated responses to animal heads or tails are not pre-dicted by EC (because humans do not often manipulate largeanimals) and suggest a more domain-general mechanism.

Importantly, in some tasks, motor facilitation effects are notrestricted to the manipulable part of the object. For instance,Anderson, Yamagishi, and Karavia (2002) presented participants

with pictures of scissors or clocks while they made orientationjudgments, and observed faster responding when the responsehand was compatible with the orientation of the scissors handlesand the orientation of the clocks hands (see also Phillips & Ward,2002). If motor associations underlie these effects, then facilitationshould have been found only with the scissors and not the clock(see also Vainio, Ellis, & Tucker, 2007, Experiment 2; Cho &Proctor, 2013; Cho & Proctor 2010; Cho & Proctor, 2011; Pelli-cano, Iani, Borghi, Rubichi, & Nicoletti, 2010). Together, theseresults suggest that facilitation effects that have been previouslyinterpreted as embodied effects may instead reflect a more general,task-dependent, stimulusresponse compatibility that is driven bythe locus of attention. Such an account suggests that some featuresof objects grab attention, and recent electrophysiological investi-gations of this interpretation provide support for the notion that, ingeneral, object handles do attract visual attention (Matheson, New-man, Satel, & McMullen, 2014). This conclusion was also reachedby Proctor and Miles (2014) who suggested that these stimulusresponse compatibility effects are best understood as arising fromdomain-general spatial biases and argue that these results shouldno longer be invoked as support of the automatic activation ofmotor representations to manipulable objects.

Spatial attentional biases may best account for the results ofstudies using simple button presses as responses. Importantly, Buband Masson (2010) directly compared button press responses toreaching and grasping. These authors showed that liftoff times forreaching toward a grasping apparatus were faster when visuallypresented objects were aligned compatibly with the reach. Thissuggests that some aspects of motor simulation may be invoked bythe visual presentation of the object, just not more general simu-lations that potentiate key-presses. A more specific effect was alsoreported by Tucker and Ellis (2001), who had participants catego-rize objects by making precision or power grips on a graspingapparatus. They reported that participants were faster at respond-ing when a visually presented object afforded a compatible action.Additionally, measuring three-dimensional movement trajectories,Till et al. (2014) have reported that a visually presented manipu-lable objects affect the online trajectories of reaching and grasping,biasing them in ways that are compatible with the presented object.Unlike the paradigms using button presses, these types of resultsdo provide support for the notion the visual presentation of amanipulable object invokes sensorimotor simulations (see alsoMasson et al., 2011). While measuring specific grasp apertures andmovement trajectories has promise for supporting more nuancedpredictions of EC, and while these measures a not likely explainedby spatial attentional biases, they still suffer from many of thecriticisms of other research in EC. Specifically, at best they showthat visually presented objects activate motor simulations, but theystill do not provide evidence that motor simulations are a part ofthe representation of visual objects, or that they are used toperform cognitive tasks. Further, it is unclear whether similareffects are observed for nonmanipulable objects.

To provide support for the causal role of sensorimotor simula-tions in cognition, it is necessary to show that disrupting simula-tions impairs cognitive performance (e.g., naming or categoriza-tion) for manipulable objects but not nonmanipulable objects. Thisquestion has been recently investigated by a number of researchersusing motor interference paradigms. For instance, Witt, Kem-merer, Linkenauger, and Culham (2010) had participants name

This

docu

men

tis

copy

right

edby

the

Am

eric

anPs

ycho

logi

calA

ssoc

iatio

nor

one

ofi

tsal

lied

publ

isher

s.Th

isar

ticle

isin

tend

edso

lely

fort

hepe

rson

aluse

oft

hein

divi

dual

use

ran

dis

not

tobe

diss

emin

ated

broa

dly.


pictures of tools and animals while they squeezed a sponge. Theyreasoned that if motor simulations are necessary for efficientsemantic processing, then squeezing a sponge will slow naming ofmanipulable objects but not nonmanipulable objects. This is in-deed what they showed, in the first study to provide such strongevidence in favor of EC. However, this interference effect was notreplicated using a different set of objects; indeed, manipulating thepicture-plane orientation of the images reversed the effect (Mathe-son, White, & McMullen, 2014b), suggesting again a more gen-eral, attentional locus of the effect. Other researchers have ex-plored the effects of concurrent motor tasks in other cognitivedomains. For instance, Pecher (2013) showed that a concurrentmotor task did not specifically impair working memory perfor-mance (holding information about the object in working memoryover a short delay) for manipulable objects versus nonmanipulableobjects. She concluded that motor simulations are not necessaryfor working memory performance. Using a similar logic, Postle,Ashton, McFarland, and de Zubicaray (2013) showed that readinghand-related words did not affect performance of a hand action anymore than reading other effector-related words (e.g., feet). To-gether then, with the exception of one study, the results from motorinterference paradigms do not support EC. While there are anumber of methodological reasons that this may be the case (e.g.,the timing of the motor interference, the specificity of the effectorsand actions involved, etc.), and while motor interference para-digms should be used to test EC predictions more specifically, thecurrent available results do not provide evidence for EC.

Overall, behavioral research has shown some evidence consis-tent with activation of motor associations to visually presentedmanipulable objects. However, a growing body of literature sug-gests that such findings may be more consistent with generalattentional effects and subsequent stimulusresponse compatibilitythat are not a consequence of visual-motor simulations, and thus,not supportive of a strong form EC in visual object perception.Additionally, interference effects are not prevalent. Though re-fined behavioral paradigms that measure more specific conse-quences of motor simulations may useful in future research (e.g.,Till et al., 2014), the current behavioral evidence in favor of EC isnot compelling.

Discussion

Summary and Conclusion

We have reviewed a large body of research from neuroimaging,neurophysiological, neuropsychological, developmental, and be-havioral studies that are relevant to the EC account of objectrepresentations. On the basis of the above review, we argue thatthe literature does not support a strong form of EC in visual objectperception and is instead better accounted for by known relation-ships between vision, action, and domain-general processes suchas visual attention. On the basis of careful consideration of re-search relevant to the main predictions of EC, we conclude thatsensorimotor simulations are not engaged for, or at least notevidenced by, the types of visual cognitive tasks reviewed here(e.g., picture or object naming).

First, neuroimaging studies show only correlational relation-ships between visual and motor representations for manipulableobjects and do not support a causal link between the two. Second,

the neuropsychological literature shows us that, though deficits invisual cognitive tasks sometimes co-occur with apraxic disorders,many apraxic patients retain the ability to name manipulableobjects, providing support for the conclusion that motor simula-tions are not necessary to perform visual cognitive tasks. Third,though the neurophysiological literature has identified neural as-semblies that integrate information about action and vision, thefunctional role of these assemblies in object processing is stilllargely unknown. Fourth, research in child development showsthat relationships between visual and motor activity may becomeweaker throughout development but, as with the neuroimagingdata reviewed here, there is no evidence that motor simulations arenecessary for object processing. Finally, the behavioral literatureusing visual cognitive paradigms shows motor-priming effects invisual tasks, but these effects are likely best accounted for bydomain-general attentional biases. In sum, there is presently littleevidence supporting the predictions of a strong form of the theoryof embodied cognition in the representation of object concepts invisual cognitive tasks.

Instead, we conclude that visual object perception (i.e., recog-nition, identification) does not require activity in motor regionsand can often be carried out on the basis of visual informationalone. This is not to suggest that motor activity is independent ofvisual processing, but rather that the paradigm and/or the stimulusitself constrains the involvement of motor systems in visual pro-cessing. For example, when one views a hammer, the visualstimulus is rich enough to specify information about the hammerwithout the need to invoke embodied processes (i.e., simulations ofauditory, motoric, and additional sensory experiences). It seemslikely that much of the motor activity recorded in imaging tasksduring object perception is more about specifying potential actionthan specifying the object concept. Similarly, some of the behav-ioral evidence suggests that attention is directed to object parts thatare the likely targets of action. Again, this type of activity does notreflect a retroactive simulation used for completing visual cogni-tive tasks (e.g., object naming) but rather simply reflects the needto isolate manipulable object features for the purpose of actingupon them. Overall, then, the interpretation of the results reviewedhere puts constraint on EC with respect to visual object represen-tations in these cognitive tasks.

What Evidence Is Needed to Supportthe Theory of EC?

These conclusions raise a number of questions about the kind ofevidence that would be needed to support the theory of EC invisual cognitive paradigms. There are at least three questions thatfuture research must explore. First, because neuroplastic reorgani-zation or compensatory strategies might allow apraxic patients tosuccessfully perform visual cognitive tasks, future research shouldmanipulate motor cortex directly by altering motor activity in thehealthy brain. In the psycholinguistics literature, paradigms usingtranscranial magnetic stimulation (TMS) have shown that stimu-lating arm representations in primary motor cortex facilitates pro-cessing of arm-related verbs (e.g., throw) but not leg-related verbs(Pulvermller et al., 2005). Further, stimulating the motor cortexincreases muscle potentials in the response hand that is compatiblewith the orientation of a visually presented mug handle (Buccino,Sato, Cattaneo, Roda`, & Riggio, 2009; see also Cardellicchio,

This

docu

men

tis

copy

right

edby

the

Am

eric

anPs

ycho

logi

calA

ssoc

iatio

nor

one

ofi

tsal

lied

publ

isher

s.Th

isar

ticle

isin

tend

edso

lely

fort

hepe

rson

aluse

oft

hein

divi

dual

use

ran

dis

not

tobe

diss

emin

ated

broa

dly.


Sinigaglia, & Constantini, 2011). Future research should explorewhether TMS applied to motor regions responsible for hand move-ment specifically increases the amplitude of motor-evoked poten-tials elicited during perception of manipulable objects compared tononmanipulable objects; the theory of EC predicts that this wouldbe the case. Further, such a paradigm is useful for determining therelationship between motor-evoked potentials and visual cognitiveperformance.

A second type of evidence needed to strongly support ECaccounts of object representation can be gleaned from behavioralinterference paradigms. As discussed above, a large behavioralliterature suggests that viewing a manipulable object facilitatesmotor responses. Interfering with these motor responses (andtherefore disrupting the normal processes that occur during asimulation) should disrupt visual cognitive performance (e.g., Wittet al., 2010). We argue that though the results have not beenpromising, methodological considerations may be more fruitful infinding support for EC by, for instance, manipulating the speci-ficity of the motor involvement (see Till et al., 2014) or the timingof interference (see Bub & Masson, 2010). Future research shouldcontinue to explore the extent to which concurrent motor tasks arecapable of interfering with visual cognitive performance in tasksthat more tightly control the timing and extent of motor interfer-ence.

The third piece of evidence necessary to support the EC accountof object representations relates to the timing of embodied simu-lations, which can be studied via temporally sensitive methodssuch as electro- or magneto-encephalography (EEG and MEG,respectively). Critically, if activity associated with the motor sys-tem reflects functional simulation of an object concept, then itshould occur early in object processing; conversely, if this activityis merely correlational (e.g., as a consequence of postsemanticimagery), then it should occur later (see van Elk, van Schie,Zwaan, & Bekkering, 2010; Hauk, Shtyrov, & Pulvermller, 2008for discussions of this issue in linguistic research). Using a task inwhich participants were shown manipulable and nonmanipulableobjects, at least one EEG study has shown that manipulable objectsare differentiated from nonmanipulable objects by about 270 mspoststimulus and has localized this difference to motor regions(Proverbio, Adorni, & DAniello, 2011). However, because par-ticipants were engaged in a task detecting the appearance ofoddball stimuli (i.e., plants) and not explicitly processing themanipulable objects, it is still unclear whether such activity isrelated to cognitive performance such as naming or categorization.Future research should explore whether manipulable objects aredifferentiated from nonmanipulable objects at stages that precedeknown cognitive effects (e.g., P3 or N400) during tasks thatinvolve conceptual processing.

Constraining Theories of Embodied CognitionIn general, there are a number of criticisms of EC (e.g., Dove,

2011). Some of these issues are not specific to the visual domainand have been raised in the linguistic literature as well (e.g.,Fischer & Zwaan, 2008; Kiefer & Pulvermller, 2012). For in-stance, many authors maintain that modality-specific activity isinsufficient to represent all semantic information (e.g., Shelton &Caramazza, 1999). In the literature, widespread semantic impair-ments are easily explained by amodal accounts (e.g., Hillis, Rapp,

Romani, & Caramazza, 1990), for instance, by hypothesizingdifferent stores of semantic information or differences in howthese stores are accessed. This raises the important question as towhether an embodied perspective is even necessary. Shapiro(2011) points out succinctly that in some cases it is not clearwhether embodied hypotheses and amodal information processinghypotheses are mutually exclusive (indeed, they may not even beasking the same types of questions). For instance, some resultsmight be best explained with embodied theories (e.g., behavioralmotor-interference effects), while others might be best suited toamodal accounts (e.g., the naming abilities of apraxic individuals),much as light can be described in terms of waves and in terms ofparticles. If these theories are not mutually exclusive, then eluci-dating object representations from both perspectives will provide amore complete understanding of the nature of object representa-tion.

At the very least, recent reviews highlight the futility in exam-ining the embodied hypothesis as a binary question, concludinginstead that future research must investigate when and how sen-sorimotor activity is recruited in cognitive tasks (Willems &Francken, 2012). Some authors suggest that a spectrum of amodal(or multimodal) and modality-specific processes (embodied anddisembodied processes) contribute to the full range of concrete andabstract semantic processing (Pulvermller, 2013). This conclu-sion is echoed by Zwaan (2014), who, in a discussion of embodiedlanguage, has suggested adopting a pluralist perspective: Embod-ied cognition is needed to explain how we build representations ofthe world, while amodal accounts are needed to address the mostabstract aspects of cognition (e.g., metaphor; though see Lakoff &Johnson, 1999). Zwaan (2014) posits five levels of embodiment (inpsycholinguistics) that differ in the degree to which comprehen-sion relies on embodied versus amodal representations. Indeed, acontinuum of amodal and modality-specific or embodied repre-sentation may better account for the available data and placesemphasis on the need for disembodied cognition in tasks thathappen quickly and effortlessly (e.g., in following the speech ofanother person; see Binder & Desai, 2011). Some philosophicalauthors maintain that extreme embodiment can be rejected becausecertain cognitive processes (e.g., motor control) cannot be under-stood without recourse to sensorimotor representation and abstract,creative thought, requires the combination of sensorimotor repre-sentational content (e.g., Thagard, 2012). This line of thought isreflected in the neuroimaging literature investigating how combi-nations of sensorimotor features are selected and represented (e.g.,Coutanche & Thompson-Schill, 2014). Similarly, in a review ofthe empirical literature on linguistic processing, Meteyard,Cuadrado, Bahrami, and Vigliocco (2010) have suggested thatpurely amodal, symbolic theories and strong forms of embodimentcan both be rejected; what remains is determining which aspects ofthe evidence for embodiment constitute representational contentand which aspects are merely correlated with that contentor asZwaan (2014) suggests, When do we need which and how do theyinteract? (p. 230).

The current review challenges the EC account of object repre-sentation when representations are accessed from vision. However,this conclusion only constrains EC, helps develop it further, anddoes not provide evidence against the general principles of theapproach or models like the CDZ (Meyer & Damasio, 2009).Indeed, evidence for embodied representations is robust, especially

This

docu

men

tis

copy

right

edby

the

Am

eric

anPs

ycho

logi

calA

ssoc

iatio

nor

one

ofi

tsal

lied

publ

isher

s.Th

isar

ticle

isin

tend

edso

lely

fort

hepe

rson

aluse

oft

hein

divi

dual

use

ran

dis

not

tobe

diss

emin

ated

broa

dly.


in linguistic paradigms and studies of action understanding (Bar-salou, 2008). Given the discrepancies between these literatures andthe literature of embodied visual object processing, we suggest thatthe theory of embodied cognition and the simulation processes itposits might not apply equally across all cognitive domains. In-deed, it appears that embodied effects may apply differentially todifferent domains, with a strong embodiment in linguistic domain,weaker in action understanding, and weakest still in visual cogni-tive tasks (see also Meteyard et al., 2012, for ideas concerningembodiment and the depth of processing). We suggest that therelative reliance on sensorimotor simulations changes dependingon how far removed the stimulus is from direct experience. Withvisual objects, simulations are less important because much of thevisual information is available in view; in contrast, with words,much more elaborate simulations are required to fully make senseof the stimulus and make use of it in cognitive tasks. Overall, thishypothesis suggests that simulation/retroactivation processes maybe more useful for some cognitive tasks than others. This conclu-sion is reached by Zwaan (2014), who suggests that embodiedsimulations may be more important for recognizing a word thatrefers to a tool, rather than identifying a tool that is in view.Though this is speculative, this proposal suggests that future re-search will need to explore the ways in which embodied effects aremanifested in different cognitive domains and opens the possibilityof domain-specific embodiment.

Embodiment and BeyondThough information processing theories of object processing

and other cognitive functions continue to dominate the psycholog-ical literature, research that takes an embodied perspective (or atleast claim to take an embodied perspective) is increasingly chal-lenging these approaches. The excitement the embodied approachhas generated promises to offer new insights into not only howobjects in the world are processed, but how cognition unfolds inthe environment more generally. Importantly, the predictions ofembodiment are applicable beyond object processing and havebeen used in an attempt to understand a wide variety of psycho-logical domains, including language use (e.g., Pulvermller,2013), human emotions and empathy (e.g., Niedenthal, 2007), theunderstanding of developmental disorders such as autism spectrumdisorder (e.g., Eigsti, 2013) and other neuropsychiatric conditions(e.g., Parkinsons; Kemmerer, Miller, MacPherson, Huber, &Tranel, 2013), in addition to child development (Byrge, Sporns, &Smith, 2014) and applied areas such as educational psychology(Ionescu & Vasc, 2014). Indeed, one attractive feature of theembodied approach is that it may offer a way of unifying thediverse areas within psychology and neuroscience (Matheson &White, 2011; see also Glenberg, 2010, for a discussion of theunifying potential of the concept of embodiment within psychol-ogy). However, while researchers are discovering clever ways ofinvestigating the predictions of EC within these various domainsof psychology, our review here points to the importance of tem-pering excitement about findings that are merely consistent withembodied predictions and suggests the need for a critical assess-ment of whether experimental findings in these areas do in factoffer evidence in favor of embodied accounts. Indeed, at this stagein the literature, our review shows that excitement about support-ing the theory may lead to overinterpretations that are just not

justified based on the data, or may blind researchers from seekingalternative, simpler, more domain general (or amodal) explana-tions for the data. Indeed, we hope to show that it is not sufficient,based on the conceptualization hypothesis, to show that the bodyinteracts with some form of psychological processes (e.g., bodybalance influencing political attributions; see Dijkstra, Eerland,Zijlmans, & Post, 2012) and that it is important to fully describethe assumptions of the embodied theory we are adopting and whatthe predictions for brain and behavior will be (see also A. D.Wilson & Golonka, 2013). In the present review, we have provideda case study in the domain of visual cognition. We feel that todevelop this theory to its full explanatory power will requirecritically examining the existing evidence, determining whichquestions need addressed, and identifying the methods and depen-dent measures that are best suited to doing so in all domains ofpsychology and neuroscience.

ReferencesAllport, D. A. (1985). Distributed memory, modular subsystems and dys-

phasia. In S. D. Newman & R. Epstein (Eds.), Current perspectives indysphasia (pp. 207244). New York, NY: Churchill Livingstone.

Anderson, S. J., Yamagishi, N., & Karavia, V. (2002). Attentional pro-cesses link perception and action. Proceedings of the Royal Society ofLondon Series B: Biological Sciences, 269, 12251232. doi:10.1098/rspb.2002.1998

Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychol-ogy, 59, 617645. doi:10.1146/annurev.psych.59.103006.093639

Barsalou, L. W. (2010). Grounded cognition: Past, present, and future.Topics in Cognitive Science, 2, 716724. doi:10.1111/j.1756-8765.2010.01115.x

Barsalou, L. W., Simmons, W. K., Barbey, A. K., & Wilson, C. D. (2003).Grounding conceptual knowledge in modality-specific systems. Trendsin Cognitive Sciences, 7, 8491. doi:10.1016/S1364-6613(02)00029-3

Binder, J. R., & Desai, R. H. (2011). The neurobiology of semanticmemory. Trends in Cognitive Sciences, 15, 527536. doi:10.1016/j.tics.2011.10.001

Borghi, A. M., & Caramelli, N. (2003). Situation bounded conceptualorganization in children: From action to spatial relations. CognitiveDevelopment, 18, 4960. doi:10.1016/S0885-2014(02)00161-2

Brooks, R. A. (1991, September 13). New approaches to robotics. Science,253, 12271232. doi:10.1126/science.253.5025.1227

Bub, D. N., & Masson, M. E. J. (2010). Grasping beer mugs: On thedynamics of alignment effects induced by handled objects. Journal ofExperimental Psychology: Human Perception and Performance, 36,341358. doi:10.1037/a0017606

Bub, D. N., Masson, M. E. J., & Cree, G. S. (2008). Evocation of functionaland volumetric gestural knowledge by objects and words. Cognition,106, 2758. doi:10.1016/j.cognition.2006.12.010

Buccino, G., Sato, M., Cattaneo, L., Roda`, F., & Riggio, L. (2009). Brokenaffordances, broken objects: A TMS study. Neuropsychologia, 47,30743078. doi:10.1016/j.neuropsychologia.2009.07.003

Buxbaum, L. J., Kyle, K. M., & Menon, R. (2005). On beyond mirrorneurons: Internal representations subserving imitation and recognition ofskilled object-related actions in humans. Cognitive Brain Research, 25,226239. doi:10.1016/j.cogbrainres.2005.05.014

Byrge, L., Sporns, O., & Smith, L. B. (2014). Developmental processemerges from extended brainbodybehavior networks. Trends in Cog-nitive Sciences, 18, 395403. doi:10.1016/j.tics.2014.04.010

Capitani, E., Laiacona, M., Mahon, B., & Caramazza, A. (2003). What arethe facts of semantic category-specific deficits? A critical review of theclinical evidence. Cognitive Neuropsychology, 20, 213261. doi:10.1080/02643290244000266

This

docu

men

tis

copy

right

edby

the

Am

eric

anPs

ycho

logi

calA

ssoc

iatio

nor

one

ofi

tsal

lied

publ

isher

s.Th

isar

ticle

isin

tend

edso

lely

fort

hepe

rson

aluse

oft

hein

divi

dual

use

ran

dis

not

tobe

diss

emin

ated

broa

dly.


Caramazza, A., & Mahon, B. Z. (2003). The organization of conceptualknowledge: The evidence from category-specific semantic deficits.Trends in Cognitive Sciences, 7, 354 361. doi:10.1016/S1364-6613(03)00159-1

Cardellicchio, P., Sinigaglia, C., & Constantini, M. (2011). The space ofaffordances: A TMS study. Neuropsychologia, 49, 13691372. doi:10.1016/j.neuropsychologia.2011.01.021

Chao, L. L., Haxby, J. V., & Martin, A. (1999). Attribute-based neuralsubstrates in temporal cortex for perceiving and knowing about objects.Nature Neuroscience, 2, 913919. doi:10.1038/13217

Chao, L. L., & Martin, A. (2000). Representation of manipulable man-made objects in the dorsal stream. NeuroImage, 12, 478484. doi:10.1006/nimg.2000.0635

Cho, D. T., & Proctor, R. W. (2010). The object-based Simon effect:Grasping affordance or relative location of the graspable part? Journal ofExperimental Psychology: Human Perception and Performance, 36,853861. doi:10.1037/a0019328

Cho, D. T., & Proctor, R. W. (2011). Correspondence effects for objectswith opposing left and right protrusions. Journal of Experimental Psy-chology: Human Perception and Performance, 37, 737749. doi:10.1037/a0021934

Cho, D. T., & Proctor, R. W. (2013). Object-based correspondence effectsfor action-relevant and surface-property judgments with keypress re-sponses: Evidence for a basis in spatial coding. Psychological Research,77, 618636. doi:10.1007/s00426-012-0458-4

Chollet, F., DiPiero, V., Wise, R. J. S., Brooks, D. J., Dolan, R. J., &Frackowiak, R. S. J. (1991). The functional anatomy of motor recoveryafter stroke in humans: A study with positron emission tomography.Annals of Neurology, 29, 6371. doi:10.1002/ana.410290112

Clark, A., & Chalmers, D. (1998). The extended mind.

Accesing Embodied Object Representations1

Documents

Transcript of Accesing Embodied Object Representations1