Visuospatial Representation

Spatial Knowledge, Imagery, Visual Memory

Representation

What is a representation? Four aspects of representation

The represented world The representing world Set of informational relations on how

the two correspond Set of processes that extract and use

information from the representation

Meaning Mental representations are carriers of

meaning In order to interact appropriately with the

environment we represent info from it and manipulate those representations

Correspondence Meaning derived from how representation

stands in consistent relation to the represented world

Conceptual Meaning determined by relations to other

representations

Spatial Knowledge

How we represent and use spatial information

Separate from strictly verbal knowledge Semantic propositions

Dependent on the linear dimension of space.

Spatial Cognition

How is the representing world like the represented world?

The represented world is a space The representing world is a space

What kinds of processes might be involved?

Space as a representation Spatial representation Representing world is a space. What is a

space? Geometric entity in which locations are specified

relative to a set of axes Dimensionality defined by the number of axes that

can point in independent directions Of interest is the distance between items, which can

be measured in different ways Euclidian

Straight line Non-independent dimensions

Saturation and brightness City-block

Distinct dimensions Color and size

Space as a representation Physical world experienced (at least

perceptually) has three dimensions (+ time)

However, the representing world is not confined to any number of dimensions

Represented world does not need to be spatial Conceptual info can be represented spatially More on that later

Spatial Representation

Analog representation Representation mimics the structure of

the represented world Multidimensional scaling

Propositional Abstract assertions regarding the state

of the represented world Not tied to a particular sensory modality

Multidimensional Scaling (MDS)

MDS Mathematical technique for taking a set of distances and

finding the best-fitting spatial configuration that corresponds to those distances

Input: a distance or proximity matrix that describes how close every object in a set is to every other object

N objects are represented by N(N-1)/2 numbers (distances) Output: a geometric representation where

every object is represented as a point in D-dimensional space

Each object is represented as a point in space N objects are represented by ND numbers (coordinates)

Purposes of MDS Give psychological interpretations to the dimensions Reveal the dimensionality of a data set

Difficult to get a sense of relative distance by means of this information

MDS

MDS recovers absolute original locations for the objects from the distances

Flipping on horizontal axis would give us a rough approximation of NSEW

Analog representation

MDS

Propositional Representation

(A,B) 10 miles east

(E,C) 20 miles south, 10 miles east

(F,D) 10 miles south, 10 miles west

Analog vs. Propositional Analog

Good for configural info Easy incorporation of new info

Propositional Time-consuming Lots of info must be represented

E.g. one point added may require many propositions

Allows for communication of spatial knowledge and incorporation of additional information not related to distance

Going south on I35, one must pass through Denton to get to either Fort Worth or Dallas

Cognitive Maps

Where is Seattle? Where is Terrill Hall?

Large vs. small-scale space Hierarchical representation

Small vs. Large-scale space

Maps of small-scale (navigable space) Cognitive geography

Maps of large-scale space What is our sense of the locations of

items in the world?

Small scale space Survey knowledge

Bird’s eye view (map knowledge) Good for global spatial relations Easy acquisition Not so great for orientation

Route knowledge Gained from navigating through the environment

Locate landmarks and routes within a general frame of reference

Landmark knowledge Salient points of reference in the environment

More difficult to acquire but better for navigation in irregular environments

May lead to survey knowledge Perhaps a different type Cognitive collage vs. orientation free

Large scale space

Which is farther north: Denton, TX or Chicago, IL? Portland, OR or Portland, ME?

Hierarchical representation of locations

Hierarchical representations Relative locations of smaller regions are

determined with respect to larger regions. States are superodinate to cities, countries

superordinate to states USA is south of Canada

Maine is just south of Canada Oregon is well south of Canada

Oregon must be south of Maine Cities in Oregon must be south of cities in Maine In this case such cognitive economy works against us

Portland OR is north of Portland ME

Hierarchical representations Judge relative

position of cities (Stevens and Coupe)

When superordinate info congruent with question, performance better

Is x north of y when one of right side maps presented

Hierarchical coding Huttenlocher & Hedges

Category-adjustment model Combine info across hierarchical levels

If info at subordinate is known with near certainty, there is no appeal to categorical info

If info at subordinate (fine-grained) levels is at all uncertain, people use categorical info in estimation

Bias toward center of category Bayesian approach utilizing prior knowledge

Gist: errors in estimation are due to categorization rather than nonmetric spatial relations

How are maps learned From descriptions

Taylor & Tversky: People learned maps from survey and route descriptions

From navigation People can assess distance and direction

traveled Integration of information

Visual information Vestibular information

Maps formed from video games are less accurate than maps in which people really move

Rotation is particularly important

Using spatial cognition Adaptive context

Locating and way finding Tool Use Mental rotation vs. mental movement

Symbolic representations of space Drawings, maps, models Language

Thinking

Adaptive context Locating and way finding Consider

Hatchling sea turtles finding the sea Salmon finding way back home

However these are more behavioral instinct and imprinting than pure navigation

Desert ant finding direct route home after meandering paths in featureless environment

Marsh tit stores seeds in holes in a hundred various places for later retrieval

Locating and way finding

Ego-centered systems Environment-centered systems Hierarchical coding

Locating and way finding Ego-centered system

Location of objects coded relative to self Updated as we move through the world Nonconscious

Rieser, Guth, Hill (1986) Participants asked to point out previously learned

locations in unfamiliar room after blindfolded and led along path

Did not matter whether previously told which location they’d be asked about, suggesting attentional focus did not assist in the process

Problem: may not always be accurate over larger distances without detailed environmental information

Locating and way finding Environment-centered system

Object location coded in relation to stable features of the environment

Requires feature-rich environment providing info to dominate sense used by organism

If conditions met, then superior to ego-centered

Allows for rechecking of position (no drift from accumulation of small errors)

Works better for retaining info over long periods of time

Cognitive maps Both humans and animals display errors

in judgment that cast doubt in positing a true ‘cognitive map’

Animal studies suggest approximation of distance from a single landmark

Humans make many errors in spatial judgments that suggest no real metric representation Distance from A to B judged different from B

to A Though again this sort of distortion may be

related to categorization (hierarchy)

Tool use

Making, using and designing tools for interaction with the environment involves cognitive processes such as mental rotation and imagery for success

Shephard & Metzler (1971)

Mental rotation vs. Mental Movement

Logically equivalent However evidence suggests that mental

rotation and perspective-taking/mental movement are psychologically distinct

Selection task Which these arrays/models would be the correct

view from over there? Item question

What object would be nearest to you if you were over there?

Specify frame of reference of relative to the observer

Mental rotation vs. Mental Movement

Selection task Piaget

Kids (< 10) not so hot at such a task Usually pick egocentric view

Huttenlocher & Presson They do much better when asked to do mentally rotate Can also physically move to new location that matches

a particular array Suggests conflict between current physically

present perspective and the new (imagined) one they are trying to obtain

MR allows them to stay put in the physically present room

Physical movement physically transforms that perspective

Mental rotation vs. Mental Movement

Item questions If kids do not move item questions help (even

as young as 3) Again, this helps them maintain that egocentric

perspective If asked to mentally rotate, item questions can

actually hurt performance compared to selection tasks

It may be that in item questions, whole array must be rotated to determine object relations vs a simple ‘rotation’ of the person or single object in selection task

Gist: mental rotation and mental movement can be differentially affected depending on the nature of the question asked, suggesting there may be different underlying processes involved

Drawings maps and models Spatial learning from maps differs from

learning by means of navigation Map learning may aid configural knowledge and allow

for better estimates of distance between points while navigational learning allows for better route distance estimation and location of unseen points

Recall survey vs. route knowledge Orientation-specific vs. orientation-free learning

Studies show evidence that navigational learning is more a collection of multiple views than orientation-free, though may lead to a sort of orientation-free type of knowledge

Sholl & Friedman

Spatial Language Contrasting experience with communication Experience spatial relations continuously,

but language is usually discrete (e.g. near vs. far) Spatial terms function much like other categories

(e.g. fuzzy boundaries, prototypes) Experience multiple spatial relations

simultaneously, but speak of one relation at a time

A frame of reference must be agreed upon in order to communicate spatial relations

Spatial Language Despite the difficulties in communicating spatial

knowledge, ambiguities are generally overcome and information encoded (survey, route knowledge)

However it does seem that spatial language may bias or constrain spatial representation, and may even affect the development of spatial concepts and categories

Even so, the actual link between spatial language and spatial representation is not entirely clear

Impaired sight individuals may have difficulties with a variety of spatial tasks but have intact spatial language

Thinking Spatial cognition also contributes to

logical reasoning, metaphor, and creativity

Transitive reasoning A > B, B > C A ? C

Metaphor The future stretched in front of them My heart is a flame turned upside down Structural alignment of spatial and temporal

concepts Diagrams as aids to understanding

Show conceptual similarity of items, connections amongst various concepts etc.

Creativity E.g. visualization for problem solving

Taking someone else’s point of view?

Imagery Some information in memory is purely

verbal Who wrote the Gettysburg address?

Other memories seem to involve mental images Trying to recall a procedure Making novel comparisons of visual items

What is a mental image? How are mental images represented and

processed? Are mental images like visual images?

Evidence for use of visual imagery

Selective interference Segal & Fusella Imagery interferes with detection of

stimuli (sensitivity decreased) Auditory imagery interfered with

auditory detection, visual imagery with visual stimuli

Manipulation of images Mental rotation studies

Evidence for use of visual imagery

Kosslyn Learn a map Mentally travel from

one point to another Measure time to

make this mental trip

Results Time to make trip

increases with distance

Times increase with imagined size of the map.

Evidence for use of visual imagery Moyer 1973

Subjects were given the names of two common animals and asked to judge which was larger

Which is larger, a moose or a roach?

Wolf or Lion? The time delays as a

function of size difference were similar to those usually found for perceptual judgments.

Kosslyn Kosslyn 1975 Scenario I: Imagine an elephant

standing next to a rabbit. Does a rabbit have a beak?

Scenario II: Imagine a fly standing next to a rabbit. Does a rabbit have an eyebrow?

People made faster judgments when relying on a larger mental image (such as the rabbit next to the fly) than when using a smaller mental image (such as the rabbit next to an elephant)

Kosslyn suggested that the size of an image is an important factor in determining how fast we can make judgments about it.

RT

0Elephant Fly

True

False

Inconsistent

Consistent

Paivio's Dual-Coding Theory Information is mentally represented

either in a verbal system (propositional) or a nonverbal (analogical) system (or both). Each system contains different kinds of

information. Each concept is connected to other

related concepts in the same system and the other system.

Activating any one concept also leads to activation of closely related concepts.

Paivio

The hypothesis of multiple codes (verbal and spatial) is based on the demonstration of independence of effects. Pictures of objects Words of objects

Paivio (1975) compared reaction times for consistent and inconsistent visual stimuli

If the stimuli are processed semantically, there should be no difference between consistent and inconsistent presentations.

If stimuli are processed spatially, inconsistent stimuli should require a mental conversion to appropriate size. Which takes time

Consistent

Inconsistent

Results “Which is larger?”

RT

0

Inconsistent Consistent

Paivio Consistent

RABBIT FLY

Paivio

Inconsistent

RABBIT ELEPHANT

Paivio

Congruity Effect only for Pictures (not words) Imagery relies on perceptual detail and semantic does not Such findings as this and picture superiority effect

(pictures are better recognized than words), and that verbal + imagery encoding leads to best recall, suggest a Dual Code Theory

RT

0

Picture Words

Inconsistent

Consistent

Santa 1977 More evidence of

dual coding Ss presented array of

objects or words On test presentation

asked whether the elements were same as studied

E.g. In geometric condition first two would be yes responses

Santa 1977 Results of positive

responses Spatial configuration

is preserved in geometric encoding

Compared to verbal presentation, which was encoding in typical English reading style and benefited from the linear configuration

Representation of images

What is the relationship between imagery and perception?

Can imagining interfere or facilitate detection of stimuli?

Similar processes involved?

Spatial Knowledge Symbolic Distance Effect (Moyer 1973)

Process of imagery = process of perception As perceptual distance increases so does psychological

distance (RT). Items “farther apart” are more quickly distinguished Which is larger?

Rabbit-Elephant Rabbit-Dog

Representation of images Contrary evidence Chambers and

Reisberg Images are (committed

to) a particular interpretation

E.g. The rabbit comes once drawn but was only a duck as imaged

Contrast with perception which requires interpretation, images are already interpretations

Are visual images visual? Plenty of evidence to suggest a spatial

component to visual imagery, but perhaps the visual part is represented propositionally

Kerr Congenitally blind also take longer to

imagine longer map routes like the one in Kosslyn

Are visual images visual? Images are also not as sharp as

real pictures Form a mental image of a tiger

Does it have stripes? How many? It is hard to examine details of

mental images that would require eye movements

Making new pictures Finke, Pinker, Farah Example

Imagine a capital letter H and a triangle

Rotate the H 90 degrees Place the triangle on top of it What is it?

Suggests images can take on new interpretations

Are visual images visual? Facilitation and interference (Farah)

Have people imagine a letter (H or T) Present one of the letters to the screen briefly

(20 ms), or present nothing, followed by mask Asked if they saw a letter

People are more likely to detect the stimulus if it was the same as what they were imaging, suggesting that visual and imaginal representations joined or fed into the same process

Are visual images visual? Evidence from neuroscience Patients with lesions of

visual cortex that lead to perceptual problems also have problems with mental imagery

ERP evidence PET evidence: Visual imagery leads to activation of visual cortex. Auditory imagery does not

In general, results of studies from mental rotation to brain imaging support the idea of both visual and spatial representation of images

Translating Words to images

Franklin and Tversky Create a mental image

based on the description Asked to identify location of

items in that imagined environment

Results are what one might expect given an imagined spatial environment

Up-down, front-back more relevant in navigating real world

Left-right confusion in real world and imagined world

Visual memory

Although our visual memory seems to be excellent, it turns out not to be that great in many respects

In general, our memory for details is lost, much like with other types of memory

Visual memory

Memory for pictures is quite good generally Again, don’t get too detailed Standing

Presented 10000 photos over several days Old-New memory over 80%

Picture superiority effect Better memory for pictures than words