Mental Imagery and Human Memory - Universität...
Transcript of Mental Imagery and Human Memory - Universität...
Mental Imagery andHuman Memory
Seminar Spatial CognitionSummer Term 2003
Thomas Barkowsky
What is mental imagery?
The trunk exampleYou've got a car, numerous pieces of baggage, sports equipment, …
A geographic exampleIn which direction with respect to Bremen are located:
Berlin?London?Vienna? …
Outlook
Mental imagerydefinitionproperties
Human memorylong- term memoryworking memoryshort- term memory
Imagery Modelspsychologyartificial intelligence
Mental imagery – A definition
Mental imagery is "the mental invention or recreation of an experience that in at least some respects resembles the experience of actually perceiving an object or an event, either in conjunction with, or in absence of, direct sensory stimulation" (Finke, 1989)
… invention or recreation …
mental image constructionbased on memorycombined from elements
image inspectionutilization of image constructed
image modificationalternatives in constructed and inspected images
… of an experience …
issue of interest: cognitive principles (not experience!)indirect measurement!are mental images 'real'?cf. imagery debate
… visual perception …
imagery tightly coupled with visionimagery with perception
e.g. combination of seen and mentally constructed elements
imagery without perceptionpure construction from memory
no perception without imageryimagery precondition for vision processes
Mental rotation
[Shepard & Metzler, 1971]
Mental scanning
[Kosslyn et al., 1978]Learn artificial mapScanning the imagined mapReaction time proportional to distance
Conclusion
Spatio-analogical representation in the mindCritics: difference between
experienceexperimental behaviorphysical realization in the mind / brain
The imagery debate
Do 'mental images' have a specific representation format?
dual codingjust propositional coding
e.g. "who 'looks' at the internal picture?"evidence from patients with brain damagesmental imagery quite accepted
Mental images: Characteristics
not epiphenomenalquasi-pictorial mental representation structures
not retrieved in a ready-made formmental constructionsmental storage differs from form in usageuse of pieces of knowledge
Mental images: Characteristics
composition from well-organized pieces of knowledge
hierarchical structure in long-term memoryorganization in mental storage influences image construction time
image construction from both pictorial and propositional information
e.g. image construction from verbal descriptions
Unifying principles
implicit encodingperceptual equivalencespatial equivalencetransformational equivalencestructural equivalence
[Finke, 1989]
Implicit encoding
imagery allows to exhibits information not explicitly stored in memoryretrieve implicit information from memoryadvantageous w.r.t. space requirementcf. diagrammatic reasoning in AI
Perceptual equivalence
"Imagery is functionally equivalent to perception to the extent that similar mechanisms in the visual system are activated when objects or events are imagined as when the same objects or events are actually perceived"Imagery is essential for vision, not the other way around!
Spatial equivalence
"The spatial arrangement of the elements of a mental image correspond to the way objects and their parts are arranged on actual physical surfaces or in an actual physical space"Visual buffer structure both in vision and in imagery
Transformational equivalence
Cf. mental rotation"Imagined transformations and physical transformations exhibit corresponding dynamic characteristics and are governed by the same laws of motion"Cf. mental animationAlso used in diagrammatic reasoning systems
Structural equivalence
Structure of images corresponds to that of real objects
organized, coherent structurereorganizationreinterpretation
e.g. in configuration tasks, anticipation and planning of motion and change
Human memory
long-term memoryworking memoryshort-term memory
Working memory [Baddeley 1986]
information from the sensesinformation from long-term memory(at least) 2 sub-systems
visuo-spatial sketchpadarticulatory loop
Centralexecutive
Articulatoryloop
Visuo- spatialscratchpad
Central executive
mediates betweenworking memory subsystems long-term memory
controls rehearsal processestranslates between modalities
Visuo-spatial sketchpad
short-term memorytransient structure
fading (approx 1.5 sec)rehearsal
subdivision by Logie, 1995visual informationspatial information
Capacity restriction in working memory
7+/-2 items (Miller, 1956)4 items (Cowan, 2001)swapping with long-term memorychunking in working memory
Long-term memory
network structurespreading activationhierarchical organization
conceptual hierarchiesindividual hierarchies
Memory for mental imagery
imagery takes place in working memoryimage construction and modification
information from long-term memoryactivation of information
Memory for mental imagery (contd.)
working memory consists of long-term memory and short-term memoryimage construction in visual buffer = visuo-spatial sketchpadimage inspection in visual bufferimagery processes in central executive
Integration of memory systems
short-termmemory
(not activated)
long-termmemory
(activated)workingmemory
activation
representation/
maintenance
Imagery models
Psychological modelsimplemented (Kosslyn, 1980)conceptual (Kosslyn, 1994)
Artificial intelligence modelssimulation (Funt, 1980)spatial reasoning (Khenkhar, 1991)computational imagery (Glasgow & Papadias, 1992)
Kosslyn, 1980
cathode-ray tube metaphorsurface vs. deep representation
surface representationshort- term / working memoryquasi- pictorial positional structure (raster matrix)limited spatial extendapprox. circular shapedefinite grain and limited resolutionresolution decreases towards peripheryfading of image parts
Kosslyn, 1980 (contd.)
deep representationlong-term memory2 kinds of deep representations
perceptual / literal image representationskeletal encodingsindividual encodingshierarchical organization
discursive description (propositions)qualitative spatial descriptionsinformation about parts, location, and category
Kosslyn, 1980 (contd.)
image constructionPICTURE, FIND, PUT, IMAGE (control)
image inspectionLOOKFOR (control), RESOLUTION, REGENERATE
image modificationZOOM, PAN, ROTATE, SCAN
Kosslyn, 1994
conceptual modelbased on high-level vision processes (neurospsychology)interaction between subsystemsimplementation only of partial aspects
Kosslyn, 1994 (contd.)
Kosslyn, 1994 (contd.)
1. pattern code
2.
2.
3.
3. 4.
image construction
Kosslyn, 1994 (contd.)
1. pattern code
2.
2. 3. 4.
adding additional parts
5.
Kosslyn, 1994 (contd.)
2.
image inspection
1.
1.2.
spatial properties
object properties3.
Funt, 1980 (WHISPER)Reasoning about mechanicsPrediction of motionPositional representation of initial stateRepresentation of intermediate statesComponents
diagramhigh-level reasoner'retina'
Funt, 1980 (contd.)high-level reasoner
procedural propositional problem solverqualitative physical knowledge
Funt, 1980 (contd.)
'retina'positional diagrammatic structurecircular structure of concentric ringsparallel processorsneighborhood structuresupervising processorvariable position
Funt, 1980 (contd.)
retina: percptual primitivesfocus center of entitycheck for symmetrycheck for congruencescaling, rotationconnections, collisionsboundary features (convexity, slope)identify neighboring entities
simulation of tentative movement of objects
Depic-2D (Khenkhar, 1991)
hybrid system: propositional and pictorial reasoningregular rectangular cell structuredepiction of objects in cells
Depic-2D (Khenkhar, 1991)
depiction of objects in cellsuse of neighborhood structure
spreading activatione.g. buffering
Depic-2D (Khenkhar, 1991)
Computational imagery (Glasgow & Papadias, 1992)
diagrammatic reasoning architecture for technical applications
chemistrygeography
3 subsystems
visualrepresentation
interpret
store
spatialrepresentation
deeprepresentation
perceive reconstruct
Computational imagery (contd.)
surface representation subdividedspatial representation
'where' aspects: location w.r.t. other objectsvisual representation
'what' aspects: shape, color, texture
visualrepresentation
interpret
store
spatialrepresentation
deeprepresentation
perceive reconstruct
Computational imagery (contd.)
deep representationlong-term storage (before and after processing)hierarchical organization
Computational imagery (contd.)visual representation
positional structure: occupancy arraysshape, distance, (relative) distancestexture, color, surface orientation3D, viewer independentstandard computer graphics operations
Computational imagery (contd.)
spatial representationqualitative relationships between objectsrelational structure: symbolic arrays
Computational imagery (contd.)
topology and (coarse) orientationno exact shapes and sizeshierarchiesprocesses for
information transferplacing, detecting, moving, deletingchecking for adjacencypattern recognition by attention control
Thanks!
questions?comments?