Post on 29-Jun-2015
description
Embodied Cognition as a basis for researching and designing
interaction
Lecture 3Sensorimotor Coupling
And wrap-up..
Three variations of Embodied Cognition
Cognition is Distributed Computation & Representation
Socially Situated! Sensorimotor coupling!
Next lecture (in two weeks from now)
• Socially Situated Practice studies talk a lot about social interaction, communication, collaboration, etc…
• But the body here is used really secondary: it is used as a means to communicate between people.
• What about the embodiment of cognition?• What about bodies ‘as such’• About their activity• About movement• The senses• Skilled actions?• Habits?• Coupling?• Temporal dynamics?
The body
• The brain to a great extent regulates the internal integrity of the body
• Signals to and from the body form an extra layer on that internal organisation
• The flow of information from the senses to the motor output is loaded with bodily value: Damasio’s Somatic Markers (Damasio, 1996)
• http://www.iee.unibe.ch/unibe/philnat/biology/zoologie/content/e7493/e7854/e8920/e8926/DamasioPhilTransRSocB96.pdf
Action-perception coupling
• A continuous flux going from the world, through the sensory system, into the body and out into the world again, in the form of action.
• No starting point• Over time, couplings emerge: stable patterns of
behavior through which the body ‘gets a grip’ on the environment.
• For example: throwing darts, shooting hoops, becoming better at it. Or even better: juggling, riding your bike: suddenly the coupling falls into place.
Behavior-based robots
• qwef
Modeling robot-environment systems
‘Braitenberg vehicles’
The mechanics and physical form of the body as a an element in the
system dynamics
Use ‘traces’ left in physical space as elements in the system dynamics
Gibson’s direct perception
• Gibson, 1979
• Many things we can detect in the environment as ‘invariants’ and for detecting them we do not need any internal mechanism (no internal ‘thought process’). The invariants are simply ‘picked up’ as we go along, they are directly perceived.
• There is much more information already to be ‘picked up’ from the way light reflects from objects and surfaces than we think. We need no complex brain processing to explain how we perceive our surrounding world.
• For example: distance cues: occlusion, texture,
Gibson’s notion of Affordances
• Perception of objects is first and foremost a perception of ‘how I would act on it on the next instant’
• To recognize and object is to see how it is to be handled (to already ‘start handling it’, even)
• An object is first and for most a ‘to ….(action)’: A to-sit-on, a-to-jump-over, a ‘to-pick-up-with-one-hand’, and so on.
• To determine it’s ‘objective’ properties (e.g. it’s objective size, colour, name, and so on, comes later. It is a post-hoc reasoning that is not part of object perception as such.
• It is not an ‘objective property of the object’, let alone a ‘designed information cue’: it is a property of the relation between you, ‘caught in the act’ and the object, situated in its context, at that moment.
Skill
Kite-surf ‘know-how’
The theory
Scaffolded learning
MasterySee: Dreyfus skill paper!
Practice-as-skill: craftmanship
Practice-as-skill: craftmanship
(Not discussed in detail in lecture)
• Know-how• Reflection-in-action• Craftmanship (Sennett)• Heidegger’s Zuhandenheit
Enactment
• Enacting a world• In working to maintain its own integrity (stay
alive) the organism at the same time also brings forth a world of meaningful objects. The world we perceive is coupled to our process of self-maintainance: we see what is meaningful for us (we perceive a world of affordances)
A sailor enacting ‘the sailing environment’ in action
Design for sensorimotor couplings
• What are the existing couplings, as part of people’s skilled practices?
• Describe the practice in terms of how perception and action get coupled over time into a practical skill, in action, in the context, etc…
• How could (interactive) technology mediate these couplings?– Boost – Temper– Trigger– Break– Transform/morph– Create new (but how does that affect the existing practice)
“Biological re-design”: Moken “Sea Gypsy” children
Tangibles: what do they do for sensorimotor coupling?
Bach Y Rita: Sensory Substitution
Seeingwithsound.com
How about this one: does it elicit a sensorimotor loop? Or do you need to
consciously reason about the meaning of the soundscapes in order to use it?
Nike+ Fuelband: Does this create or enhance a sensorimotor
coupling?
MYO, thalmic labs gesture recognition arm band: does this integrate with the
sensorimotor skill of cooking?
Google glass: does it change our perception-action coupling, or does it take us out of these
couplings?
Spaceglasses
Integrating the perspectives
• Socio-sensorimotor coupling• Socially situated ‘in action’• AND• Sensorimotor coupling ‘in action’
• People make sense of what they are doing in action, as a sensorimotor coupling, and in the social situation, as something collaboratively achieved, between people.
movie
Sensemaking in action
What about “distributed representations”
• … text, pictures, graphs, maps, signs, symbols, speech, plans, procedures, methods, rules, prescriptions, recipes,
• External representations are used as part of the process of sensorimotor coupling
• They bias (‘reorient’) us
• They are often traces from people’s activities
• These traces are used by people to guide their actions
• This can be enhanced by deictic pointing (keeping a shared attention by looking at, or dealing with, a shared object)
What about symbols, language, plans, procedures, scripts, pictures: representations?