Embodied Cognition as a basis for researching and designing

interaction

Lecture 3Sensorimotor Coupling

And wrap-up..

Three variations of Embodied Cognition

Cognition is Distributed Computation & Representation

Socially Situated! Sensorimotor coupling!

Next lecture (in two weeks from now)

• Socially Situated Practice studies talk a lot about social interaction, communication, collaboration, etc…

• But the body here is used really secondary: it is used as a means to communicate between people.

• What about the embodiment of cognition?• What about bodies ‘as such’• About their activity• About movement• The senses• Skilled actions?• Habits?• Coupling?• Temporal dynamics?

The body

• The brain to a great extent regulates the internal integrity of the body

• Signals to and from the body form an extra layer on that internal organisation

• The flow of information from the senses to the motor output is loaded with bodily value: Damasio’s Somatic Markers (Damasio, 1996)

• http://www.iee.unibe.ch/unibe/philnat/biology/zoologie/content/e7493/e7854/e8920/e8926/DamasioPhilTransRSocB96.pdf

Action-perception coupling

• A continuous flux going from the world, through the sensory system, into the body and out into the world again, in the form of action.

• No starting point• Over time, couplings emerge: stable patterns of

behavior through which the body ‘gets a grip’ on the environment.

• For example: throwing darts, shooting hoops, becoming better at it. Or even better: juggling, riding your bike: suddenly the coupling falls into place.

Behavior-based robots

• qwef

Modeling robot-environment systems

‘Braitenberg vehicles’

The mechanics and physical form of the body as a an element in the

system dynamics

Use ‘traces’ left in physical space as elements in the system dynamics

Gibson’s direct perception

• Gibson, 1979

• Many things we can detect in the environment as ‘invariants’ and for detecting them we do not need any internal mechanism (no internal ‘thought process’). The invariants are simply ‘picked up’ as we go along, they are directly perceived.

• There is much more information already to be ‘picked up’ from the way light reflects from objects and surfaces than we think. We need no complex brain processing to explain how we perceive our surrounding world.

• For example: distance cues: occlusion, texture,

Gibson’s notion of Affordances

• Perception of objects is first and foremost a perception of ‘how I would act on it on the next instant’

• To recognize and object is to see how it is to be handled (to already ‘start handling it’, even)

• An object is first and for most a ‘to ….(action)’: A to-sit-on, a-to-jump-over, a ‘to-pick-up-with-one-hand’, and so on.

• To determine it’s ‘objective’ properties (e.g. it’s objective size, colour, name, and so on, comes later. It is a post-hoc reasoning that is not part of object perception as such.

• It is not an ‘objective property of the object’, let alone a ‘designed information cue’: it is a property of the relation between you, ‘caught in the act’ and the object, situated in its context, at that moment.

Skill

Kite-surf ‘know-how’

The theory

Scaffolded learning

MasterySee: Dreyfus skill paper!

Practice-as-skill: craftmanship

Practice-as-skill: craftmanship

(Not discussed in detail in lecture)

• Know-how• Reflection-in-action• Craftmanship (Sennett)• Heidegger’s Zuhandenheit

Enactment

• Enacting a world• In working to maintain its own integrity (stay

alive) the organism at the same time also brings forth a world of meaningful objects. The world we perceive is coupled to our process of self-maintainance: we see what is meaningful for us (we perceive a world of affordances)

A sailor enacting ‘the sailing environment’ in action

Design for sensorimotor couplings

• What are the existing couplings, as part of people’s skilled practices?

• Describe the practice in terms of how perception and action get coupled over time into a practical skill, in action, in the context, etc…

• How could (interactive) technology mediate these couplings?– Boost – Temper– Trigger– Break– Transform/morph– Create new (but how does that affect the existing practice)

“Biological re-design”: Moken “Sea Gypsy” children

Tangibles: what do they do for sensorimotor coupling?

Bach Y Rita: Sensory Substitution

Seeingwithsound.com

How about this one: does it elicit a sensorimotor loop? Or do you need to

consciously reason about the meaning of the soundscapes in order to use it?

Nike+ Fuelband: Does this create or enhance a sensorimotor

coupling?

MYO, thalmic labs gesture recognition arm band: does this integrate with the

sensorimotor skill of cooking?

Google glass: does it change our perception-action coupling, or does it take us out of these

couplings?

Spaceglasses

Integrating the perspectives

• Socio-sensorimotor coupling• Socially situated ‘in action’• AND• Sensorimotor coupling ‘in action’

• People make sense of what they are doing in action, as a sensorimotor coupling, and in the social situation, as something collaboratively achieved, between people.

movie

Sensemaking in action

What about “distributed representations”

• … text, pictures, graphs, maps, signs, symbols, speech, plans, procedures, methods, rules, prescriptions, recipes,

• External representations are used as part of the process of sensorimotor coupling

• They bias (‘reorient’) us

• They are often traces from people’s activities

• These traces are used by people to guide their actions

• This can be enhanced by deictic pointing (keeping a shared attention by looking at, or dealing with, a shared object)

What about symbols, language, plans, procedures, scripts, pictures: representations?