VT

2

The Ecological Approach to Mobile Communication

Barry SmithInstitute for Formal Ontology and Medical

Information ScienceUniversity of Leipzig

http://ifomis.de

3

Frank Zappa

everything in the world is resonating to the One Big Note

Murray Schaefer Tuning of the World

Tony Schwarz Resonance theory of communication

L (advertising follower of MacLuhan)

4

Formal OntologyInstitute for Formal Ontology and Medical Information Science

5

Roman Ingarden

Material Ontology

Realism

Theory of Causality

Theory of Relatively Isolated Systems

Modularity

6

Adolf Reinach

Ontology of Social Reality

7

Maurice Merleau-Ponty & J. J. Gibson

Ontology of Cognitive Prosthetics

8

9

Part One: Ontology of Cognitive Prosthetics

Part Two: Situated Computing and the Intentionality of E. Coli

Part Three: How is Unified Cognition Possible?

10

Technologies of Mobile Communication

Global Positioning Systems (GPS)

11

Digital cameras

Technologies of Mobile Communication

12

Digital video cameras

Technologies of Mobile Communication

13

chemical

biological

… wearable computers

… brain-wave sensors to catch cheaters

Technologies of Mobile Communication

Microsensors

14

PalmPilot context aware

Display the wiring/plumbing behind this wall

Display seismographic features of a terrain a geologist is viewing

Display vital signs of a patient a doctor is examining

15

European Media Lab, Heidelberg

Tourism information services

Intelligent, speaking camera plus map display

Display all non-smoking restaurants within walking distance of the castle

Read out a history of the building my camera is pointing to

16

Intelligent mobile phones

Inform a person walking past a bar of his buddies in the bar

17

How to Understand Mobile Information

Systems?

18

Traditional Syntactic/Semantic Approach to Information

Systems011011101010001000100010010010010010010001001111001001011011110110111011

derived intentionality

19

Husserl’s Methodological Solipsism

noesisnoesisnoesisnoesisnoesisnoesisnoesisnoesis

noema noema noema noema noema noema noema noema

20

Fodor’s Methodological Solipsism

011011101010001000100010010010010010010001001111001001011011110110111011

21

Knowledge =

what can be transmitted down a wire

(effectively reducible to patterns of 1s and 0s)

22

Humans, Machines, and the Structure of Knowledge

Harry M. CollinsSEHR, 4: 2 (1995)

23

Knowledge-down-a-wireImagine a 5-stone weakling who has his brain loaded with the knowledge of a champion tennis player.

He goes to serve in his first match

-- Wham! –

his arm falls off.

He just doesn't have the bone structure or muscular development to serve that hard.

24

Types of knowledge/ability/skill

1. those that can be transferred simply by passing signals from one brain/computer to another.

2. those that can’t:

25

Sometimes it is the body which knows (the hardware)

26

I know where the book is= I know how to find it

I know what the square root of 2489 is= I know how to calculate it

27

Not all calculations are done inside the head

Not all thinking is done inside the head

28

A. Clark, Being There

we rely on

external scaffolding = maps, models, tools, language, culture

we act so as to simplify cognitive tasks by "leaning on" the structures in our environment.

29

Merlin Donald

Origins of the modern mind: Three stages in the evolution of culture and cognitionCambridge, MA: Harvard University Press, 1991

30

Merlin Donald

radical transition in the emergence of modern human culture when humans began to construct elaborate symbolic systems ranging from cuneiforms, hieroglyphics, and ideograms to alphabetic languages and mathematics

31

Merlin Donald

from this point human biological memory becomes an inadequate vehicle for storing and processing our collective knowledge. from this point the modern mind is a hybrid structure built from vestiges of earlier biological stages together with new

external symbolic memory devices

32

Types of knowledge/ability/skill

1. those that can be transferred simply by passing signals from one brain/computer to another.

2. those that can’t: -- here the "hardware" is important;abilities/skills contained (a) in the body(b) in the natural world(c) in the marked-up world

33

From

The Methodological Solipsist Approach to Information Processing (Fodor, Husserl)

ToThe Ecological Approach to Information

Processing (Gibson, Merleau-Ponty)

34

Ecology

The digital streams connecting one mobile phone to another are not so important

What is important is the environment in which both are embedded

35

Deliberative intellectual reasoning is not so important

Hayek: General concepts are tools for being lazy

36

Fodorian Psychology

To understand human cognition we should study the mind/brain in abstraction from its real-world environment

(as if it were a hermetically sealed Cartesian ego)

37

Gibsonian Ecological Psychology

To understand human cognition we should study the moving, acting human person as it exists in its real-world environment

and taking account how it has evolved into this real-world environment

We are like tuning forks – tuned to the environment which surrounds us

38

Fodorian View of Information Systems

To understand information systems we should study their manipulation of syntactic strings

39

Gibsonian Ecological View of Information Systems

To understand information systems we should study the hardware as it exists embedded in its real-world environment

and taking account the environment for which it was designed and built

Information systems are like tuning forks – they resonate in tune to their surrounding environments

40

Merleau-Ponty

what we see, what we experience, what the world in which we find ourselves is like

depend upon our purposes of the moment

41

The Basic Layer of Experience

When seeing an event, reading a page

we find, as a basic layer of experience, a whole already pregnant with an irreducible meaning: not sensations with gaps between them, into which memories may be supposed to slip, but the features, the layout of a landscape or a word, in spontaneous accord with the intentions of the moment ‘

(Phenomenology of Perception, 21f.)

42

Special Role of the BodyIf my arm is resting on the table I should never think of saying that it is beside the ash-tray in the way in which the ash-tray is beside the telephone. The outline of my body is a frontier which ordinary spatial relations do not cross. This is because its parts are inter-related in a peculiar way: they are not spread out side by side, but enveloped in each other. (PoP, p. 98)

43

Affordances

The bench, scissors, pieces of leather offer themselves to the subject as poles of action … they delimit a certain situation … which calls for a certain mode of resolution, a certain kind of work. The body is no more than an element in the system of the subject and his world, and the task to be performed elicits the necessary movements from him by a sort of remote attraction (PoP, p. 106)

44

Gibson: Environment as Array of Affordances

“The affordances of the environment are what it offers the animal, what it provides or furnishes, either for good or evil.”

James J. Gibson, The Ecological Approach to Visual Perception

45

Gibson: We are tuned to our environment

the phenomenal forces at work in my visual field elicit from me, without any calculation on my part, the motor reactions which establish the most effective balance between them, the conventions of our social group, or our set of listeners, immediately elicit from us the words, attitudes and tone which are fitting. (PoP, p. 106)

46

The Ecological Approach to Human Communication

When I motion my friend to come nearer, my intention is not a thought prepared within me and I do not perceive the signal in my body. I beckon across the world, I beckon over there, where my friend is; the distance between us, his consent or refusal are immediately read in my gesture; there is not a perception followed by a movement, for both form a system which varies with the whole. (PoP, p. 111)

47

Embrangled Styles

When I chat with a friend whom I know well, each of his remarks and each of mine contains, in addition to the meaning it carries for everybody else, a host of references to the main dimensions of his character and mine, without our needing to recall previous conversations with each other. These acquired worlds … (PoP, p. 130)

48

Geometry in the Legs

the word ‘sediment’ should not lead us astray: … acquired knowledge is not an inert mass in the depths of our consciousness. My flat is, for me, not a set of closely associated images. It remains a familiar domain round about me only as long as I still have ‘in my hands’ or ‘in my legs’ the main distances and directions involved, and as long as from my body intentional threads run out towards it. (PoP, p. 130)

49

the unity of the body is not simple coordination

I desire a certain result and the relevant tasks are spontaneously distributed amongst the appropriate segments ..I can continue leaning back in my chair provided that I stretch my arm forward …. All these movements are available to us in virtue of their common meaning. (PoP, 149)

50

Language

The speaking subject does not think of the sense of what he is saying, nor does he visualize the words which he is using. To know a word or a language is … not to be able to bring into play any pre-established nervous network. But neither is it to retain some ‘pure recollection’ of the word … (PoP, p. 180)

51

Speaking as: Being Faithful to One’s Self

I do not need to visualize the word in order to know and pronounce it. It is enough that I possess its articulatory and acoustic style as one of the modulations, one of the possible uses of my body. (PoP, p. 180)

52

Language

one particular cultural object [plays] a crucial role in the perception of other people: language. In the experience of dialogue, there is constituted between the other person and myself a common ground; my thought and his are interwoven into a single fabric, my words and those of my interlocutor are called forth by the state of the discussion, and they are inserted into a shared operation of which neither of us is the creator. (PoP, p. 354)

53

A miniature civil society

We have a dual being, where the other is for me no longer a mere bit of behaviour in my transcendental field, nor I in his; we are collaborators for each other in consummate reciprocity. Our perspectives merge into each other and we co-exist through a common world. (PoP, p. 354)

54

Spontaneous unification

Consider how the human mind copes with complex phenomena in the social realm, e.g. a promise, which involves (REINACH):

experiences (speaking, perceiving), intentions, language, action, deontic powers, background habits, mental competences, records and representations

55

Prosthetic Cognition

When a typist performs the necessary movements on the typewriter, these movements are governed by an intention, but the intention does not posit the keys as objective locations. It is literally true that the subject who learns to type incorporates the key-bank space incorporates the key-bank space into his bodily space.into his bodily space. (PoP, p. 145)

56

The Organist

Between the musical essence of the piece as it is shown in the score and the notes which actually sound round the organ, so direct a relation is established that the organist’s body and his instrument are merely the medium of this relationship. Henceforth the music exists by itself and through it all the rest exists. There is here no place for any ‘memory’ of the position of the stops, and it is not in objective space that the organist in fact is playing. (PoP, p. 145)

57

The Organist

In reality his movements during rehearsal are consecratory gestures: they draw affective vectors, discover emotional sources, and create a space of expressiveness as the movements of the augur delimit the templum.

58

TEMPLUM

from the Greek „terminus“: to cut off

templum = any place which was circumscribed and separated by the augurs from the rest of the land by a certain solemn formula

59

60

Part One: Ontology of Cognitive Prosthetics

Part Two: Situated Computing and the Intentionality of E. Coli

Part Three: How is Unified Cognition Possible?

61

Computerized Agents

computer systems situated in an environment capable of flexible, autonomous action in

that environmentinteracting with other agents, including:

communicating, negotiating, coordinating actions

often within some organizational context

62

Rodney Brooks

63

Orthodox methodology

described by Brooks

as the SMPA view

64

SMPASense Model Plan Act

the agent first senses its environment through sensors

then uses this data to build a model of the world

then produces a plan to achieve goals

then acts on this plan

65

SMPA

belongs to the same methodological universe as Fodorian cognitive science (solipsism)

If we want to build an intelligent agent, there need to be representations (‘models’) inside the agent of the domain within which the agent acts

The agent’s reasoning processes act not on the real-world environment but on these models

66

Brooks’ Engineering Approach

takes its inspiration from evolutionary biology

lends very little weight to the role of representations or models

AI should use the world in all its complexity in producing systems that react directly to the world

67

The starting-point for our understanding of intentionality

should be the insect’s relations to its surrounding physical environment

68

69

Intentionality tactile and chemical

70

The movement of E. coli as a biased random walk

In the absence of a stimulus, E. coli simply wanders around, smoothly swimming by rotating its flagella counterclockwise. These runs are terminated by chaotic events, called tumbles, when flagella rotate clockwise. Following a tumble, the cell runs again, picking a new direction, more or less at random. When the cells swim in a spatial gradient of a chemical attractant, runs that happen to carry it up the gradient are extended, whereas those that happen to carry it down the gradient are not. Thus, the cell drifts in a favourable direction …

71

The life of E. coli

is a life of falling

down

sugar

wells

72

the bacterium is a single cell,

Thus it does not have a multicelled nervous system

But it has receptor molecules acting as sensors, it has a signal transduction system, and a highly complex machinery of movable flagella.

Different receptors react to different stimuli, including single oxygen molecules as well as bigger carbohydrate molecules.

See Bruce Alberts et al.: The molecular biology of the cell

73

E. coli bacteria

react to differences in concentrations of sugar molecules with a behavior shift (as a dog reacts to a smelt trace of another animal)

74

Attribution of intentionality

does not depend upon the existence of a nervous system

we can ascribe simple biological intentionality to single, movable cells;

intentionality is dependent only upon the existence of sensors, information mediation (automatic interpretation, if you like) and motor responses resulting in adaptable behavior.

75

Frederik Stjernfelt, “Biosemiotics and Formal Ontology", Semiotica 127 - 1/4 1999, 537-66

76

E coli bacteria

are attracted by peaks of sugar density

– but they can be fooled

77

Brooks’ Engineering Approach

An intelligent system embodies a number of distinct layers of activity (compare: sub-personal layers of human cognition)

These layers operate independently and connect directly to the environment outside the system

Each layer operates as a complete system that copes in real time with a changing environment

Layers evolve through interaction with the environment (artificial insects/vehicles/teenagers SMS-module…)

78

Brooks: An intelligent system

must be situated

it is situatedness which gives the processes within each layer meaning

because

the world serves to unify the different layers together and to make them compatible/mutually adjusting

79

Organisms, especially humans, fix their beliefs not only in their heads but in their worlds, as they attune themselves differently to different parts of the world as a result of their experience. And they pull the same trick with their memories,not only by rearranging their parsing of the world (their understanding of what they see), but by marking it. They place traces out there [and this] changes what they will be confronted with the next time it comes around. Thus they don't have to carry their memories with them.

“Intelligence without Representation”

80

J. J. Gibson The Ecological Approach to Visual Perception

we are like (multi-layered) tuning forks – tuned to the environment which surrounds us

(we have evolved in such a way as to be attuned to our environment;

in part because we ourselves have created it via what Lewontin calls ‘ecosystem engineering’)

81

Organisms are tuning forks

They have evolved to resonate automatically and directly to those quality regions in their niche which are relevant for survival

– perception is a form of automatic resonation– when the insect stumbles through uneven

terrain the insect’s motor system is resonating to the reality beyond

82

Merlin Donald

the modern mind is a hybrid structure built from vestiges of earlier biological stages together with new

external symbolic memory devices together with cognitive prosthetic devices

(keyboards, touch-screens, mobile phones, … )

83

A New Biological Theory of Intentionality

– cognitive beings like ourselves resonate to speech acts and to linguistic records

– cognitive beings like ourselves resonate deontically

– mathematicians resonate to the structures of mathematical reality

84

Gibson’s Ecological Approach

To understand cognition we should study the moving, acting organism as it exists in its real-world environment

and for human organisms this is a social environment which includes records and traces of prior actions in the form of communication systems (languages), storage systems (libraries), transport systems (roads), legal systems

85

Humans

resonate on many levels to patterns

and to patterns of those patterns

Humans differ from animals in that they can train themselves to resonate to new sorts of patterns

(… nursing expertise …)

86

Gibson: Environment as Array of Affordances

“The affordances of the environment are what it offers the animal, what it provides or furnishes, either for good or evil.”

The environment of a commercial organism includes affordances such as prices.

87

The world of affordances

includes not merely walls, doors, furniture, temperature gradients, patterns of movement of air and water

but also: traffic signs, instructions posted on notice boards or displayed on computer screens – whole strata of what is marked by signs

88

Roger G. Barker’s Eco-Behavioral Science

Gibson: Ecological Psychology of Perception

Barker: Ecological Psychology of Social Action

P. Schoggen, Behavior Settings: A Review and Extension of Roger G. Barker’s Ecological

Psychology. Stanford, CA: Stanford University Press, 1989.

89

Roger Barker: Niche as Behavioral SettingNiches are recurrent settings which serve as the environments for our everyday activities:

a newspaper kiosk in the morning rush-hour,

your table in the cafeteria,

the 5pm train to Long Island.

90

Behavior Settings

Each behavior setting is associated with certain standing patterns of behavior.

We are tuned to an environment of behavior settings

91

The Systematic Mutual Fittingness of Behaviour and Setting

The behaviour and the physical objects … are intertwined in such a way as to form a pattern that is by no means random: there is a relation of harmonious fit between the standard patterns of behaviour occurring within the unit and the pattern of its physical components.

92

Settings, for Barker,

are natural units in no way imposed by an investigator.

To laymen they are as objective as rivers and forests

— they are parts of the objective environment that are experienced as directly as rain and sandy beaches are experienced. (Barker 1968, p. 11)

93

Barker on Unity of Social Reality

“The conceptual incommensurability of phenomena which is such an obstacle to the unification of the sciences does not appear to trouble nature’s units.

Within the larger units, things and events from conceptually more and more alien sciences are incorporated and regulated.”

94

Barker on Unity of Social Reality

“As far as our behaviour is concerned, … even the most radical diversity of kinds and categories need not prevent integration”

Because we have been tuned both phylogenetically and ontogenetically to resonate to environments like this

95

Lacan

The style is the man

to whom one is speaking

96

The life of a human being

is a life of falling

down

style

wells

97

98

Part One: Ontology of Cognitive Prosthetics

Part Two: Situated Computing and the Intentionality of E. Coli

Part Three: How is Unified Cognition Possible?

99

How does a Global Positioning System work?

Your GPS device knows its location because at any given moment it is receiving quite specific signals from satellites

and because these signals contain information to which it is sensitive in virtue of its precise location in any given moment.

100

Organisms are tuning forks

Homing pigeons are sensitive to highly nuanced features of the earth’s magnetic field. Human beings are sensitive to the information contained in other human beings’ faces. Human beings who can drive are sensitive to traffic signs, to small variations in movement of the vehicles around them.Human beings who can read are sensitive to the astonishingly variable types of information contained in printed texts.

101

Recall how the human mind

copes with complex phenomena in the social realm, involving

utterances,intentions, action, deontic powers, background habits, style, mental competences of the speaker, records and representations

102

How do we, directly and spontaneously, bring about the

integration of such transcategorial phenomena?

ANSWER: We do not

It is the world which is responsible for unification

103

A theory of intentionality

must be a (biologically based) theory of the sorts of environments, on different levels of granularity, into which human beings have evolved and are still prosthetically evolvingour patterns of behavior and cognition on different levels are unified together not via some central monad but by the world itself

(our environments fit together physically)

104

The riddle of representation

two humans, a monkey, and a robot are looking at a piece of cheese;

what is common to the representational processes in their visual systems?

105

Answer:

The cheese, of course

106

END