Research supported by the South African National Research Foundation

What must the world be like to have information about it?

IS4IS 2015 Triangular RelationJohn Collier

University of KwaZulu-Natal

Information about an object

For a representer to have information about some object is to represent either implicitly or explicitly features of the object such that it is possible to – Anticipate behaviour of the object– Anticipate interactions with the object• Its effects on the representer• The effects of the representer on the object

(I use “object” here in much the sense of C.S. Peirce, presuming that the information the representer has of the object is presented as a sign. This is not required for my discussion, but something like it is.)

Anticipation is computationalI assume that anticipation is computational. This implies:• Representation is computational.• Reasoning is computational.• Both are causal, embedded in computational processes that

are realized in the representer.In fact I think that all causal connection is computational (transfer of information, Collier 1999), but this need not be assumed here. The computational nature of representation and reasoning is a corollary as a special case that I will make more explicit in the following. The special case can be developed on its own.

Representation may be digital or analog

• Explicit representation is digital (Dretske 1981). This means that the representation and contains no more than and no less than its object, and its causal (computational) basis has exactly its logical properties.

• Implicit (tacit) representation is typically analog and contains more than what its object. Its causal basis has more connections than what its object implies.

• I see no reason why a representation may not have both an explicit and an implicit part, the latter leading to connotations and other possible connections to other representations. Representers can typically discriminate the two aspects.

The problem

What must the world be like such that representations of it are possible and that these representations contain information about the world?This problem has three aspects:• Representers must be embedded in the world to allow

interaction with it. Representers are part of the world (naturalism, embedding).

• The world must be such that it can be represented with information.

• There must be an information channel or channels from the world to representers (Every Thing Must Go).

Embedding

• Embedding implies that representers follow basic laws of the sort of thing they are, physical, chemical, biological, cognitive, social, though representers may also follow more restrictive constraints.

• In particular, interaction requires that representers follow the 2nd Law of Thermodynamics, and that their anticipations are in the same direction as entropy increase.

• This implies that entropy places a limit on the amount of information a representer can have about the world, and that they are not at equilibrium.

Distinctions in the world

• The world must contain distinctions for representers to have information about it.

• These distinctions must be accessible.• To be accessible they must not be at equilibrium (or

accessibility violates 2nd Law).• So the world, or any system a representer can have

information about must be nonequilibrium• This is possible if it is 1) structured, or 2)

unstructured but not at equilibrium, or 3) some combination of 1 and 2.

Information channels

I use the notion of information channel in Barwise and Seligman (1997). This requires that:1. The possible states of the object and of a representation fall

into classes (types) with particular states (tokens) occupying these classes.

2. That there is a channel (possibly distributed) between the object classifications and representer classifications.

3. Channels are composed of infomorphisms that guarantee that a state in a representation has the information that the object is in a certain state (falls under a specific class).

4. Channels are causal connections, but need not be direct. They could even involve wide spatial and/or temporal separation.

Types and tokens are duals• The logic of types and tokens are duals, so they may play each

other’s roles.• For example, a red receptor in the eye classifies red light in each

instance it is stimulated, but it also creates an informorphism between an instance of red light hitting it (token) and a neural response (token). Information is carried by tokens, never classes.

• The distributed character of channels allows some tokens to act as constraints (classifiers) on other infomorphisms, so that information already received can act as a constraint on other information. (Thus fulfilling Shannon’s claim the information and constraints are basically the same thing.)

• Thus already received information can serve as part of a channel for new information.

Prior attempts at semantic information

• Carnap and Bar Hillel based on ideal language (1964), an uncertainty account

• Dretske account of perceptual information (1981), a causal account• Barwise and Perry using situation semantics (1983), a transmission

accountEach account has some elements of the others, but focusses on the aspect I mention. I think there is something right about all three of these, but they also are either incomplete or have flaws. I will look at each in turn. There are other accounts in play, but these epitomise the main trends, uncertainty, causal, and transmission that I think need to be combined. They are also widely known and easily accessible. (Collier 1990)

Carnap and Bar Hillel 1

• A top-down formal approach: formulate ideal language for representations and define information with respect to it in terms of state descriptions in the ideal language ruled out.

• A state description is a conjunction of atomic statements assigning each primitive monadic predicate or its negation (but never both) to each individual constant of the language.

• Evidence, in the form of observation statements, contains information in virtue of the class of state descriptions the evidence rules out.

• Information content, then, is inversely related to probability, as intuition would suggest.

Carnap and Bar Hillel 2Two measures were suggested, each of which satisfies certain intuitions:1 – p(i)Fails to be additive (independent information is additive) and fails some natural assumptions about conditional probability.(1/(1-cont(i))) = p(i), in bitsSatisfies additivity and conditional probability, but violates a common intuition that contrary (“negative”) information cannot lead to less than the absolute information of i. cont satisfies this intuition. I don’t share this intuition because it requires work to recover from an error.

Problems with Carnap and Bar Hillel 1

• Because they use methods restricted to first order languages with identity, Carnap and Bar-Hillel cannot deal with complex scientific predicates like mass, temperature and energy.

• This could perhaps be corrected with a more sophisticated inductive logic using a more holistic view of theories. Any move towards a more holistic view, however, undermines the use of basic sentences to determine a priori probabilities.

• The probabilities of individual sentences of a theory become dependent on the overall a priori probability of the theory. At the least, this will greatly complicate calculations of information content.

Problems with Carnap and Bar Hillel 2

• Pure observation statements dubious: Observation statements, if they are not purely demonstrative, assign predicates to evidence (Quine). This amounts to a classification of the evidence. Which is a theory (Barwise and Seligman). But this must come from somewhere else.

• So the top-down formal approach will not work on its own, but it does give an upper limit on a satisfactory theory of information in representations. Bottom-up aspects are also needed to introduce further constraints.

Dretske approach• Uses states of affairs directly, so information in a representation is

determined by its probability:

• A representation is intentional if it is digital in the sense given above. This is top-down approach for intentionality.

• "The ultimate source of intentionality inherent in the transmission and receipt of information is, of course, the nomic regularities on which the transmission of information depends.“ This is bottom-up for intentionaity.

• Dretske also goes further to a bottom-up approach. The idea is that concrete situations are sources of information that can be transmitted through channels that don't themselves generate new information (Dretske, 1981: 115)

Problems with Dretske

• We don’t typically think of natural laws (nomic regularities) as intentional, though they are modal (a start), so “ultimate source of intentionality” needs to be cashed out. (It is at least incomplete as it stands.)

• However this is an important clue. We need an isomorphism between the laws governing a representation corresponding to their logic and those governing its object (see above). This will be grounded in causation.

• Dretske requires certainty (100% probability) which many critics find too strong.

Barwise and Perry• The goal of this approach is to connect meaningful representations to concrete

situations, using situations (more similar to Peircean objects than “external” objects) rather than states of affairs. Situations are, roughly, partial states of affairs.

• The interpretation of a representation is the information it conveys (transmits).• This connection is based on nomic regularities, which Barwise and Perry call

“constraints“. This idea was later cashed out in the notion of an information channel in Barwise and Seligman, Information Flow (1997). This notion will play an important role later in this talk.

• The information indicated by a situation is relative to the causal chains connecting the indicating situation both to our beliefs and to the situation the information is about.

• Meaning is determined not just by us but by the world. Ultimately meaning is out there in the world.

Problems with Barwise and Perry

• Other than a lack of clarity about central commitments, most people will balk at meaning having to be out there in the world. It does not seem to be intentional (notice the similarity to one objection to Dretske).

• This assumption is made partly because of the role of the world in determining meaning of representations with multiple possible interpretations, but partly to close the gap between intentionality in representations and in the world.

• This problem could be solved if an nonintentional object could transmit information to a representation that is intentional, say by meeting Dretske’s requirement of digitality. I will argue for this solution later.

Summary to here

• Previous approaches invoke causality, transmission, and uncertainty (or probability). These are reasonable things to require in some way.

• Top-down formal approaches require logic and language we do not have and probably cannot have.

• Bottom-up approaches seem to require intentionality in the world, or else cannot explain intentionality.

• A combination approach looks most promising that describes the world (or objects) in a way that is isomorphic to intentional information, not intentional itself, and allows a channel from the world to representations and vice versa.

Order, causal power and work• Objects must be ordered to be able to affect anything else (cause

a change). This can either be by applying work, or having work applied, or both.

• This suggests that causal power is related to order (to make a difference).

• Work capacity is related to negative entropy (called intropy by engineers).

• Numerous authors have proposed that

Where is the actual entropy of a system, and is the entropy if all internal constraints are removed (the system relaxes to equilibrium). This is a work potential, available energy.

Order and entropy• We can define as ), P being the number of microstates

(complexions), and being the probability of the ith microstate.

• is then klogP, representing equiprobability, equilibrium and macroscopic indistinguishability of microstate.

• But there is another sort of order in a system, that which is intrinsic to it being that system. Elsewhere I have called it the system cohesion. This order is not released when the system goes to equilibrium (or the system would disrupt). I call it enformation. This evokes the form of the object. It has also been called structural information.

Intrinsic information• Intropy and enformation are both time-scale dependent, e.g., a

steam engine requires intropy to do work, but over time it will decay and eventually lose its enformation via intropy to the environment as its parts dissipate. Complete equilibrium is to become one with your environment. But at any given time the two are distinct in terms of immediate dynamic possibilities.

• We can define the intrinsic information of a system in terms of the sum of the two values, if enformation has a commensurate measure to intropy:

intropy Despite using the word “information” there is no implication of intentionality. Algorithmic information theory is best suited for measuring enformation, but its results are commensurate with the logarithmic approach (Kolmogorov).

A different approach to intrinsic information

• My student, Scott Muller, has developed an asymmetry principle of information based on an idea I proposed that information originates in symmetry breaking (bifurcations, or the production of distinctions in a system).

• Using group theory, Scott was able to show that any object has a unique number of asymmetries that can be recognized (using Jaynes’ IGUS notion).

• Asymmetries are directly related to intropy (asymmetric microstate distribution) and to enformation (distinct constraints on system identity), where information is the logarithm of the asymmetries.

• This gives us an alternative way to define intrinsic information.

Why call it intrinsic information?• The basic reason is that it represents the number of asymmetries or

differences that can make a difference to something else (potential IGUSs, for example) within a system.

• The measure is the same as several basic measures of information.• Furthermore, an infomorphism between an object so characterized and a

representation implies that the relation satisfies the requirement of Barwise and Seligman for information flow.

• This infomorphism will correlate the asymmetries in the object with the asymmetries in its representation.

• In the best case (fully accurate representation of the object) this will be an isomorphism.

• The isomorphism holds between causal powers of the object and causal powers of the embodiment of the representation.

• But the information is not intentional. If you don’t like that, find another name.

Brief technical aside on anambiguity concerning objects

• Talk of the object of a representation is ambiguous between what Peirce called the immediate object (of a sign) and the dynamical object.

• The immediate object is the reference of the sign in what logicians now call the intensional sense (note the ‘s’). As Peirce says, this is somewhat like an idea, and it is determined by the immediate interpretant of the sign, what we take it to be now, briefly.

• However the dynamic object is the causal basis of the sign, and it need not be what we think it is now, though we hope that a final interpretant will tell us exactly what it is, in the long run.

• The dynamic object is in this sense external to us, and may contain surprises that require us to revise our interpretations.

• When I refer to “complete accuracy” it would be about the dynamical object, and not the immediate object.

• Since information channels are directed, usually, we can still have information about a dynamical object if we have a channel from it to a sign’s immediate object. It is just incomplete as knowledge of the dynamical object.

Summary of intrinsic information• Causal power involves two parts, a capacity to guide work and

a capacity to do work. The first is the enformation and the latter is the intropy, and the distinction depends on time scale. Together they constitute causal power.

• They can be represented using standard methods of information theory.

• By the theory of information flow, both can transmit information to a representation, passively or actively.

• I pass over as obvious that representers must have both intropy and enformation, and so must their representations, individually. Any channels must be constrained, and thus must have enformation. Since constraint and information are interchangeable, channel constraints are also information.

An apparent paradox• The direct relationship between information and causal power

may seem paradoxical from the top-down view of information as a reduction of possibilities. It would seem that a situation with less information would allow more possibilities than one with more information, and consequently would have more causal capabilities.

• This, however, conflicts with our intuition that more complex things are capable of doing more than simpler things. The bottom-up approach is more in accord with this intuition.

• This is due to a constraint the bottom-up approach places on, physical possibility. It isn't the range of different abstract possibilities which determine intrinsic information content, but the productive power of a situation in optimal cases.

So where does intentionality enter?• Tacit representations do not require intentionality, but they must be functional

(capable of guiding anticipative behavior). • This means they must have similar properties to intentionality, like future

directedness and possibility of failure. Terry Deacon has called this property ententionality. I have written extensively on functionality and will not repeat it here, but briefly a trait is functional if it contributes to the autonomy, or self guided organization, of that which has it.

• Recall that they tacit representations have analog information. This may involve representational properties not in the object.

• Dretske argued that intentionality arises with digital representations, by which he means representations that contain all and nothing more of their object.

• This is realized when their causal embodiment mirrors their logical character. This is satisfied by the computational character of information, grounded in distinctions. This determines their modal character.

• This further means that intentionality involves and may be constituted by, the representation of logical consequences (and possibilities – i.e., modalities – derivable therefrom).

Tacit versus explicit representation• If I have a tacit representation of a white something that is

likely prey I can chase it without knowing that it is white and that it is prey, though chasing requires the information that it is prey and the ability to chase requires the information that it is white.

• If I have an intentional (explicit) representation of a red apple, I know it is of an apple and that it is of something red without requiring further information or reasoning.

• In both cases there is an infomorphism between object and representation, but the tacit case cannot be used reliably for reasoning. It should be noted that most “quick and dirty” reasoning is tacit, and prone to error.

Final summaryWe can have information about objects because1. Objects having enformation and intropy, allow

channels to our representations.2. The channels exist.3. Explicit, intentional information requires digital

representation.4. Intentionality, then is a structure of logical

consequences. It represents the structure of the object, which implies its causal power, in ideal cases.

Thank you for your attention.

Questions, comments?

Background papers are athttp://web.ncf.ca/collier