Information value:the value of evidenceDr David J Marsay, C.Math FIMAA presentation to 19 ISMOR29.08.2002

3

Contents

1 Introduction

2 Examples

3 Theory

4 World-view

5 Implications

6 Conclusions

IntroductionSection 1

5

Introduction

• Information is the life-blood of military C4ISR.

• Any time we prefer one set of information to another we implicitly ‘value’ it.

• We think we could do better:

– lessons identified.

– studies.

• Specifically needed to support UK MOD’s ARP 14 ‘Battlefield picture compilation’.

1

6

Introduction

• We use P(A|B) to denote ‘the probability of A given B’

– P(A) is used to denote the [prior] probability.

• For hypotheses {H} and evidence E:

– Shannon’s ‘entropy’ is calculated from the ‘final’ probabilities, {P(H|E)}.

– Jack Good’s ‘weight of evidence’ is calculated from the likelihoods, {P(E|H)}.

– According to Bayes’ rule, the probabilities can be calculated from the likelihoods and ‘the priors’.

1

ExamplesSection 2

8

Examples

Control: Suppose that a source sends accurate data to a deterministic machine.

• Shannon’s concept does not apply. Nor does the notion of ‘priors’.

• The value of the data can be determined by valuing the function of the machine - no fancy method needed.

• The likelihoods make sense. They are 0 or 1.

2

9

Examples

Command - ‘soft’ aspects:

• For an information artefact (e.g., an INTSUM) to represent the same information implies that all recipients had the same priors. Thus everyone receives everything in the same order.

– Is this realistic?

• Alternatively, one could define some privileged ‘central’ viewpoint for which the information is defined.

– Does this fit doctrine?

– Is it helpful?

2

10

Examples

Command - ‘soft’ aspects:

• The likelihoods {P(E|H)} are a rating of the source of E. They are thus relatively ‘objective’, ‘knowable’ and ‘shareable’.

• Likelihoods relate to current practice (reliability, accuracy).

2

11

Examples

Compilation:

• The work being reported on has looked at the relatively ‘hard’ problem of compilation, particularly ‘Battlefield picture compilation’ under ARP 14.

• Weights of evidence can be used. (See accompanying paper.)

• When is this reliable?

2

Theory

Section 3

13

Theory

Jack Good’s evidence:

• Likelihoods are often straightforward.E.g., P(‘Heads’|‘Fair Coin’) = 0.5 by definition.

• Lab and field testing traditionally establish, in effect, likelihoods.

• Surprise = -log(likelihood).

• Weight of evidence (woe) is surprise, normalised by the prior expected surprise for the same evidence. (So that only ‘relevant detail’ counts.)

3

14

Theory

Evidence is more fundamental than Shannon’s information

• Shannon’s entropy is expected surprise.

• The more useful cross-entropy is likely surprise.

• Woe supports alternative decision methods, such as sequential testing, hypothesis testing.

3

15

Some questionable assumptions

• Shannon assumes that systems of interest are ‘Markov’.

• Shannon noted that ‘state-determined systems’ are ‘Markov’ with probability 1.

• But Smuts (e.g.) noted that evolution drives dynamical systems to adopt synergistic ‘emergent’ structures.

• These had a priori probability 0.

• So for social systems, international relations, military conflict ... we cannot rely on Shannon’s ‘information’.

3

16

Some questionable assumptions

• But can likelihoods be used?

• If we abandon Markov models, how are we to judge if a given algebra of likelihoods is valid?

• We need a ‘world meta-model’ to replace Markov.

3

World-viewSection 4

18

• Allows one to investigate emergence within complex systems.

• Evidence of piece-wise Markov behaviour.

• Developed under the MOD CRP TGs 0,5,10.

SMUTS(synthetic modelling of uncertain temporal systems)

Delayed Double Vipert = 0.0502

BUBs2D5.5

t = 0.2005

4

19

Alternative ideas

• I postulate a model in which systems of interest to the military are Markov in space-time ‘zones’, with more interesting transitions at their boundaries.

• Thus Markov locally, but not globally.

• In essence emergence only happens when an over-adaptation is exploited. (E.g. Ashby, Piaget.)

• Thus, as long as we can learn at least as quickly, we should be able to recognise these situations too.

4

20

Supporting evidence

Applications to, and experiences of:

• warfare

• economics

• international relations.

(My subjective view)

4

21

Reuters data for the Balkans, the 90s4

Evidence of locally Markov behaviour

Balkans April 1989- March 1999KEDS data from www.ukans.edu/~keds

Entropy / Value Aggregated Monthly Phase plot

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

0 10 20 30 40 50 60 70 80 90 100

RSS Value

Tra

nsacti

on

al

En

tro

py,

aft

er

Sh

an

no

n

4/89-4/91

4/91-11/91

11/91-10/95

10/95-4/97

4/97-9/97

9/97-1/99

1/99-3/99

ImplicationsSection 5

23

Implications for ‘information’

Technical differences:

• The difference between the expected weight of evidence (woe) and Shannon’s entropy is not a constant.

• Systems of interest tend to have ‘long range’ sources of uncertainty, in addition to the ‘local’ entropy.

• We need to allow for this and ‘expect the unexpected’ to achieve robustness.

5

24

Implications for ‘information’

Some cases where Shannon might not be appropriate

• Poor ‘local’ information.

• The ‘situation’ cannot necessarily be recognised.

• The ‘target’ is adaptable (particularly if adapting against us).

5

25

Implications for ‘information’

Typical symptoms that Shannon is inadequate:

• Mistakes often reflect a need to validate assumptions.

• Ossification, atrophy and vulnerability (Ashby / Piaget)

5

26

Implications for ‘information’

Notes:

• We can’t expect to have considered all possible hypotheses in advance.

• However, we do know when the truth is ‘something else’ because the weights of evidence are poor for the assumed hypotheses.

• Thus we can detect deception and ‘fixation’ (a form of self-deception).

5

ConclusionsSection 6

28

Conclusions

• The common concepts of information assume that systems are globally ‘simple’.

• Our systems of interest are not simple, but may be piece-wise ‘simple’.

• Jack Good’s ‘weight of evidence’ can be used to ‘bridge’ ‘islands of simplicity’.

• Using ‘weight of evidence’ gives significant ‘added value’ to using just Shannon information.

6