From logic and probability to argument and evidence: a

cognitive perspective

John FoxLondon Research Institute

Cancer Research UK

Argument and evidence: ancient forms of thought

Gods

Oracles

Meaning 1

Meaning 2

Observations & reports

Dreams

May, CanWill …

Diagnosesprophecies

Humours

Thanks to Jason Davies

The heart of it

“Some philosophers … have an ineradicable suspicion of our everyday forms of thought … In their view, the development of science, and the displacement of all our ordinary, pre-scientific ideas by the more refined notions of the theoretical sciences, hold out the only hope of salvation for incoherence, fallacy and intellectual confusion.” S E Toulmin, 1958 p92

“… there is essentially only one way to reach a decision sensibly. First, the uncertainties present in the situation must be quantified in terms of values called probabilities. Second, the consequences of the courses of actions must be similarly described in terms of utilities. Third, that decision must be taken which is expected on the basis of the calculated probabilities to give the greatest utility. The force of ‘must’ used in three places there is simply that any deviation from the precepts is liable to lead the decision maker in procedures which are demonstrably absurd” D Lindley, 1983

“As living and moving beings, we are forced to act … [even when] our existing knowledge does not provide a sufficient basis for a calculated mathematical expectation.” J M Keynes, 1936

Overview of talk

• What is an argument? Toulmin’s alternative to “scientific” views of logic and probability

• Qualitative and quantitative aspects of argument and evidence

• Dung’s calculus of opposition; LA a logic of argument

• Talking about argument and evidence – the place of ordinary language

• Argument as a “meta-language”

What is an argument?

• Classical viewP, (P Q) Q

Since a man born in Bermuda will generallyBe a British citizen

Harry was born

in Bermuda

So, presumably,

Harry is a british citizen

On account of the followingStatutes and other legalProvisions …

Unless

• Both his parents were aliens• He has become a naturalised American

• Non-traditional view (Toulmin)

Data Qualifier, Claim

since

Warrant unless

Rebuttal

On account of

Backing

Argument and evidence: Toulmin form

Background knowledge

Field 2

Field n

Claim 1 Claim 2

Qualifiers (possibly, probably,

presumably …)

Data

Field 1

Qualified claims

Toulmin in practiceSteele et al, Proc. European Conf. AI in Medicine 2003

-+

Argument and evidence: modern form (medicine)

Knowledge base

Epidemiology

Genetics

Hypoth-esis 1

-++

Argument aggregation

Strength of evidence

Argument pattern

Modalities (possibly, probably,

presumably …)

Records, symptoms, test results

Immunology

Qualified hypotheses

Anatomy

Metabolism

Pathology

Hypoth-esis 2

Medical arguments

“Fields” of reasoning

• Immunology• Physiology• Anatomy • Biochemistry• Genetics• Morphology• Epidemiology• Mental health• Social dysfunction

Modes of reasoning

• Causal• Statistical• Functional• Structural• Spatial• Temporal• Modal• Deontic

} anatomy

Argument and evidence: Bayesian form

Knowledge of variables

CPs 2

CPs n

Hypoth-esis 1

Hypoth-esis 2

0.88

Probability revision

Belief revision

Data and priors

CPs 1

Posteriorprobabilities

0.440.33

0.22

Protein structure prediction

Argument and evidence: protein structure prediction

Knowledge base

Predict 1

-++

Argument pattern

Modalities (possibly, probably,

presumably …)

Amino acidsequence

Predict 2

Physical folding

Structural argumentse.g. packing constraints

Physical folding

Structural argumentse.g. packing constraints

Causal argumentse.g. charged residues on outside of molecule

Physical folding

Structural argumentse.g. packing constraints

Causal argumentse.g. charged residues on outside of molecule

Energetic argumentse.g. folding minimises free energy

Tentative models

Other constraintsFunctional arguments

e.g. reducing osmotic pressure

Other constraintsFunctional arguments

e.g. reducing osmotic pressure

Evolutionary argumentse.g. conservation of structures over generations

Other constraintsFunctional arguments e.g. reducing osmotic pressure

Evolutionary arguments e.g. conservation of structures over generations

Similarity arguments e.g. homologies between related proteins

Argument aggregation

Probabilities

Formalising argument: Dung’s “calculus of opposition”

(1995)

• The basic intuition is that a statement is “believable” if we can identify an acceptable argument for it

• An acceptable argument is – one that is not attacked, or – if it is attacked the attacking

argument can be defeated

Attack

Defeat

The calculus of oppositionDung (1995) et seq

• Support can include ordinary logical proofs with negation as failure, and Defeat is a contradiction of this proof (rebuttal of the conclusion or undercutting of a premise)

• Dung provides formal criteria for– Admissibility of arguments – Stable extensions, Preferred extensions– Fixpoint semantics, Grounded semantics

• All argument games, debates, disputes, etc. are to be analysed solely in terms of these concepts

Araucaria argument modelling tool: Chris Reed and Glenn Rowe, U Dundee

Is the Dung interpretation sufficient? Genetic risk assessment

-

+c

Coulson et al Methods of Information in Medicine 2001. Emery et al British Medical Journal 2000, 2001,

LA: A logic of argument

Syntax – Propositional symbols: , , …– Connectives: , , , , – Dictionary of qualifiers:

• +, -, ++, --• Variables (Q, Q’, Q’’)

– Auxiliary symbols: (, ), :

Data Theory LA (Claim : Qualifier)

Introduction rules

(I) :Q :Q`: min(Q,Q`)

:++(I)

:Q :Q

(I) :Q :Q:Q :Q

:++(I)

:Q:Q

Elimination rules

(E) :Q :Q :Q :Q

(E) :Q :Q` : min(Q,Q’)

:++ :++

(E) :Q :Q` :Q``

:min(Q,Q `,Q``)

(E) :Q :Q` :min(Q,Q`)

Weakening :++ :+

.

.

.

.

.

.

.

.

....

Fox et al Proc. Eur. Conf. AI 1992; Fox et al Proc. Uncertainty and AI 1993; Krause et al Comp. Intelligence, 1994

LA: Logic of argument

Modelling uncertainty in LA: qualifiers and dictionaries

• Symbolic dictionaries{+,-} delta dictionary (“pros

and cons”){++,--, +, -} bounded delta

• Quantitative dictionaries[0,1] probability, possibility[-1,+1] certainty factors{1,2,3,…n} integer weights

• Linguistic dictionariesP-modals (Possible, probable, plausible …)

- +

Argument and evidence: LA form

Deductive database

Theory 2

Theory n

World 1 World 2

-++

Argument aggregation

Measures of strengthe.g. from [0,1]

Argument pattern

Modalities (e.g. possibly, probably,

presumably …)

Situation beliefs, goals, plans

Theory 1

Defeasible claims

Fox et al Proc. Eur. Conf. AI 1992; Fox et al Proc. Uncertainty and AI 1993; Krause et al Comp. Intelligence, 1994

Talking about evidence and riskthe place of natural language

J Fox “Will it happen? can it happen? a new approach to formal risk analysis” Risk and public policy (1999).

Informal semantics: talking about uncertainty, risk and evidence in

everyday language

Guidelines for the evaluation of chemicals for carcinogenicity International Agency for Research on Cancer

confirmed Carcinogenepidemiological data and/or established causal relationship

possible Carcinogenpotential hazard recognised

probable Carcinogenbetter evidence than merely recognition of possible carcinogenic activity

improbable Carcinogenpossible carcinogenic activity, but strong evidence against action

equivocalhazard recognised and both evidence for and evidence against

not Carcinogentest case data or direct chemical analysis disconfirms carcinogenic activity

openno information regarding potential hazard available

From arguments to modalities

Given a set of arguments for and against a claim we can map the set into many different qualifiers or modalities

{ (Claim : Warrant : Qualifier) } (Claim : Modality)

Data Theory LA (Claim : Warrant : Qualifier)

P is if it is any well-formed formula in the language of the logic

P is if an argument, possibly using inconsistent data, can be constructed

P is if a consistent argument can be constructed (we may also be able to construct a consistent argument against)

P is if a consistent argument can be constructed for it, and no consistent argument can be constructed against it.

P is if it satisfies the conditions of being probable and, in addition, no consistent arguments can be constructed against any of the premises used in its supporting argument

P is if it is a tautology of the logic (meaning that its validity is not contingent on any data in the knowledge base).

Logical acceptability classes

Elvang-Gorannson, Krause and Fox “Logic and linguistic uncertainty terms” Proc. UAI (1993)

Logical acceptability classes: “Linguistic annotations”

P is openif it is any well-formed formula in the language of the logic

P is supportedif an argument, possibly using inconsistent data, can be constructed

P is plausibleif a consistent argument can be constructed (we may also be able to construct a consistent argument against)

P is probableif a consistent argument can be constructed for it, and no consistent argument can be constructed against it.

P is confirmedif it satisfies the conditions of being probable and, in addition, no consistent arguments can be constructed against any of the premises used in its supporting argument

P is certainif it is a tautology of the logic (meaning that its validity is not contingent on any data in the knowledge base).

Elvang-Gorannson, Krause and Fox “Logic and linguistic uncertainty terms” Proc. UAI (1993)

Combining quantitative and qualitative methods(REACT: Risk, Events, Actions and their Consequences over

Time)

David Glasspool, Tito Castillo, Vicky Monaghan, Ayelet Oettinger

Argument and evidence: legal form

Case law

Expert opinion

Guilty

-++

Argument aggregation

Quantitative talkArgument

pattern

Language based talk

Claimedfacts

Witnesstestimony

Judgement (subject to appeal)

Not guilty

Charting the evidence(The Umilian case, Wigmore 1931)

Argument as meta-language

 CLAIM Uncertainty must be represented with a well-behaved measure ([0,1] probability)

We often cannot measure uncertainty, but must still act (authority: Keynes) Medical and countless everyday examples

Measures that do not satisfy the [0,1] probability axioms will be “incoherent” Dutch book and similar arguments (e.g. Lindley) This does not necessitate that all formalisms are incoherent in all circs.

the Delta argument There is a vast body of well-understood, classical techniques available

Classical methods don’t address all the problems Example, the “ill-formed problem” problem Example, the human communication problem Example, the knowledge representation problem Example, the meta-representation problem

Non-probabilistic methods lack precision and cannot work well There is a significant body of empirical evidence that they do Theory is a stronger guide than mere evidence

That depends upon your point of view!

CLAIM “[0,1] probability is the only correct representation for reasoning under uncertainty”

There is a theoretical case against There are sound alternatives to probability

Example, possibility theory (Quantitative) Example, modal logic (Qualitative)

There are formalisms that are more expressive Example, first-order logic (description logics?)

There is a practical case against We need more general methods in medicine than probability We commonly lack a basis for estimating probabilities

We can manage this, e.g. by upper-lower bounds This compounds the problem technically Example, estimation difficulties Example, computational costs We may not know whether other assumptions are satisfied

Example, conditional independence Example, probability estimates are valid

The logical structure (topology) is what matters not the weightings A substantial body of empirical information supports this

Argument as meta-languageCLAIM “[0,1] probability is the only correct representation for reasoning under

uncertainty”

(the case against)

Psychological arguments In the end uncertainty is an entirely polysemous and subjective notion

Example, linguistic terms (belief-doubt, possible-probable) Uncertainty can be treated as a uni-dimensional quantity measure

Objective (frequentistic) Subjective (Bayesian)

People find mathematical probability difficult to understand We need to develop rational theory, not make people happy That depends on your point of view

Representational arguments We need to use the representation with the highest practical benefit and power

No, we must use a standard, normal form i.e. probability The normal form should be the most general form (logic of argument!)

There are arguments for and against probability as an objective mathematical idea Scientific truth excludes inconsistency, you can’t “cherry pick” theories

Even scientific theories must live with ambiguity Example: Waves and particles Theories are always incomplete and inconsistent

Lakatos, Kuhn, Popper, Feyerabend …

Argument as meta-languageCLAIM “[0,1] probability is the only correct representation for reasoning under

uncertainty”

(the case against, continued)

Argument and evidence: generic form

“Background” knowledge

Area 2

Area n

Claim 1

-++

Argument aggregation

Quantitative talkArgument

pattern

Language based talk

Knownsituation

Area 1

Conceivable situation

Claim 2

Conclusions• We currently view much “common sense” reasoning as

degenerate forms of reasoning (e.g. talk about evidence or risk) • Toulmin criticised the scientific view, claiming that natural

argument implements a different but valid reasoning model• Recent work in AI and non-classical logic provides a clearer

interpretation and sound formalisation for such ideas• LA provides a way of reconciling natural patterns of reasoning

with scientific and mathematical forms, possibly grounded in Dung semantics (in part)

• Studies of risk assessment, decision-making and planning in medicine suggests that the approach is practically effective

• Reactions suggest that the approach is also intuitive for those who are formally trained and those who are not.

• The challenge is to develop our understanding of the relationship between quantitative and qualitative perspectives on uncertainty, risk and evidence rather than seeing them as competitors.

Some references

• Fox J et al “Argumentation as a general framework for uncertain reasoning” Proc. Uncertainty in AI, 428-434, 1993

• Dung P M “On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games” AI Journal 77, 321-357, 1995

• Krause P et al “A logic of argumentation for reasoning under under uncertainty” Computational Intelligence, 11, 113-131, 1995

• Fox J “Logic, probability and the cognitive foundations of rational belief” J Applied Logic, 1, 197-224, 2003

Aleatory matrixAleatory: depending on an uncertain event or contingency as to both

profit and loss

Subjective Objective

Situation

Believe, think, assumeExpect, anticipate

Equivocal

Probable, plausible, possible, necessary, will be, may be

Ambiguous

Action PreferIntend, plan

Permitted, obligatory

Aleatory

Expected value, supported, opposed

Cost-benefit, advantages-disadvantages, benefits-harms

Aleatory semanticsAleatory: depending on an uncertain event or contingency as to both

profit and loss

Subjective Objective

Situation

Epistemic arguments

Possible worlds, frequentistic probability

Action Deontic arguments

Possible worlds,

Aleatory

Goals Value functions

Semantics of medical knowledge

Breast Cancer Ontology Fragment (Total c.100,000 concepts) REPRODUCTIVE SYSTEM AND BREAST DISORDERS

IS_AWOMAN WITH POSSIBLE BREAST CANCER HYPOTHESIS DISEASE OF THORAX

HAS-HEALTHCARE-PHENOMENON IS_AIS_A IS_A BREAST FINDING

BREAST CANCER HYPOTHESIS BREAST DISEASE

IS-HYPOTHESIS-OF IS_A IS-RANGE-OF-DOMAIN-OF

MALIGNANT NEOPLASM OF BREAST BREAST SPECIALIST

IS_A IS-SPATIAL-PART-OF

MALIGNANT NEOPLASM IS_A BREAST STRUCTUREIS_A

HAS-WE-STATE IS_A IS-CONSEQUENCE-OF

MALIGNANT NEOPLASM BREAST CANCER STAGE I

IS_CREATIVE_RESULT_OF

NEOPLASTIC PROCESS ADVANCED BREAST CANCER ADVANCED

IS_A IS_AHAS-HEALTHCARE-PHENOMENON HAS-WE-STATE

ADVANCED CANCERCANCER PATIENT

IS_A IS_A

PATIENT BREAST CANCER PATIENT

Spoken/typed input

Applying the knowledge

Breast Cancer Ontology Fragment (Total c.100,000 concepts) REPRODUCTIVE SYSTEM AND BREAST DISORDERS

IS_AWOMAN WITH POSSIBLE BREAST CANCER HYPOTHESIS DISEASE OF THORAX

HAS-HEALTHCARE-PHENOMENON IS_AIS_A IS_A BREAST FINDING

BREAST CANCER HYPOTHESIS BREAST DISEASE

IS-HYPOTHESIS-OF IS_A IS-RANGE-OF-DOMAIN-OF

MALIGNANT NEOPLASM OF BREAST BREAST SPECIALIST

IS_A IS-SPATIAL-PART-OF

MALIGNANT NEOPLASM IS_A BREAST STRUCTUREIS_A

HAS-WE-STATE IS_A IS-CONSEQUENCE-OF

MALIGNANT NEOPLASM BREAST CANCER STAGE I

IS_CREATIVE_RESULT_OF

NEOPLASTIC PROCESS ADVANCED BREAST CANCER ADVANCED

IS_A IS_AHAS-HEALTHCARE-PHENOMENON HAS-WE-STATE

ADVANCED CANCERCANCER PATIENT

IS_A IS_A

PATIENT BREAST CANCER PATIENT

Spoken/typed inputSpeech recogniser

(commercial)

Finite state machine for low level response

selection

High level dialogue manager

Applying the knowledge

Argument and decision

General knowledge, expertise

Area 2

Area n

Option 1 Option 2

-++

Argument aggregation

Strength of evidencee.g. [0,1]

Argument pattern

Status of hypothesis (possibly, probably,

presumably true …)

Knownsituation

Area 1

Conceivable situation