OntologiesReasoningComponentsAgentsSimulations

Reasoning about actions, Reasoning about actions, change, events and timechange, events and time

Jacques Robin

OutlineOutline

Non-monotonic reasoning When and where to reason about

actions, change, event and time? Reasoning services and illustrative

case study Roadmap of approach The Situation Calculus (SC)

Key ideas Representation Reasoning The frame problem The ramification problem The qualification problem Limitations

The Event Calculus Key ideas Normal logic programs and negation

as failure Representation Reasoning Abductive logic programs and

abductive planning Solutions to the frame and

ramification problems Limitations and comparison with SC

Transaction Logic Key ideas Transaction logic program syntax Transaction logic program semantics Representing action and changes in

transaction logic Solution to the frame and

ramification problems Limitations and comparison with SC,

EC

Non-monotonic reasoningNon-monotonic reasoning

Classical logic reasoning is monotonic If KB |= f, then KB g |= f Inference engine only performs ask and tell to the KB, never retract

Non-monotonic reasoning Allows KB |= f, and then KB g | f Previously derived facts can be retracted upon arrival (for example from

sensors) of new, conflicting evidence Two distinct sources of non-monotonicity

Ontological: in non-stationary environments E KB must reflect environment changes as time goes by when fact f true in E(t) no longer true in E(t+1), (i.e., the agent’s position) if KB does not use a historically cumulative knowledge representation scheme,

sentence s in KB(t) that represents f must retracted from KB(t+1) and so must all sentences in KB(t) proven using s!and so must all sentences in KB(t) proven using s! (truth-maintenance)

Epistemological: in partially observable environments E Decision making with partial knowledge requires using abduction instead of pure

deduction Some sentences in KB(t) were deductively derived from sensors and KB(0) While others were merely abduced (i.e, hypothesized as true or false by default) In the face of

Two orthogonal sources of Two orthogonal sources of non-monotonicitynon-monotonicity

Ontological:Ontological: in non-stationary environment E KB must reflect environment changes as time goes by when fact f true in E(t) no longer true in E(t+1) unless KB uses a historically cumulative knowledge representation

scheme sentence s in KB(t) that represents f must retracted from KB(t+1), and so must all sentences in KB(t) proven using s! (truth-maintenance)

Epistemological:Epistemological: in partially observable environment E Decision making with partial knowledge requires using abduction in

addition to deduction KB must reflect changes of the agent’s beliefs as new evidence

becomes available through sensing KB(t) = D(t) A(t,H(t)), where

D(t) are sentences derived purely deductively from percept sequence and KB(0)

A(t,H(t)) are sentences derived using at least one abductive step relying on some hypothesis in H(t)

When a D(t) and a A(t-1,H(t-1)) , a must be retracted from KB(t) and so must all sentences in A(t-1,H(t-1)) derived using a!

When to reason about actions, When to reason about actions, change, events and time?change, events and time?

Non-stationary environments Sequential (only actions and change)

Concurrent synchronous (actions, change and events)

Perc

ep

t

Environment

Agent

Acti

on

State 1

Reasoning

Perc

ep

t

Acti

on

State 2

Reasoning

Perc

ep

t

Acti

on

State 3

Reasoning

State N...

...

Perc

ep

t

Agent

Acti

on

State 1

Reasoning

Perc

ep

t

Acti

on

State 2

Reasoning

State 4 State 5State 3Environment

When to reason about actions, When to reason about actions, change, events and time?change, events and time?

Non-stationary environments Concurrent asynchronous (actions, change, events and time)

...P

erc

ep

t

Agent

Acti

on

State 1

Reasoning

State 2 State 4State 3

Perc

ep

t

Acti

on

State 5

Reason

ing

State 6Environment

Where to reason about actions, Where to reason about actions, change, events and time?change, events and time?

Enviro

nm

ent

Sensors

Actuators

Environment model(past and present)

Percept interpreter:percepts(t) model(t-1) model(t)

Action chooser:model(t) actions(t)

Environment model updater:model(t-1) model(t)model(t) model’(t)actions(t) model(t) model(t+1)

Automaton Agent

Where to reason about actions, Where to reason about actions, change, events and time?change, events and time?

Enviro

nm

ent

Sensors

Actuators

Model of past and current

environments

Percept interperterpercepts(t) model(t-1) model(t)

Action chooser result([action(t),...,action(t+n)]) = hyp(model(t+n)) hyp(model(t+n)) goal(t) do(action(t))

Environment model updatermodel(t-1) model(t)model(t) model’(t)do(action(t-1)) model(t)

Goal updatermodel(t) goals(t-1) goals(t) Goals

Future environments predictormodel(t) hyp(action(t)) hyp(model(t+1))model(t) model(t+1) Model of

hypotheticalfuture

environments

DeliberativeAgent

Reasoning servicesReasoning services

Environment model decomposed in: EternalsEternals: properties and relations not affected by actions, events or the

passing of time FluentsFluents: properties and relations that change as result of the execution of

an action, the occurrence of an event or the passing of time Single action (event) consequence:

Given current environment state model Compute fluents resulting from the execution (occurrence) of a single

action (event) Temporal projection:

Given current environment state model and a sequence of hypothetical actions to execute and events to occur Compute fluents of the resulting environment state

Planning: Given current environment model and a set of goal fluents Compute action sequence whose execution will turn these goal fluents true

Illustrative case study: the block Illustrative case study: the block worldworld

Agent: robot with one arm building stacks from blocks on a table

Eternals: isclear(table) block(B)

Fluents: on(Block,Loc) isclear(Loc) above(B1,B2)

Action: safeMove(B,L1,L2) preconditions

on(B,L1) isclear(L2) isclear(B)

intended effects on(B,L1) on(B,L2)

side effects block(L2) isclear(L2)

Action: riskyMove(B,L1,L2) preconditions

on(B,L1) isclear(L2) intended effects

on(B,L1) on(B,L2) side effects

X, above(X,B) (above(X,L1) above(X,L2))

block(L2) isclear(L2)

ab c

table

abc

table

Roadmap of the approaches: Roadmap of the approaches: situation and event calculisituation and event calculi

Full Classical First-Order Logic

(FCFOL)

SC Inference Engine:Non-monotonic reasoningabout actions and change

FCFOL Theorem Prover:Monotonic deduction

Situation Calculus

(SC)

added constructs (domain-independent):• function: result• predicates: preconds, posEffect, negEffect

reuses

EC Inference Engine:Non-monotonic reasoning

about actions, events, change and time

Tabled Prolog Engine:Negative abduction

Event Calculus

(EC)

added constructs (domain-independent):• predicates: holds, happens, initiates, terminates, clips, <• clauses defining EC semantics on top of ALP/NLP semantics

reuses

(Pure) Definite Logic Program

(DLP)

ALP Engine:Positive and

negative abduction

reuses

AbductiveLogic Program

(ALP)added constructs:• abducible predicates• integrity constraint clauses

NormalLogic Program

(NLP)

added construct:• connective Negation As Failure (NAF)

Roadmap of the approaches: Roadmap of the approaches: transaction logictransaction logic

TL(HL,{BTI,BTD}) Inference Engine:Non-monotonic reasoning about

and programming actions and change

Tabled Prolog Engine:Negative abduction

Transaction Logicw/ Data Oracle: Hilog& Data Update Oracle: btinsert(HiLogTerm), btdelete(HiLogTerm)(TL (HL,{BTI,BTD})

added constructs):• backtrackable knowledge base update predicates: btinsert, btdelete• sequential connectives: ,

reuses

(Pure) Definite Logic Program

(DLP)

HiLog:Negative abductionand meta-abduction

reuses

NormalHiLog Program

(NHP)added constructs:• high-order functions• high-order predicates• reified clauses

NormalLogic Program

(NLP)

added construct:• connective Negation As Failure (NAF)

The situation calculusThe situation calculus

FCFOLConnective FCFOLFormula FCFOLAtomicFormula FCFOLTerm

FCFOLFunctionalTerm

FCFOL NonFunctionalTerm

Ontology of the Situation CalculusOntology of the Situation Calculus

Functor Arg1..2

Arg1..* Arg 1..*

Fluent Atom

EternalAtom

Fluent Term

Eternal Term

Result Term

Action Term

Situation

PrecondAtom

EffectAtomNegEffect

Atom

PosEffectAtom

PrecondAxiom

EffectAxiom

FrameAxiom

SuccessorStateAxiom

RamificationAxiom

Precondition and effect axiomsPrecondition and effect axioms

Precondition axiomsPrecondition axioms B,L1,L2,S block(B) on(B,L1,S) isclear(L2,S) isclear(B,S) precond(safeMove(B,L1,L2),S) B,L1,L2,S block(B) on(B,L1,S) isclear(L2,S)

precond(riskyMove(B,L1,L2),S) Effect axiomsEffect axioms

B,L1,L2,S,S1 precond(safeMove(B,L1,L2),S) (S1 = result(safeMove(B,L1,L2),S) (S1 = S) on(B,L1,S1) on(B,L2,S1) (block(L2) isclear(L2,S1)))

B,L1,L2,S,S1 precond(safeMove(B,L1,L2),S) (S1 = result(riskyMove(B,L1,L2),S) (S1 = S) on(B,L1,S1) on(B,L2,S1) (block(L2) isclear(L2,S1)))

The frame problemThe frame problem

In most non-stationary environments: Most fluents remain the same most of the time Each action only affects a tiny percentage of all fluents

Effect axioms only specify the fluents changed by an action But the execution of an action changes the situation How to specify that fluents unchanged by the last executed

action carry over to the new situation that it created? Naive approach: frame axiomsframe axioms

B,L1,L2,S,S1,B1,B2 precond(safeMove(B,L1,L2),S) on(B1,B2,S) (B = B1) (B = B2) (S1 = result(safeMove(B,L1,L2),S) (S1 = S) on(B1,B2,S1)

Leads to exponential number of frame axioms

Solving the frame problem Solving the frame problem in the situation calculusin the situation calculus

Substituting implicative effect and frame axioms by successor state axiomsuccessor state axiom using equivalence

Successor state axiom schema (precond(A,S) (fi(result(A,S)) ((posEffect(A,fi))

fi(S) negEffect(A,fi))) .... posEffect(aj,fi) ... negEffect(ak,fi) ...

where S is a situation, A,aj,ak an action and fi are fluents

Example: (A,B,L,S precond(A,S)

(on(B,L,result(A,S)) posEffect(A,on(B,L)) (on(B,L,S) negEffect(A,on(B,L)))) A,B,L1,L2,S posEffect(safeMove(B,L1,L2), on(B,L2)) A,B,L1,L2,S posEffect(safeMove(B,L1,L2), isclear(L1)) A,B,L1,L2,S negEffect(safeMove(B,L1,L2), on(B,L1)) A,B,L1,L2,S negEffect(safeMove(B,L1,L2), isclear(L2)) A,B,L1,L2,S posEffect(riskyMove(B,L1,L2), on(B,L1)) ...

The ramification problemThe ramification problem

Actions have: intended effects that satisfy the agent’s goal and justify their

execution side effects related to preconditions of other actions

Modeling side effects together with intended ones in successor state axioms link them to actions leading to combinatorial explosion

Side effects best modeled orthogonally as ramification axiomsramification axioms that relate fluents among themselves independently of actions in the same situation

Example: B1,B2,B3,S above(B1,B2,S)

(on(B1,B2,S) (on(B1,B3,S) above(B3,B2,S))) Execution of riskyMove(b,c) in S-1 w/ above(a,b,S-1) Successor state axiom allows deducing: on(b,c,S) above(a,b,S) Ramification axiom then allows deducing: above(a,c,S)

The qualification problemThe qualification problem

Because it is based on classical logic, the situation calculus requires explicitly modeling all preconditions and all side-effects of every action in every situation

In practice, rarely occurring combinations of fluents are easily overlooked when specifying preconditions and side-effects

This is the qualification problem: think of everything that can go wrong

Best solved by probabilistic knowledge representation, aggregating all unforeseen cases in one “catch-all” situation

Unique name axiomsUnique name axioms

Being based on classical logic, a KB using the situation calculus must contain unique name axiomsunique name axioms stating that no two members of the Herbrand base are equal unless explicitly specified

Example: a = b a = c a = table ...

... (B,L1,L2,S,B’,L1’,L2’,S’ safeMove(B,L1,L2,S) = safeMove(B1’,L1’,L2’,S’) (B = B1, L1 = L1’, L2 = L2’, S = S’)) ...

Limitations of the situation calculusLimitations of the situation calculus

Expressiveness: Punctual and implicit representation of time by way of situations All changes result from a single agent executing actions Limited to sequential, mono-agent environments Non-intuitive for being based on full classical first-order logic

Efficiency: Very space inefficient

Guards the entire history of the environment If a fluent changed only once in a 2001 steps process, the knowledge

base contains 2000 copies of it, each one w/ a different situation argument

Time inefficient Reasoning relies on theorem proving in full classical first-order logic Determining the truth of a fluent requires regressing through the entire

history of the environment

The Event Calculus (EC)The Event Calculus (EC)

Exists in various version of increasing expressiveness Most expressive versions overcome all the expressiveness

limitations of the SC except the ramification problem Also more time efficient than the SC for relying on abductive

logic programming in Horn first-order logic instead of theorem proving in classical first-order logic

Explicitly represents time points, time intervals, punctual events, events with durations, sequential and concurrent events

An agent executing an action is a special case of event

tnot(holds(Fluent)) holds(Fluent)

initiates(Event,Fluent)

terminates(Event,Fluent)

not(holds(Fluent))

General logic programsGeneral logic programs

General (or normal) logic programs extend pure (or definite) logic programs with Negation As Failure (NAF) connective notnot that can precede atoms in a clause premise

Inference engine is extended with step to prove premise not A Generally it tries to prove goal A:

if it succeeds, then not A fails if it finitely fails to do so, it assumes A false (negatively abduces A)

and not A succeeds abducing not A from failure to prove A it is a completely different

inference than deductively proving A in classical first-order logic Tabled logic programming engine can detect loops through NAF

such as query q with program {p :- not q. q :- not p.} It computes the Well-Founded Semantics (WFS)Well-Founded Semantics (WFS) in ternary logic When A neither succeeds nor can it finitely fail due to NAF loops, the

WFS assigned truth value unknown to both A and not A

Representing a domain with ECRepresenting a domain with EC

Domain-dependent knowledge represented as abductive logic program clauses using EC predicates: holds(F,T): fluent F is true at time T precond(E,T): precondition of event E are fulfilled at time T do(A,T): action A is executed at time T happens(E,T): event E occurs at time T initiates(E,F,T): fluent F becomes true as a positive effect of event E terminates(E,F,T): fluent F becomes false as a negative effect of event E clipped(F,T1,T2): fluent F becomes false between time T1 and time T2

Domain-independent EC clauses that specify generic relations between EC predicates, that axiomatize their temporal reasoning semantics holds(F,T2) :- happens(E,T1), initiates(E,F,T1), T1 T2,

not clipped(F,T1,T2). clipped(F,T1,T3) :- happens(E,T2), terminates(E,F,T2),

T1 T2, T2 T3. happens(E,T) :- precond(E,T), not action(E). happens(E,T) :- precond(E,T), action(E), do(E,T).

Block world EC clausesBlock world EC clauses

Initial state clauses: holds(on(a,b),0). holds(on(b,c),0). holds(on(c,table),0).

holds(isclear(a),0). Precondition clauses:

precond(safeMove(B,L1,L2),T) :- holds(on(B,L1),T), holds(isclear(L2),T), holds(isclear(B),T).

Positive effect clauses: initiates(safeMove(B,L1,L2), on(B,L2),T). initiates(safeMove(B,L1,L2), isclear(L1), T).

Negative effect clauses: terminates(safeMove(B,L1,L2), on(B,L1),T). terminates(safeMove(B,L1,L2), isclear(L2), T).

Ramification clauses: holds(above(B1,B2),T) :- holds(on(B1,B2),T). holds(above(B1,B3),T) :- holds(on(B1,B2),T), holds(above(B2,B3),T).

Variations of the ECVariations of the EC

Basic EC already handles concurrent actions and time intervals Additional arguments to EC predicates and additional EC

clauses allow extending basic EC to handle: Events with durations: happens(E,T1,T2) Multiple agents: do(Agent,Action,T)

Abductive logic programmingAbductive logic programming

Extend general logic programs with: Integrity constraints I A distinguished subset of predicate symbols called abduciblesabducibles Atoms with an abducible functors (abducible atoms) are generally

restricted to appear only in clauses premises they have no definitions where they appear in conclusion

An abducible atom goal is assumed true (positive abduction) whenever: Its assumption does not allow deducing conclusions that violate

the integrity constraints Abduction can be used to construct plans using an EC

representation of the domain D: holds atoms represent the goal G to be reached do atoms represent the hypothesis H to abduce The abductive proof procedure computes H such that:

D H |= G D H I | false

Transaction Logic (TL): starting pointTransaction Logic (TL): starting point

A set of Prolog clauses defining a predicate have two distinct semantics: A declarative, logical one (Clark’s completion or least Herbrand

model) A procedural one (implicit)

Example p :- q, r. p :- u. Declarative semantics: p ((q r) u) Procedural semantics:

define p {{call q; if q = true then {call r; return r} else return fail}; {call u; return u}}

In the declarative semantics the order of q, r and u does not matter

In the procedural semantics, the order is crucial Transaction logic unifies the two semantics by representing the

procedural one explicitly and declaratively

Transaction logic connectivesTransaction logic connectives

Extends normal logic programs with backtrackable clause update predicates and two new connectives

btinsert{C1, ..., Cn} premise has side effect to explicitly add clauses C1, ..., Cn to the logic program

btdelete{C1, ..., Cn} premise has side effect to explicitly delete all clauses that match C1 or ... or Cn from the logic program

Serial conjunction : Declaratively captures procedural semantics of Prolog’s conjunction Distinct from classical conjunction In TL: a, b b, a, but a b b a

Serial disjunction : Declaratively captures procedural semantics of Prolog’s disjunction Distinct from classical disjunction In TL: a; b b; a, but a b b a

Transaction semantics: If one element of a serial conjunctive premise fails ex, r fails in c :- btinsert(a) btdelete(b) r s then b is put back in the KB and a is retracted from it

The block world in TLThe block world in TL

Initial state clauses: block(a). block(b). block(c).

on(a,b). on(b,table). on(c,table). isclear(table).

Ramification clauses: isclear(B) :- block(B), not(on(C,B)). above(B1,B3) :- on(B1,B2) ; (on(B1,B2), above(B2,B3)).

Precondition clauses: preconds(safeMove(B,L1,L2)) :- on(B,L1), isclear(B), isclear(L2).

Effect clauses: effects(safeMove(B,L1,L2)) :- btdelete(on(B,L1))

btinsert(on(B,L2)). Action execution clauses:

do(A) :- preconds(A) effects(A). Planning clauses:

achieve(Goal) :- Goal. achieve(on(B,L)) :- do(safeMove(B,_,L)).

TL x EC x SCTL x EC x SC

TL far more space and time efficient than EC and SC Because it does not keep the history of the environment It only maintains its current state But this history is available in the trace of its proof mechanism that

can be backtracked upon demand Like SC and unlike SC, TL does not explicitly represent time

points and durations With additional arguments and clauses it does concisely and

efficiently support planning and temporal projection Complex, chained ramifications require truth-maintenance

which is not supported as built-in by available TL engines