Background slides for invited challenge demo at Rule-ML 2017

A software agent controlling 2 robot arms in co-operating concurrent tasks

Robotic agent programming in QuLog and TeleoR

Keith Clark Imperial College & University of Queensland

Joint work with Peter Robinson

University of Queensland

1

QuLog - LP+FP+AR Language •  Flexibly typed, multi-threaded, higher order •  Relation and function defining rules

•  Function calls and set expression arguments in relation calls •  Function rule guards can query relations •  Relation rules more declarative than Prolog, closer to predicate

logic •  Relations are typed and moded

•  Dynamic relations defined only by facts •  Used for the agent’s Belief Store

•  Top layer of action rules •  Update dynamic relations, fork threads, do I/O, communicate with

other Qulog, C, Python or Java processes •  Inter-process comms uses our Pedro pub/sub and addressed

message router •  Threads execute actions calling funs and rels as needed •  Rel and fun rules cannot call actions

2

TeleoR •  Development of Nilsson’s T-R robotic agent language of guard ~> action rules sequenced in parameterized procedures. •  guard is a QuLog deductive query to the BS •  action is a tuple of robotic actions executed in parallel or

a T-R procedure call, maybe a recursive call •  TeleoR has forms of rules and actions not in T-R

–  Extra rules for more fine grained behaviour control –  Can combine robotic action with QuLog action sequence: action ++ qact1 ;...; qact1 –  This allows communication and updates of BS as parallel non-robotic actions

•  Compile time guarantee: –  all rule actions are correctly typed and fully instantiated when a rule is fired

•  The concept of task atomic procedures: –  Used for multi-tasking sharing one or more robotic resources –  No interference, no starvation, deadlock free –  Compiler generates code that uses BS for co-ordination 3

Deductive Belief Store Dynamic facts

Relation and function rules

? Percepts Handler

Message Handler ag@host

Percept fact Interpretations of Sensor data

Mid-level Action control messages

Incoming Messages and replies

Control Thread

?Atomic re-evaluation of rule guards after each atomic update

Atomic Updates

Multi-threaded Agent Architecture

Only outgoing Messages

4 pedro Other Agents

Pub/Sub and Addressed Message Router

TeleoR + QuLog

QuLog

One arm block tower building

5

Task Ontology

6

def block::= 1..9

durative pickup(block), put_on_block(block), put_on_table()

percept on(block,block), on_table(block), holding(block)

rel sub_tower(list(block)), tower(list(block))

sub_tower([B]) <= on_table(B) sub_tower([B1,B2,..Blocks]) <= on(B1,B2) & sub_tower([B2,..Blocks])

tower([B,..Blocks]) <= not on(_,B) & sub_tower([B,..Blocks])

Nilsson’s universal conditional top-level plan for block tower building

7

tel makeTower(list(block))

makeTower(Blocks){

tower(Blocks) ~> () % Goal is achieved

sub_tower(Blocks) & Blocks=[B,..] ~> make_clear(B)

Blocks=[B] ~> move_to_table(B)

Blocks=[B1,B2,..Bs] & tower([B2,..Bs]) ~> move_to_block(B1,B2)

Blocks=[B,..Bs] ~> makeTower(Bs) }

Action achieves

Mutually recursive procedures

8

tel make_clear(block) make_clear(Block){

not on(_,Block) ~> () % Goal is achieved

on(OnBlock,Block) ~> move_to_table(OnBlock) } tel move_to_table(block) move_to_table(OnBlock){

on_table(OnBlock) ~> () % Goal is achieved

holding(OnBlock) ~> put_on_table()

not on(_, OnBlock) & not holding(_) ~> pickup(OnBlock)

not holding(_) ~> make_clear(OnBlock)

holding(_) ~> put_on_table() }

Interleaving the building of several towers

9

Just need to add:

task_start makeTower

task_atomic move_to_table, move_to_block

Baxter concurrent tower building

10

TR procedures

Sensor data

Control actions for different robotic resources

BeliefStore

Dynamic Facts

Percepts Handler

TR Evaluation Threads All

incoming messages

Message Handler Task1

Using R1,R2

Task2 Using

R3

Outgoing messages

Multi-tasking architecture

Task3 Waiting for

R1,R3

Task4 Waiting for

R1,R5 Fixed Facts & Rules

?

Elements of the two arm tower builder program

12

def table ::= table1 | shared | table2 def block ::= ... def arm ::= arm1 | arm2 def resource::= arm || table % resource a reserved type name

durative pickup(arm, block, table), put_on_block(arm, block, table), put_on_table(arm, table)

tel makeTower(arm, list(block), table) makeTower(Arm,Blks,Tbl){ % Tbl is where tower must be built, tower(Blks, Tbl) ~> {} sub_tower(Blks, Tbl) & Blocks=[B,..] ~>

makeClear(Arm, B, Tbl) Blks=[B,.. Bs] & tower(Bs, Tbl) ~>

moveAcrossToBlock(Arm, B, firstOf(Bs), Tbl) Blks=[B] ~> moveAcrossToTable(Arm, B, Tbl) Blks=[B,.. Bs] ~> makeTower(Arm, Bs, Tbl) }

Entire control program 40 TeleoR rules clustered into 6 procedures.

Auxiliary procedures

13

tel moveAcrossToTable(arm, block, table) % Will fetch a block from wherever it is and put it onto the table % May need two calls to proc below using different arms to first % move the block to shared then to destination table

task_atomic oneArmMoveToTable(arm, block, table, table) % Has 3 resource args. an arm and two tables which might be the same oneArmMoveToTable(Arm, Blk, FromTab, ToTab){

on(Blk, ToTab) ~> ()

clear(Blk) ~> clearOneArmMoveToTable(Arm, Blk, FromTab, ToTab)

not holding(Arm,_) ~> makeClear(Arm, Blk, FromTab)

holding(Arm,_) ~> put_on_table(Arm, FromTab) }

TeleoR semantics and implementation

14

•  Formal State Transition Semantics •  Optimized Reference Implementation

•  Runtime system that implements state transition semantics •  Compile time analysis ensures rule guards are not re-evaluated on BS update if no relevant change made

•  Currently compiled to multi-threaded Qu-Prolog •  Will soon be compiled to specialized Abstract Machine Code similar to but simpler than Warren’s Prolog AMC

Sources and software

15

Clark & Robinson, Robotic Agent Programming in TeleoR, ICRA 2015, IEEE

Clark & Robinson, Multi-tasking Robotic Agent Programming in TeleoR, Research Report, on www.doc.ic.ac.uk/~klc

Clark et al, A Framework for Integrating Symbolic and Sub-symbolic Representations, IJCAI 2016, AAAI Press

Programming Communicating Multi-tasking Robotic Agents: A Teleo-Reactive Rule Based Approach, Springer, Early 2018 first 5 chapters at teleoreactiveprograms.net including a formal operational semantics of Nilsson’s TR lang.

Clark & Robinson, Engineering Agent Applications in QuLog, Springer, 2017/8,

TeleoR and QuLog Software for Unix, Linux and OS X at http://staff.itee.uq.edu.au/pjr/HomePages/QulogHome.html

Collaboration and users welcomed