A system for interactive learning in dialogue with a tutor

Pedagoška konferenca FRI Ljubljana, 6 June 2013

Danijel Skočaj

University of Ljubljana

Motivation

industrial robots

SF

human

cognitive robots

communication

perception action

attention goals

planning reasoning

learning

Learning

Why learning?

It is impossible to encode all the knowledge by hand!

It is impossible to envision all the data in advance!

Gather information from the environment (and a human companion)

Learning efficiency

Learn

as good as you can

as fast as you can

as autonomously as you can

number of training samples/ tutoring cost

recognition success

Interactive learning

intr

ospective learn

ing

exstr

ospective learn

ing

Robot George

George, the curious robot

George is curious.

He wants to know!

He wants to learn!

Curiosity driven extension of knowledge.

Use the information given by the tutor!

Ask the tutor for additional information!

Learn as much as you can!

Learning of categorical knowledge

Scenario setup

Table top scenario

Extendable to more general scenarios

Learning about objects

Colour

Shape

Type

Baseline knowledge/abilities

Perceptual input

Low, mid-level vision

Situated dialogue

Ontology mediated reference resolution

System architecture: CAS (CAST)

Mixed initiative learning:

tutor driven

robot driven

System overview

Self-understanding for self-extension

Behaviours

Prioritization of motive drives

Interaction drive

Answering tutor's requests Situated tutor-driven learning

Extrospection drive

Attention mechanism Situated autonomous learning Situated tutor-assisted learning Exploring the scene

Introspection drive

Non-situated tutor-assisted learning

Priority

Behaviours

Prioritization of motive drives

Interaction drive

Answering tutor's requests Situated tutor-driven learning

Extrospection drive

Attention mechanism Situated autonomous learning Situated tutor-assisted learning Exploring the scene

Introspection drive

Non-situated tutor-assisted learning

Priority

H: Look left.

H: What colour is this object?

R: It is red./Do you mean this one?

Behaviours

Prioritization of motive drives

Interaction drive

Answering tutor's requests Situated tutor-driven learning

Extrospection drive

Attention mechanism Situated autonomous learning Situated tutor-assisted learning Exploring the scene

Introspection drive

Non-situated tutor-assisted learning

Priority

H: The elongated object is red.

R: OK, red.

H: It is not blue.

R: OK.

Behaviours

Prioritization of motive drives

Interaction drive

Answering tutor's requests Situated tutor-driven learning

Extrospection drive

Attention mechanism Situated autonomous learning Situated tutor-assisted learning Exploring the scene

Introspection drive

Non-situated tutor-assisted learning

Priority

Detect and attend the objects.

Behaviours

Prioritization of motive drives

Interaction drive

Answering tutor's requests Situated tutor-driven learning

Extrospection drive

Attention mechanism Situated autonomous learning Situated tutor-assisted learning Exploring the scene

Introspection drive

Non-situated tutor-assisted learning

Priority

R: I see that this object is green.

I will update my models.

Behaviours

Prioritization of motive drives

Interaction drive

Answering tutor's requests Situated tutor-driven learning

Extrospection drive

Attention mechanism Situated autonomous learning Situated tutor-assisted learning Exploring the scene

Introspection drive

Non-situated tutor-assisted learning

Priority

R: Is this object green?

H: Yes it is.

R: OK, green.

Behaviours

Prioritization of motive drives

Interaction drive

Answering tutor's requests Situated tutor-driven learning

Extrospection drive

Attention mechanism Situated autonomous learning Situated tutor-assisted learning Exploring the scene

Introspection drive

Non-situated tutor-assisted learning

Priority

Looking around to see if there are any

new objects.

Behaviours

Prioritization of motive drives

Interaction drive

Answering tutor's requests Situated tutor-driven learning

Extrospection drive

Attention mechanism Situated autonomous learning Situated tutor-assisted learning Exploring the scene

Introspection drive

Non-situated tutor-assisted learning

Priority

R: Can you show me a white object?

H: This object is white.

R: OK.

Video

http://cogx.eu/results/george

System diagram

Individual competencies

Plane-pop-out

object detection

and tracking

odKDE-based

learning of

object properties

Incremental

view-based learning

of object models

MLN-based

reference

resolution Motive

management

Continual

planning

Robot arm

manipulation

Belief-based

information

fusion

Abduction-based

situated language

processing

CAST-based

integration

Conclusion

T-60 T-30 T T+30