EE788 Robot Cognition and Planning, Prof. J.-H. Kim

Robot Intelligence Technology Lab.

Introduction to Cognition

This lecture material is based on the tutorials of John Taylor, David Vernon, and A.-H. Tan.

Lecture 1

2

Course Objectives

Cognition for autonomous computationally intelligent system, orbiologically-inspired cognitive systems

Autonomous computationally intelligent system

Autonomy = ability to move validly under own steam (possessed by many animals)

Possession of autonomy ≠ intelligenceEg, schizophrenics (schizophrenia (mental illness) patients), persistent criminals, ….

Add intelligence (computationally created)But dangerous wars, mayhem, suicide bombers, etc.

3

Course Objectives

HOW?

Lessons from existing solution – humans

Brain arguably subtlest ‘machine’ in universe (is conscious, not possessed by universe – insanely produces black holes)

Need emotional value or empathy (‘love’?) Need careful guidance of development of ‘conscious’ machine

with ‘guided creative fun’ (learns value of No)

As should be in human upbringing

4

Intelligence technology for cognitive systems that can

reason in a variety of ways, using substantial amounts of appropriately represented knowledge;

learn from its experiences so that its performance improves as it accumulates knowledge and experience;

explain itself and can accept direction;

be aware of its own behavior and reflect on its own capabilities; and

respond in a robust manner to surprises.

DARPA IPTO, http://www.darpa.mil/iptoIPTO (Information Processing Technology Office)

Course Objectives

5

Biologically-inspired cognitive systems Brain guidance

Look for brain principles (intelligence and emotion)

Many suggested

Propose highest level control: attention

Biased by emotion valuations of world

Internal rules developed with biasing valuations

Needs to be highly adaptive (reinforcement and error learning schemes)

Bring these to build general cognitive architecture

Course Objectives

6

Brain

Global level: The human brain A very complex system:

1011 neurons, 1014 connections Many hypercolumns and modules Subtle functionality Subtle pattern emergence High level functionality

7

Brain

Overview of brain parcellation of function

8

Current information processing systems (IPS): Incredibly fast, cheap, and scalable, and can be used

for a large number of problems humans find very difficult.

BUT: they fail that even a child does without much trouble Smooth walking, showing creativity, recognizing faces,

understanding language, etc.

Brain

9

Artificial IPS have been developed to solve “artificial problems” (AP). But now, we need to start addressing “natural problems” (NP) as well.

APs are difficult for brains because there was no necessity for the brain to find solutions for them.

NPs appear to be easy for humans because they have been solved by the brain during the evolution.

Hypothesis: Solutions to NPs are realized as structure and function of the brain

i.e. as Brain’s information processing architecture.

Brain

10

Nature of Cognition

Cognition defined as

The operation of the mind by which one becomes aware of objects of thought or perception; it includes all aspects of perceiving, thinking, and remembering

Mental functions such as the ability to think, reason, and remember

Being able to understand what’s going on around you;Being able to adapt and improvise accordingly.

High level functions carried out by the human brain, including comprehension and use of speech, visual perception and construction, calculation ability, attention (information processing), memory, and executive functions such as problem-solving, planning, and self-monitoring

Cognition complex!

11

Nature of Cognition

How to look at cognition

Reasoning, planning and self-monitoring: Crucial components of cognition

Leave out speech: animals can reason Look at nonlinguistic cognition

Approaches for reasoning, planning and self-monitoring:a) Symbolic: Logical inference on language structuresb) Probabilistic: Cognition = probabilistic inferencec) Connectionist: How can inference be obtained from NN structures

at sub-symbolic level?

Neural structures as most relevant to relate to brain processing methods

12

Complicated situations Conflicting information Missing information Incorrect information Real-time information

Need to adapt and predict

Cognition: A process by which a system achieves behaviour that is Robust Adaptive Anticipatory Autonomous

Entails embodied perception, action, and interaction

Cognition

13

A first attempt at a definition:A cognitive system produces effective – adaptive, anticipatory, and purposive goal-directed – behavior through perception, action, deliberation, communication, and through either individual or social interaction with the environment

can operate in unforeseen circumstances can view a task in more than one way can improve its performance with time might be able to explain what it is doing and why can ‘live in the future’ (cf. anticipation and prospection)

Cognition

14

Cognitive systems

Anticipate Assimilate Adapt

Predict future events when selecting actions Learn from what actually happens Modify subsequent predictions Autonomously

Learn and develop

Cognition

15

Cognitive systems:

Generate and use knowledge to

look forward in time: prediction

look backward in time: explanation

look outside time: imagination Exploring counterfactual scenarios

Cognition

16

What makes an action the right one to choose?

What type of behaviour does cognition enable?

What motivates cognition?

How is perception guided?

How are actions selected?

What makes cognition possible?

Cognitive skills can improve, but what do you need to get started?

What drives the developmental process?

Cognition

17

Pre-paradigmatic Discipline

Emerging discipline No one single established and widely-accepted paradigm But some evidence of convergence Need

An (accepted) definition of cognition A science of cognition (and a mathematical framework) Transferable technology (algorithms, HW, SW) Tools (development environments, languages, benchmarks)

18

Multidisciplinary Area

AIPsychologyNeuroscienceNon-linear dynamical systems theorySynergeticsAutonomous systems theoryMachine learningPattern recognitionComputer visionHaptic sensingCyberneticsNeural networksEpistemologyPhilosophyLanguageSemioticsRoboticsManipulationCommunication

Problems:

Different perspectivesDifferent languagesHard & soft science

Mathematical models

19

Emulation or Simulation?

What’s the goal?

Synthetic cognitive system: Emulate human capabilities

Model human cognition: Simulate human processes

The importance and relevance of biological plausibility

Inspiration vs. boundary conditions

20

Research Motivations

Two key research questions:1. How does our mind work?2. How to develop systems/agents with high-level cognitive capabilities

based on computational but biologically-plausible NNs?

Main capabilities of interests: Learning Situation awareness Reasoning Self-awareness Interactivity

21

One Approach to Cognitive Agents

Embodied cognition (Anderson, 2003)Cognition is a process deeply rooted in the body’s interaction with the world, i.e. “Intelligence through interaction”

Sense, Act, and Reward Cycle

22

Building tables for S A or (S, A) V

Classical Reinforcement Learning(Kaelbling, 1996; Sutton, 1998)

This creates a scalability problem!

23

Approach: Learn compressed mappings of S A or (S, A) V

NN for Reinforcement Learning(Ackley, 1990; Sutton 1984)

Limitations:• Slow, iterative learning: One pattern, many passes• Instability: New learning erodes previously learned knowledge

24

“How can we continue to quickly learn new thingsabout the environment and yet not forgetting

what we have already learned?

Real world facing a situation where data is continuously changing

It produces stability-plasticity dilemma

Stability-Plasticity Dilemma(Grossberg, 1976)

25

Plasticity:System adapts its behavior according to significant events Once a BP is trained, the # of neurons and the weights are fixed.

The NN cannot learn from new patterns unless it is re-trained from scratch. Thus, we consider the BP NNs don’t have plasticity.

Stability: System behavior does not change after irrelevant events The plasticity problem can be solved by retraining the NN

on the new patterns using on-line learning rule. However, it will cause the NN to forget about old knowledge rapidly. We say that such algorithm is not stable.

Dilemma: Preservation of learned knowledge

Stability-Plasticity Dilemma

(Grossberg, 1976)

26

Unsupervised Learning

Machine Learning Paradigms

Supervised Learning Reinforcement Learning

TD

27

Problems for Autonomous Machines

Problems of machines presently:

1) Scalability (many sensors, real time)

2) Context awareness (peripheral sensitivity)

3) Robustness (against damage/loss)

4) Autonomy and self-management (stay alive)

5) User adaptability (varying profiles)

6) Fast computation (rapid guidance)

28

Solution to Autonomous Machine Problems

Learn to pay attention

Attention solves:

1) Scalability: filter out distracters

2) Context awareness: attend to the important

3) Robustness: use neural perception on purpose

4) Autonomy: determine by own goal structure

5) User adaptability: by training to a user

6) Speed: by hardware nanotechnology

29

Cognitive Brain-based Machine Projects

GNOSYS ‘Cognitive Robot’ (EC IST Project Oct 2004 – ’07)

MATHESIS ‘Learning Others’ Actions’ (EC IST Project Feb 2006-’09)

‘Attending to the World’ (EPSRC Project March 2004 – 2007)

‘Analyzing Attention Emotion’ (BBSRC Project 2005-2008)

‘Modelling Emotion’ (EC HUMAINE NoE, 2005 - 2008)

30

Cognitive Brain-based Machine Projects

Focus of GNOSYS

1) Develop percepts/concepts/rewarded goals/reasoning/abstraction

2) Learn to perform goal-directed tasks

3) Learn in novel environments

4) Reasoning by forward model

5) Globally integrated system

6) Employ various memory types

(STM/LTM/iconic memory/associative memory)

7) Interdisciplinary: Computer vision/ cognitive science/ robotics/ control/ AI/ mathematics

31

GNOSYS reasoningDomains/environments Three levels of environment

Level 1: Learn shapes/colors; move and touch: 2-D objects• Powers: concept/attention/goals/actions on objects/salience of

objects in environment

Level 2: 3-D objects and actions: pick-up, stack, learn new objects• Powers: manipulate to achieve goals

Level 3: Hierarchy of objects; run virtual object/ action sequences to achieve goals

• Powers: reasoning/ novel objects/ actions

Cognitive Brain-based Machine Projects

32

The hybrid GNOSYS brain

Cognitive Brain-based Machine Projects

33

34

EC cognitive systems Focus is on research into ways of endowing artificial systems

with high-level cognitive capabilities, typically perception, understanding, learning, knowledge representation and deliberation, thus advancing enabling technologies for natural language understanding, scene interpretation, automated reasoning and problem-solving, robotics and automation, which are relevant for dealing with complex real-world systems.

It aims at systems that develop their reasoning, planning and communication faculties through grounding in interactive and collaborative environments, which are part of, or connected to the real world.

These systems are expected to exhibit appropriate degrees of autonomyand also to learn through ‘social’ interaction among themselves and/or through human-agent cooperation; in a longer term perspective, research will explore models for cognitive traits such as affection, consciousness or theory of mind.

Cognitive Brain-based Machine Projects

35

How to assess?

Ambitious, even mentions consciousness and theory of mind.

Similar ambition in other new adventures in cognitive research: Brain Sciences Institute (BSI) in Tokyo, BICA (USA) – emphasize brain basis/use of brain guidance

BSI made good progress towards its aims.

High ambition not negative reaction

But need careful assessment of projects and results

Need to realize some goals harder than initially thought, e.g. consciousness.

Cognitive Brain-based Machine Projects

36

The EC Cognitive Systems Projects

The 23 artificial cognitive systems projects:1) BACS: Bayesian Approach to Cognitive Systems2) CASBLIP: Cognitive Aid System for Blind People3) CLASS: Cognitive-Level Annotation using Latent Statistical

Structure4) COSPAL: Cognitive Systems using Perception-Action Learning5) COSY: Cognitive Systems for Cognitive Assistants6) DECISIONS-IN-MOTION: Neural Decision-Making in Motion7) DIRAC: Detection and Identification of Rare Audio-visual Cues8) eTRIMS: eTraining for Interpreting Images of Man Made Scenes9) euCOGNITION: European Network for the Advancement of

Artificial Cognitive Systems10) GNOSYS: An Abstraction Architecture for Cognitive Agents11) HERMES: Human-Expressive Representations of Motion and

their Evaluation in Sequence

37

The EC Cognitive Systems Projects

12) ICEA: Integrating Cognition, Emotion and Autonomy13) JAST: Joint-Action Science and Technology14) MACS: Multisensory Autonomous Cognitive Systems15) MATHESIS: Observational Learning in Cognitive Agents16) Mind RACES: from Reactive to Anticipatory Cognitive Embodied Systems17) PACO-PLUS: Perception, Action and Cognition through Learning of Object-

Action Complexes18) PASCAL: Pattern Analysis, Statistical Modelling and Computational Learning19) POP: Perception On Purpose20) RASCALLI: Responsive Artificial Situated Cognitive Agents Living and

Learning on the Internet21) ROBOT-CUB: Robotic Open-architecture Technology for Cognition,

Understanding and Behaviours22) SENSOPAC: SENSOrimotor structuring of Perception and Action for

emerging Cognition23) SPARK: Spatial-temporal patterns for action-oriented perception in roving

robots

38

Assessment approach Gather the projects together under several headings, emphasizing

various approaches to cognition:– Embodiment driven (# 13, 21)– Applications-driven (# 2, 5, 7, 8, 20)– Machine-intelligence driven (# 1, 3, 18)– Neural-based (# 15)– Cognitive science based (symbolic)– Hybrid (# 10)– Dynamic systems (# 23)

Various approaches need not be most effective to achieve breakthrough in creation of autonomous cognitive machine.

Need general model of human cognition to properly assess projects viability.

The EC Cognitive Systems Projects

39

Other Groups Numerous other groups: Darwin 1–N (G Edelman, La Jolla),

K Kawamura (Developmental Robotics), ATR Lab, Kyoto (M Kawato), COG Lab (R Brooks, MIT), ‘Conscious Robot’ (O Holland, Essex) + many more

Use of HDP (BEP-based) very effective in creating motor control of walking robots (Kawato/Doya)

‘Imitating robots’ important area Progress across many fronts

Video of robot under control of G-Brain, athttp://www.ics.forth.gr/gnosys/

Results in CNS group at KCL CNS websitehttp://www.kcl.ac.uk/research/cns/cns.html

Progress in modeling animal reasoning Much more complexity needed at computational level Even then may need chip implementation to attain consciousness

The EC Cognitive Systems Projects

40

Cognitive Systems Research in 2010s

“Cognitive computing systems learn and interact naturally with people to extend what either humans or machine could do on their own.”

“Cognitive computing systems get better over time as they build knowledge and learn a domain - its language and terminology, its processes and its preferred methods of interacting.”

- IBM Research, http://www.research.ibm.com/cognitive-computing

“Biologically Inspired Cognitive Architectures (BICA) are computational frameworks for building intelligent agents that are inspired from biological intelligence”

- Biologically Inspired Cognitive Architectures Society,http://www.bicasociety.org/

41

DARPA SyNAPSE Program

SyNAPSE (Systems of Neuromorphic Adaptive Plastic Scalable Electronics) is a DARPA-funded program to develop electronic neuromorphic machine technology that scales to biological levels.

The ultimate aim is to build an electronic microprocessor system that matches a mammalian brain in function, size, and power consumption.

It should recreate 10 billion neurons, 100 trillion synapses, consume one KW (same as a small electric heater), and occupy less than two liters of space.

Started in 2008 and as of January 2013 has received $102.6 million in funding. It is scheduled to run until around 2016. The project is primarily contracted to IBM and HRL who in turn subcontract to various US universities.

42

Integrated Cognitive Architectures

Hui-Qing Chong, Ah-Hwee Tan and Gee-Wah Ng., “Integrated Cognitive Architectures: A Survey,”Artificial Intelligence Review, Vol. 28, No. 2 (Published Online February 2009) 103-130.