cse@buff

alo

The GLAIR Cognitive Architectureand Prospects for Consciousness

Stuart C. ShapiroDepartment of Computer Science & Engineering

and Center for Cognitive ScienceState University of New York at Buffalo

shapiro@buffalo.edu

cse@buff

alo

S. C. Shapiro

Outline

Overview

Integration of Acting and Reasoning

Symbol Grounding

Time

CSE 7192

cse@buff

alo

S. C. Shapiro

Cognitive Architecture

“A cognitive architecture specifies the underlying infrastructure for an intelligent system[, including] those aspects of a cognitive agent that are constant over time and across different application domains.”

[P. Langley, J. E. Laird, S. Rogers, Cognitive architectures: Research issues and challenges, Cognitive Systems Research 10 (2009) 141-160.]

CSE 7193

cse@buff

alo

S. C. Shapiro

Grounded Layered Architecture with Integrated Reasoning

• Major Concern:– Knowledge Representation and Reasoning

• Driving Motivation:– Natural Language Understanding & Generation

• Additional Concern:– Agents that act

• Question:– Where do beliefs come from?

• Partial Answer:– Agent’s being embodied– Agent’s being situated in the world

CSE 719 4

cse@buff

alo

S. C. Shapiro

CSE 719 5

KL

PMLa

PMLb

PMLc

SAL

Mind

BodyIndependentof lower-body

implementation

Hearing

Vision

Motion

Speech WORLD

I/P s o c k e t s

GLAIR Architecture

Dependenton lower-bodyimplementation

Proprioception

cse@buff

alo

S. C. Shapiro

Sensori-Actuator Layer

• Sensor and effector controllers

CSE 7196

cse@buff

alo

S. C. Shapiro

Perceptuo-Motor Layer

• PMLa• PMLb• PMLc

CSE 7197

cse@buff

alo

S. C. Shapiro

PMLc

• Abstracts sensors & effectors• Body’s behavioral repertoire

CSE 7198

cse@buff

alo

S. C. Shapiro

PMLb

• Translation & Communication– Between PMLa & PMLc

• Highest layer that knows body implementation

CSE 7199

cse@buff

alo

S. C. Shapiro

PMLa• Grounds KL symbols

– Perceptual structures– Implementation of primitive actions

• Registers for Embodiment & Situatedness– Deictic Registers– Modality Registers

CSE 71910

cse@buff

alo

The Knowledge Layer• Implemented in SNePS• Agent’s Beliefs• Representations of conceived of entities• Semantic Memory• Episodic Memory• Quantified & conditional beliefs• Plans for non-primitive acts• Plans to achieve goals• Beliefs re. preconditions & effects of acts• Policies: Conditions for performing acts• Self-knowledge• Meta-knowledge

CSE 719 S. C. Shapiro 11

cse@buff

alo

S. C. Shapiro

Outline

Overview

Integration of Acting and Reasoning

Symbol Grounding

Time

CSE 71912

cse@buff

alo

S. C. Shapiro

SNePS

• A KRR system• Every non-atomic expression is simultaneously

– An expression of SNePS logic– An assertional frame– A propositional graph

• Every SNePS expression is a term– Denoting a mental entity

CSE 71913

cse@buff

alo

S. C. Shapiro

Ontology of Mental Entities• Entity

– PropositionAgent can believe it or its negationIncludes quantified & conditional beliefs

– ActAgent can perform it

– PolicyCondition-act rule agent can adopt

– ThingOther entities: individuals, categories, properties, etc.

CSE 71914

cse@buff

alo

S. C. Shapiro

PoliciesReasoning Acting

• Forward Reasoningwhendo(φ, α)wheneverdo(φ, α)

• Backward Reasoningifdo(φ, α)

CSE 71915

cse@buff

alo

S. C. Shapiro

Types of Acts I• External Acts

affect the environmentsupplied by agent designer

• Mental Actsaffect the knowledge layerbelieve, disbelieveadopt, unadopt

• Control Actssequence, selection, loop, etc.

CSE 719 16

cse@buff

alo

S. C. Shapiro

Types of Acts II

• Primitive ActsImplemented in PMLa

• Composite ActsStructured by control acts

• Defined ActsDefined by ActPlan(α, p) belief

CSE 71917

cse@buff

alo

S. C. Shapiro

Acting ReasoningControl Acts

snif({if(φ1, α1), …, if(φn, αn), [else(δ)]})

sniterate({if(φ1, α1), …, if(φn, αn), [else(δ)]})

withsome(x, φ(x), α(x), [δ])

withall(x, φ(x), α(x), [δ])

CSE 71918

cse@buff

alo

S. C. Shapiro

Goal Talk

GoalPlan(φ, p)

achieve(φ)

CSE 71919

cse@buff

alo

S. C. Shapiro

Behavior CycleEnglish

(Statement, Question, Command) (Current) Set of Beliefs

(Updated) Setof Beliefs

Actions(New Belief)

English sentence expressingnew belief answering question reporting actions

Answer

NL Analysis

NL Generation

ReasoningClarification DialogueLooking in World

Reasoning

CSE 71920

cse@buff

alo

S. C. Shapiro

Outline

Overview

Integration of Acting and Reasoning

Symbol Grounding

Time

CSE 71921

cse@buff

alo

S. C. Shapiro

FEVAHR/Cassie in the Lab

CSE 719 22

cse@buff

alo

S. C. Shapiro

Entities, Terms, Symbols, Objects

• Agent’s mental entity: a person named Stu

• SNePS term: b4

• Object in world:

23BICA 2009

cse@buff

alo

S. C. Shapiro

AlignmentMind (KL)

Body (PML/SAL)

World

KL term

PML structure

Object/Phenomenon Action

CSE 719 24

cse@buff

alo

25

World Objectsto Feature Tuples

<Height, Width, Texture, . . . >

WorldPML/SAL

S. C. ShapiroCSE 719

cse@buff

alo

S. C. Shapiro

Feature Tuplesto KL Terms

<Height, Width, Texture, . . . >

PML/SALKL

ProperName(b4, Stu)

Alignment

CSE 71926

cse@buff

alo

S. C. Shapiro

Incomplete PML-Descriptions

<Height, nil, nil, . . . >

PML/SALKL

Height(b4, b12)

CSE 71927

cse@buff

alo

S. C. Shapiro

Unifying PML-Descriptions

<x, nil, nil, ... >

PML/SALKL

b20

b30

b31

b6

<nil, y, nil, ... >

<nil, nil, z, ... >

<x, y, z, ... >

Isa

Prop

Prop

28CSE 719

cse@buff

alo

S. C. Shapiro

Deictic Registers

For being situated in the worldPML registers hold KL terms

I term denoting agentYOU term denoting dialogue partnerNOW term denoting current time

CSE 71929

cse@buff

alo

S. C. Shapiro

Modality Registers

For privileged first-person knowledge of what agent is doing

Register for each modality holdsKL term denoting act modality is engaged in

CSE 71930

cse@buff

alo

S. C. Shapiro

Acting 1

CSE 719 31

cse@buff

alo

S. C. Shapiro

Acting 2

I found a red robot.I am looking at a red robot.

Follow a red robot.

CSE 719 32

cse@buff

alo

S. C. Shapiro

Acting 3

I went to a red robot.I am near a red robot.I am following a red robot.

I found a red robot.I am looking at a red robot.

Follow a red robot.

CSE 719 33

cse@buff

alo

S. C. Shapiro

Use of Indexicals 1

Come here.

I found you.I am looking at you.

CSE 719 34

cse@buff

alo

S. C. Shapiro

Come here.

I came to you, Stu.I am near you.

Use of Indexicals 2

I found you.I am looking at you.

CSE 719 35

cse@buff

alo

S. C. Shapiro

Who am I?

Your name is ‘Stu’and you are a person.

Who have you talked to?

I am talking to you.

Talk to Bill.

I am talking to you, Bill.

Come here.

Use of Indexicals 3

CSE 719 36

cse@buff

alo

S. C. Shapiro

Come here.

I found you.I am looking at you.

Use of Indexicals 4

CSE 719 37

cse@buff

alo

S. C. Shapiro

Come here.

I came to you.I am near you.

I found you.I am looking at you.

Use of Indexicals 5

CSE 719 38

cse@buff

alo

S. C. Shapiro

Who am I?

I talked to Stuand I am talking to you.

Your name is ‘Bill’and you are a person.

Who are you?

I am the FEVAHRand my name is ‘Cassie’.

Who have you talked to?

Use of Indexicals 6

CSE 719 39

cse@buff

alo

S. C. Shapiro

<green, nil>

PML/SALKL

b20

b30

b6

<nil, robotic>

Isa

Prop

40CSE 719

m2

Find a green robot.

(find )

cassie

m75

m76

robbie

<green, robotic>

VISION

Language-Mind-World-Mind

WORLD

cse@buff

alo

S. C. Shapiro

Outline

Overview

Integration of Acting and Reasoning

Symbol Grounding

Time

CSE 71941

cse@buff

alo

S. C. Shapiro

B6

Representation of Time

find

lex

action object

B1

!

agentact

eventtime

NOW

!!before after before after

????

????

????

?

I

CSE 71942

cse@buff

alo

Building Episodic Memory

CSE 719 S. C. Shapiro43

KL

PML

e1

I

a1 b1

!

t1 e2

a2

!

t2

!

NOW

COUNT n

hom0

q!

before afterevent eventtime

time

time

act act

agent

agent

duration

ACT

cse@buff

alo

S. C. Shapiro

Performing a Punctual Actt1 t3!

before after

NOW NOW

CSE 71944

t2!before after

!

time

event

cse@buff

alo

S. C. Shapiro

Performing a Durative Actt1

NOW

t3!

supintsubint

CSE 71945

NOW

!before after

t2

!

time

event

cse@buff

alo

S. C. Shapiro

Aligning NOW using MTF

NOW

Semi-lattice of times, all of which contain *NOW,any of which could be meant by “now”Finite---only conceptualized times of conceptualized states

Maximal Temporal Frame based on *NOW

KL

PML

CSE 71946

cse@buff

alo

S. C. Shapiro

Moving NOW with MTF

NOW

KL

PML

Moves when Cassie acts, newly observes a state,or is informed of a new state. Always includes times of states in modality registers.

CSE 71947

cse@buff

alo

S. C. ShapiroCSE 71948

Collaborators

Past and present members ofSNeRG: The SNePS Research Grouphttp://www.cse.buffalo.edu/sneps/