Cognitive Control within a Parallel Distributed MultiAgent-Based Cognitive Robot Architecture: From...
-
date post
19-Dec-2015 -
Category
Documents
-
view
216 -
download
0
Transcript of Cognitive Control within a Parallel Distributed MultiAgent-Based Cognitive Robot Architecture: From...
Cognitive Control within a Parallel Distributed MultiAgent-Based Cognitive Robot Architecture:
From Body-Sensor Integration to Body-Mind Integration
New Challenge
Integration of Body and MindIntegration of Body and Mind
Cognitive Robot Cognitive Robot ††
Conscious RobotConscious Robot
† K. Kawamura, D. Noelle, et al., A Multi-Agent Approach to Self-Reflection for Cognitive Robots”, the 11th International Conference on Advanced Robotics, Coimbra, Portugal, June 30 – July 3, 2003.
MultiAgent-Based Cognitive Architecture for ISAC
Compound AgentsCompound Agents Self Agent – Representation of Robot’s SelfSelf Agent – Representation of Robot’s Self Human Agent – Representation of HumansHuman Agent – Representation of Humans
Memory StructuresMemory Structures STM – Sensory EgoSphereSTM – Sensory EgoSphere LTM – Procedural memoryLTM – Procedural memory WM – WM – under constructionunder construction
Four Key Cognitive Abilities
We assume that there are four key cognitive abilities cognitive robots must have in order to become effective members of human society:
1. Self-awareness System Health Monitoring Task Monitoring
2. Awareness of others Intention detection Mutual Spatial & Timing Control
3. Cognitive Control Executive Controller Working Memory
4. Self-reflection Ability to reflect experiences gained Ability to think ahead
Self Agent
Human Agent
Mental Experimentation Agent
Central Executive Controller
Self AgentSelf Agent with memory structures
Pronoun Agent
Description Agent
Anomaly Detection Agent (ADA)
Mental Experiment Agent (MEA)
WM
Human AgentSelf Agent
Intention Agent
Interaction Agent
STM LTM
SES
PM
PM
Central Executive Controller
Atomic Agents
Meta Level Behaviors
DM
DM
ISAC Memory Structure
SES=Sensory Ego-SpherePM= Procedural MemoryDM=Declarative Memory
STM: Sensory EgoSphere (SES)††
Purpose of the SESPurpose of the SES Spatio-temporal method for Spatio-temporal method for
storing sensory eventsstoring sensory events Integration of multiple sensory Integration of multiple sensory
modalitiesmodalities Short-term memory of eventsShort-term memory of events Egocentric, topological Egocentric, topological
mapping of locationsmapping of locations
R.A. Peters II, K.A. Hambuchen, K. Kawamura, and D.M. Wilkes, "The Sensory Ego-Sphere as a Short-Term Memory for Humanoids", Proc. IEEE-RAS Int'l. Conf. on Humanoid Robots, Waseda University, Tokyo, Nov 22-24, 2001, pp. 451-459.
SMC-STM Experiment - Object Learning from Human Pointing
ISAC is taught to reach 5 objects on a table
SMC-STM Experiment - Object Learning from Human Pointing
Data Flow for the human-guided motion generation and behavior derivation demo
SMC-STM Experiment - Object Learning from Human Pointing
Robot learns objects upon human instruction
Scans table for human pointing finger
Registers object on Sensory EgoSphere (SES)
Robot retrieves position of objects at human direction
Parses request Retrieves object information
from SES Looks and/or points to object
of interest
• Navigation based on Egocentric representations
• SES represents the current perception of the robot
• LES represents the expected state of the world
• Comparison of these provides the best estimate direction towards a desired region
LES
SES
SES- and LES-Based Navigation
K. Kawamura, R.A. Peters II, D.M. Wilkes, A.B. Koku and A. Sekman, “Towards Perception-Based Navigation using EgoSphere”, Proc. of the International Society of Optical Engineering Conference (SPIE), October 28-20, 2001.
Novel Approach: Range-free perception-based navigation
PM stores motion skills such as how to reach to a point
PM is a combination of primitive motions which are generated by several motion generation techniques
Interpolation method we are using is called the Spatio-Temporal Isomap
LTM database currently contains Primitive Memory (PM)
Long-Term Memory (LTM)
Behavior Generation and Learning
Basic behaviors are derived Basic behaviors are derived from the motion taught by from the motion taught by humanhuman
Spatio-temporal Isomap is Spatio-temporal Isomap is used to derive primitive and used to derive primitive and meta-level behaviorsmeta-level behaviors
Behaviors are stored in Behaviors are stored in Procedural Memory(PM) in Procedural Memory(PM) in Long Term Memory of robotLong Term Memory of robot
Isomap Method †† for Behavior-Based Motion Generation
†† J . B. Tenenbaum, V. de Silva, and J. C. Langford,”A global geometric framework for nonlinear dimensionality reduction”, Science, 290(5500), pp. 2319–2323, 2000.
Behavior Generation and Learning (cont’d)
Structure of LTM database including steps to generate behaviors
Verbs and Adverbs
Method of generating a wide variety of behaviors from a few exemplar motions
First used in animation community º
Similar to primitive motion modules
º C. Rose, M. F. Cohen and B. Bodenheimer, “Verbs and adverbs: multidimensional motion interpolation”, IEEE Computer Graphics and Applications, 18:5, Sept-Oct 1998, 32–40.
Verbs and Adverbs (con’t)
ISAC performing a reaching motion using the verbs and adverbs technique.
ISAC preparing to grasp Barney
Verbs and Adverbs (con’t)
Verbs General description of
motions Joint angles as
function of time Adverbs
Parameters of a verb A verb can have any
number of adverbs Adverbs are
independent of each other
Can be subjective values such as sad or happy
Working Memory (WM)
Maintains transient information that is critical for decision-making in current context
Allows one to remember a phone number - retaining it only long enough to dial it
Of particular interest are the lateral portions of the prefrontal cortex (PFC)
PFC
Schematic diagram of the visual DA model; mechanisms in this example detect shape or color to drive the desired visual response
1. PFC: http:www. driesen.com/prefrontal_cortex.htm
2. A. Cohen and U. Feintuch, “The dimensional-action system: a distinct visual system,” Common Mechanisms in Perception and Action: Attention and Performance XIX. W. Prinz and B. Hommel, Eds. New York: Oxford University Press, 2002.
Cognitive Control and the CEC
Cognitive Control in the human brain:
The anterior cingulated cortex (ACC) is responsible for executive or cognitive control in our brain
ACC helps focus attention and select actions
Central Executive Controller (under construction) executes tasks based on:
Attention system supervision Memory management Context processing Non-routine behavior
selection/control
Goal Oriented Perception-Action Mapping
STM LTMWM
CEC
Gating
PerceptionEncoding
Stimuli
ActionDecision Actuators Action
Command
Data flow
Control flow
Cognitive Experiment
Demonstrates how the CEC, STM, WM and LTM work together to carry out a cognitive task
CEC
WM
LogicalInferenceEngine
Motionbuffer
MotionInterpolator
Objectrepresentation
IntentionAgent
Arms
PM
PM
PM
LTM
LTM Manager
Feedback
Command
Human
Task planner
Stimuli
STM
SES
ContextAnalyzer
weight
Color module
Shape module
Task
Attention Network
Action
Cognitive Control and the CEC
The Central Executive Controller (CEC) , currently under construction, is responsible for the cognitive control during action decision
A basic CEC is designed to provide the following functions:
Controls task contexts via gating Allows reactive processes to process tasks whenever
possible Provides deliberation when reactive processes cannot
handle the task by themselves Allows access to self reflection when impasses arise
Cognitive Experiment -Stroop Experiment
RED BLUE
RED YELLOW
GREEN YELLOW
YELLOW RED
BLUE GREEN
GREEN BLUE
A word and color are shown, then the subject must name the color in which a word is written
Modified Stroop Experiment
RED
BLUE
BLUE
RED
Robot is told to choose a particular color/ word (e.g., red “BLUE”)
SES maintains different representations of the environment for color and word (Note: Color recognition is VERY sensitive to environmental lighting conditions)
These are held in WM after passing through the attention system
RED
BLUE
BLUE
RED
Modified Stroop Experiment (Cont’d)
“Point to red BLUE”
Send attention parameters
Attention systemfilters data
Human
CEC
Attention
Store filtered dataWM
Retrieve motionsCEC
Perform ActionCEC
Word BLUE
Color red
Point to BLUE
Self-Reflection
Self Reflection here is defined as a cognitive activity of a robot to:
Reflect on past experiences Control intentional actions
We have not yet finalized the exact specifications nor the roles of the Self Reflection Agent into a multi-agent architecture. Possible roles could include:
Reflective process may inspect the LTM database
Modify this database to generate new behaviors
Reflective Processes in ISACA Proposal
We propose to take a multi-strategy approach: reflection for an impasse and reflection on behavior.
Three proposed agents, interacting with/within the Self Agent, implement ISAC’s reflective capabilities:
The Anomaly Detection Agent (ADA), during normal operation, gathers information about the distribution of inputs and outputs for each agent
The Mental Experimentation Agent (MEA) is invoked upon impasse. It conducts a search through the space of control parameters for those which are expected to resolve the impasse
The Central Executive Controller (CEC) Agent
Knowledge Base
facts & rules facts & rulesabout the domain about reasoning processes
Working Memory
goals & intentions plans
relevant informationabout past states/potential future states
ReflectiveProcesses
Representations of the Current State
Actions to be Taken Now
Reflective Process
Concept for The Anomaly Detection Agent(ADA)
Passively record the input/output distributions of each agent, conditioned on the current intention. This enables the identification of agents operating
outside of their normal operating range.
Self Agent
ADA
ObservedAgent
Inputs Outputs
Current Intention
Concept for The Mental Experimentation Agent(MEA)
When the Self Agent recognizes an impasse, it activates the MEA
The MEA searches for control signals which will resolve the impasse
Agents are run in “simulation mode” in order to determine the likely effects of control signal changes
The search is guided to suspected agents by the ADA
MEA
SuspectedAgent
Inputs Outputs
Control
Conclusion
During the past decade, we have seen major advances in the integration of body and sensor.
The next grand challenge will be in the integration of body and mind.
An adaptive working memory, an executive controller , and neuron-base brain processing may be key to this integration.
ISAC: System Information
Interaction competencies for a personal robot
Hardware Software
OBSERVATIONAL
Presence of Person: Infrared (IR)
Passive IR motion detector array
Digital I/O
Sound: Event Localization Condenser Microphones Matlab
Speech: Detection/Recognition Handheld Microsoft Speech Recognition Engine 4.0
Vision: Face and Gesture Sony Color CCD Cameras Visual C++ routines, some with Intel Libraries
DEMONSTRATIVE / RESPONSIVE
Speech: Synthesis PC Speakers AT&T Natural Voices Engine
Motor behaviors: Head Directed Perception PTU-46-70 IMA wrapper for serial port
Motor behaviors: Arms Pneumatic Muscles Visual C++ routines and control
Intelligent Machine Architecture
Hardware or Resource Agent interfaces to sensor or actuator
hardware
Behavior or Skill Agent encapsulates basic behaviors or
skills
Environment Agent provides an abstraction for
dealing with objects in the robot’s environment
Sequencer Agent performs a sequence of
operations
Multi-Type Agent combines the functionality of at
least two of the first four agent types
RightCamera
LeftCamera
Pan-TiltHead
LeftArm
LeftHand
RightArm
RightHand
Block Beanbag
LegendHardware
AgentEnvironment
Agent
Spoon Cup
IMA is a multi-agent based robot control architecture developed at Vanderbilt. Atomic agents (e.g. hardware, behavior,environment) are the building blocks of IMA.