TOWARDS COGNITION IN ROBOTICS: ACHIEVEMENTS AND … · 8 The need For Cognition to learn from...

79
1 TOWARDS COGNITION IN ROBOTICS: ACHIEVEMENTS AND CHALLENGES Fakhri Karray Pattern Analysis Machine Intelligence Research Lab University of Waterloo, Canada http://pami.uwaterloo.ca AIS’10, Povoa de Varzim, Portugal

Transcript of TOWARDS COGNITION IN ROBOTICS: ACHIEVEMENTS AND … · 8 The need For Cognition to learn from...

  • 1

    TOWARDS COGNITION IN ROBOTICS:

    ACHIEVEMENTS AND CHALLENGES

    Fakhri KarrayPattern Analysis Machine Intelligence

    Research LabUniversity of Waterloo, Canada

    http://pami.uwaterloo.ca

    AIS’10, Povoa de Varzim, Portugal

  • Adam is first robot to ever complete a reasoning cycle from hypothesis to experiment to reformulating the hypothesis without human intervention

    (One of Time’s Top 10 Discoveries of 2009)

  • 3

    IEEE Spectrum Online March 12, 2009 (reporting on Boston Globe article of March 4, 2009)

  • 4

  • 5

    Outline

    Part1 : Towards Cognition in Robotics

    Part2 : Major Aspects of Cognitive Robotics

    Part3 : Recent Advances

    Part4 : Applications

    Part5 : Opportunities /Challenges

  • 6

    Toward Cognition in Robotics

    Part2 : Major Aspects of Cognitive RoboticsPart3 : Recent AdvancesPart4 : ApplicationsPart5 : Challenges

    Part 1:

  • 7

    Evolutionary Stages

    IndustrialRobotics

    Service Robots forPersonal Use

    Personal/socialRobotics

    ServiceRobotics

    Evolution of Robotics What is Cognition?

    Service Robots for Professional Use

    Toward Cognition in Robotics

  • 8

    The need For Cognition

    to learn from experience and apply the learned knowledge to deal with dynamically changing environments

    to ‘understand’ human and naturally communicate with them

    to reason and self-reflect to take new task initiatives

    We expect next generation of robots to have the following capabilities :

    Evolution of Robotics What is Cognition?

    Toward Cognition in Robotics

  • 9

    These skills emerge in human as a result of complex, bidirectional interaction among body, brain, and environment

    Self-awareness

    Perception

    Learning

    Cognitive robots are robots imparted with ‘human-like’ cognitive abilities

    Major Properties in Human Cognition

    Evolution of Robotics What is Cognition?

    Knowledge

    Reasoning

    Planning and decision making

    Toward Cognition in Robotics

  • 10

    Psychology

    Neuro-science

    Biology

    Cognition

    Understanding of human cognition involves three major disciplines

    Toward Cognition in RoboticsEvolution of Robotics What is Cognition?

  • 11

    Understanding of cognitive robotics involves :

    Psychology

    Neuro-science

    Biology

    Computer/Software Engineering

    Mechanical/ Mechatronics Engineering

    AI/ Computer Science

    Systems Engineering

    Toward Cognition in RoboticsEvolution of Robotics What is Cognition?

    Cognitive Robotics

  • 12

    Cognitive Robotics

    Toward Cognition in RoboticsEvolution of Robotics What is Cognition?

    Processing Units

    Machines Units

    1

    1

    10s

    100s

    100s10s

    SI

    DAI

    AI Robotics Centralized Control

    Multiple MachinesMachine

    MultiagentDistributed

    Robot System

    SwarmRobotics

    MEMS-based Multiple Machines

  • 13

    Major properties required for cognition in Robotics

    Physical embodiment

    Social situatedness

    Role of experience

    Self-awareness

    Value system

    Toward Cognition in RoboticsEvolution of Robotics What is Cognition?

  • 14

    Cognition Pathways

    Perception pathway

    Reasoning pathway

    Planning pathway

    Action pathway

    Sens

    ors

    Actu

    ator

    s

    Mem

    ory

    Evolution of Robotics What is Cognition?

    Toward Cognition in Robotics

  • 15

    Major Aspects of Cognitive Robotics

    Part3 : Recent AdvancesPart4 : ApplicationsPart5 : Challenges

    Part1 : Toward Cognition in Robotics

    Part2 :

  • 16

    Major Aspects of Cognitive Robotics

    The major aspects in cognitive robotics research deal with

    Various proposed theories of cognition/architectures

    Essential building blocks for achieving cognitive systems

  • 17

    Major Aspects of Cognitive Robotics

    The research on robot cognition is still in its infancy. A complete and unified theory for robot cognition is not yet developed

    Group 1 Theory of Robot cognition

    Group 2 Discrete Cognitive Abilities

    Main lines of current research

    Components TheoriesLine of research

  • 18

    Major Aspects of Cognitive Robotics

    Group 1 General theories for robot cognition

    Design Principle Architectural issues Cognitive developmental robotics Incremental intelligence

    Implementation Issues Developmental engineering Software/hardware requirements

    Social factors Human/Machine interaction Autonomous learning development

    Components TheoriesLine of research

  • 19

    Major Aspects of Cognitive Robotics

    Group 2 : Discrete cognitive skills

    Visual attention Theoretical models Robotic models Task-specific models

    Social cognition Joint attention Social imitation

    Value system

    Components TheoriesLine of research

  • 20

    Computational perception

    Knowledge Representation

    Learning

    Reasoning and planning

    Interaction and communication

    The major building blocks of Cognitive Robots involve the following components :

    Components Theories

    Major Aspects of Cognitive RoboticsLine of research

  • 21

    2- Knowledge Representation

    1- Computational perception

    3-Learning4-Reasoning and planning

    5- Interaction and

    communication

    Major Aspects of Cognitive RoboticsComponents TheoriesLine of research

    Perception in cognitive robotics : allows robot to receive multi-sensory input from external

    environment

    is crucial in realizing self-awareness capability

    is connected to all other components

    Attention is the main mechanism required in this process

  • 22

    Knowledge representation: allows ease of information storage and retrieval

    performs inference to obtain new information from learned data

    The robot deploys mechanisms such as self-organization, self-production, and self-maintenance to autonomously develop its own representation with interaction and experience.

    2- Knowledge Representation

    1- Computational perception

    3-Learning4-Reasoning and planning

    5- Interaction and

    communication

    Major Aspects of Cognitive RoboticsComponents TheoriesLine of research

  • 23

    Construct new skills, knowledge and capabilities through: Exploration: e.g. reaching, grasping, and manipulating what is

    around it.

    Social Interaction: which involves interacting with the user in the learning process.

    Observation and Imitation: can be imitation of low level features such as joint trajectories, or higher level features such us complete actions, and behaviors.

    2- Knowledge Representation

    1- Computational perception

    3-Learning4-Reasoning and planning

    5- Interaction and

    communication

    Major Aspects of Cognitive RoboticsComponents TheoriesLine of research

  • 24

    Consists of a set of three simultaneous processes: Reactive processes: they mimic the reflexive behavior observed in

    biological systems.

    Deliberative processes: they are in charge of realizing what is commonly known as thinking in biological systems (e.g. Motion planning).

    Reflective planning: they involve high-level reasoning where robot computes how to perform a given task based on its cognitive abilities, learned knowledge, and embodiments.

    2- Knowledge Representation

    1- Computational perception

    3-Learning4-Reasoning and planning

    5- Interaction and

    communication

    Major Aspects of Cognitive RoboticsComponents TheoriesLine of research

  • 25

    2- Knowledge Representation

    1- Computational perception

    3-Learning4-Reasoning and planning

    5- Interaction and

    communication

    Major Aspects of Cognitive Robotics

    The human role and the level of interaction will vary but the human is still part of the system.

    CooperaTIoN

    Components TheoriesLine of research

  • 26

    Multimodality

    Adaptivity

    Multimodal Interaction

    MultimodalModules

    Social Interaction

    Multimodal&

    AdpativityModules

    Adaptive Interaction

    AdpativityModules

    Direct Manipulation Interaction

    WIMP-basedGUIs

    S R

    /R I

    S R/R I: Service Request/Response Interface

    S R

    /R I

    S R

    /R I

    S R

    /R I

    S R

    /R I

    S R

    /R I

    S R

    /R I

    S R

    /R I

    Alaa Khamis and Mohamed Kamel, and Miguel A. Salichs, “Human-Robot Interfaces for Social Interaction”, International Journal of Robotics and Automation, vol. 22, (206), 2007.

    2- Knowledge Representation

    1- Computational perception

    3-Learning4-Reasoning and planning

    5- Interaction and

    communication

    Major Aspects of Cognitive RoboticsComponents TheoriesLine of research

  • 27

    Major Theories on Cognitive Robotics

    Major Aspects of Cognitive RoboticsComponents TheoriesLine of research

    D e v e l o p m e n t i n r o b o t c o g n i t i on

    1990 20052000

    Cognitive Developmental RoboticsAsada 2001, 2007, 2009

    Autonomous Mental DevelopmentWeng 2000

    Conscious MachineKawamura 2005

    Incremental IntelligenceBrooks 1991

    Theory of developmentPiaget 1953

    1950

    Brain Based DevicesKrichmar 1998

    Developmental EngineeringSandini 1997

    2007

    Confabulation TheoryHecht-Nielsen 2005, 2007

    2009

  • 28

    Incremental Intelligence

    Developmental Engineering

    Brain-based Device

    Autonomous Mental Development

    Cognitive Developmental Robotics

    First introduced by Brooks, 1991.

    The main implications of this theory are requirement for :

    developmental (incremental) nature of intelligence,

    necessity of social situatedness and communication,

    and robot capability to integrate multi-modality sensory data.

    Major Aspects of Cognitive RoboticsComponents TheoriesLine of research

    Conscious Machine

  • 29

    First introduced by Sandini, et al. 1997.

    DE has emphasis on four basic principles:

    physical embodiment,

    social interaction,

    development and experience, and

    dynamic of development.

    Major Aspects of Cognitive Robotics

    Incremental Intelligence

    Developmental Engineering

    Brain-based Device

    Autonomous Mental Development

    Cognitive Developmental Robotics

    Components TheoriesLine of research

    Conscious Machine

  • 30

    First introduced by Krichmar et al., 2003 as a platform to test computational models of nervous systems.

    BBD devices use authentic simulation of the primate nervous systems to realize their cognitive abilities such as reasoning and planning.

    Major Aspects of Cognitive Robotics

    Incremental Intelligence

    Developmental Engineering

    Brain-based Device

    Autonomous Mental Development

    Cognitive Developmental Robotics

    Components TheoriesLine of research

    Conscious Machine

  • 31

    Was introduced by Weng et al., 2000

    AMD requires the cognitive robot to be designed in a task non-specific manner assuming that robot is capable of developing the task specification program by itself.

    The main principles of AMD are

    embodiment,

    self-awareness,

    self effectiveness,

    and developmental program.

    Major Aspects of Cognitive Robotics

    Incremental Intelligence

    Developmental Engineering

    Brain-based Device

    Autonomous Mental Development

    Cognitive Developmental Robotics

    Components TheoriesLine of research

    Conscious Machine

  • 32

    Was introduced by Asada et al., 2001

    CDR argues for the existence of a built-in architecture and puts emphasis on the role of social interaction on autonomous development of representation.

    CDR focuses on three main issues, embodiment in concert with Books theory, embedded structure (built-in architecture), and interaction design.

    Major Aspects of Cognitive Robotics

    Incremental Intelligence

    Developmental Engineering

    Brain-based Device

    Autonomous Mental Development

    Cognitive Developmental Robotics

    Components TheoriesLine of research

    Conscious Machine

  • 33

    Incremental Intelligence

    Developmental Engineering

    Brain-based Device

    Autonomous Mental Development

    Cognitive Developmental Robotics

    Conscious Machine

    Was introduced by Kawamura et al., 2005

    CM essentially provides a method of implementation for cognitive skills inspired by findings in cognitive psychology.

    Authors of CM restrict their work into emotion and attention-based control of behavior in designing consciousness.

    CM takes a multi-agent and behavior based methodology with each (software) agent dedicated to a specific cognitive ability and maintains coordination among these agents using a built-in central architecture called intelligent machine architecture.

    Part 3: Aspects, Components and Architectures of Cognitive RoboticsComponents TheoriesLine of research

  • 34

    Recent Advances

    Part4 : ApplicationsPart5 : Challenges

    Part1 : Toward Cognition in RoboticsPart2 : Major Aspects of Cognitive Robotics

    Part3 :

  • 35

    Design of Confabulation Architecture

    Novel design of discrete cognitive skills

    Intelligent Human Machine Interaction

    Besides the theories proposed, some recent advances in Cognitive robotics were made in :

    Recent Advances

  • 36

    Confabulation Theory First development made by Hecht-Nielsen in 2004-2005

    claimed to be the first comprehensive theory to describe cognition in humans and animals.

    postulates that cognition constitutes four fundamental elements: Mental object representation: about 4000 thalamocortical modules

    each comprising 10,000 to 100,000s of symbols Knowledge links: unidirectional association between two symbols Confabulation: the symbol with maximum total input excitation is

    activated Action command origination: confabulation conclusion launches a

    set of pre-associated action commands

    Recent AdvancesConfabulation Architecture Discreet cognitive skills

  • 37

    Confabulation Theory provides a hypothesis and neuronal model for acquiring, storing and using knowledge

    Recent AdvancesConfabulation Architecture Discreet cognitive skills

    An apple object and some of the attributesKnowledge link linking word apple to color

  • 38

    Recent Advances

    Design of Discrete Cognitive Skills

    Visual attention

    Social cognition

    Joint attention

    Social imitation

    Multimodal Interaction/Multimodal attention

    Value system

    Confabulation Architecture Discrete cognitive skills

  • 39

    Recent Advances

    Design of Discrete Cognitive SkillsVisual Attention

    Visual attention is a fundamental building block of cognitive

    development.

    It allows robots to focus their processing power only on the

    behaviorally relevant information and therefore facilitates their

    interaction with humans and surrounding environment

    Confabulation Architecture Discrete cognitive skills

  • 40

    Recent Advances

    Design of Discrete Cognitive SkillsVisual Attention (Recent works)

    Recent works involved a bioinspired probabilistic model of visual attention of cognitive robots.

    Work uses Bayesian analysis to recursively estimate the orientation of the camera head of a robot such as a visually salient/behaviorally relevant stimuli resides at the center of the visual field

    Observe the postulates of Biased Competitive hypothesis (a famous neurodynamic theory of the primates visual attention) for state transition

    Confabulation Architecture Discrete cognitive skills

  • Visual Attention Model for robotic systems

    41

    Recursively estimate the head pose which enables a robot to focus on the most salient or behaviorally relevant object in the environment.

    The Proposed Model

  • 42

    Recent Advances

    Design of Discrete Cognitive SkillsProposed Model

    Confabulation Architecture Discrete cognitive skills

    Functional description of the visual attention model

  • Visual Attention Model for robotic systems

    43

    Bottom-up Competition model: Assigns high probability for head poses which let the robot to focus on the visually salient objects.

    Top-down Modulation model: Modulates the probability assigned by the Bottom-up competition model based on relevance of different objects with the current behavioral requirement of the robot.

    The Proposed Model

  • Visual Attention Model for robotic systems

    44

    Implementation of the proposed model for exploration and search

    Keynote1.wmv

  • Visual Attention Model for robotic systems

    45

    We propose to introduce an additional modality, natural speech, and occasional interaction with human. The speech input from human partner modulates the behavioral requirement of the robot in a top-down manner. Thus the sensor measurement become multi-modal.

    The Proposed Model in MultiModal

  • Visual Attention Model for robotic systems

    46

    Flow of information in case of auditory modulation of visual attention

  • Visual Attention Model for robotic systems

    47

    Implementation of the proposed model with multimodal sensory measurements

  • 49

    Recent Advances

    Design of Discrete Cognitive SkillsSocial Cognition

    Multimodal interaction: allow users to move seamlessly between different modes of interaction, from visual to voice to touch, according to changes in context or user preference.

    Method facilitating social interaction between robots and humans, divided into three categories of research :

    Confabulation Architecture Discrete cognitive skills

    Joint attention: having this ability enables robot to attend to an object of mutual interest intentionally.

    Social Imitation: powerful methods of learning and developing social behavior for cognitive robots.

  • 50

    Example task in hand: any particular arm motion of the human user

    Toward Cognition in RoboticsConfabulation Architecture Discrete cognitive skills

  • 51

    Recent Advances

    Design of Discrete Cognitive SkillsSocial Cognition – Joint Attention

    Confabulation Architecture Discrete cognitive skills

  • 52

    Recent Advances

    Design of Discrete Cognitive SkillsSocial Cognition – Social Imitation

    Confabulation Architecture Discrete cognitive skills

  • 53

    Recent Advances

    Toward a Comprehensive DataFusion Architecture

    Design of Discrete Cognitive SkillsConfabulation Architecture Discrete cognitive skills

    a case study: multimodal Interaction

  • 54

    Recent Advances

    Design of Discrete Cognitive SkillsValue System

    Robot’s capability to plan action perception of salient stimuli, which makes it an essential requirement of developing human-like intelligence in robotic systems, mostly through motivation

    PredictionTask non specificDevelopmentalValue based learning

    Characteristics:

    Confabulation Architecture Discrete cognitive skills

  • 55

    Recent Advances

    Design of Discrete Cognitive SkillsValue System

    Confabulation Architecture Discrete cognitive skills

  • 56

    Towards a Generalized Performance Metric for HRI

    Emergence of new cognitive based systems which require high level of interaction and communication between humans and robots.

    Lack of a generalized set of metrics that can span much of the HRI application space.

    Lack of empirical and mathematical representation models for human-machine team performance assessment.

    Motivation

    Shortcomings of Current Performance Metrics

    Tend to focus on one agent and ignore the capabilities of the other.

    Are usually very task specific and don’t translate well to other applications.

    May not give insights on how the task is being accomplished.

    May not be extendable to multi-robots scenarios.

  • 57

    Towards a Generalized Performance Metric for HRI

    Developing a common toolkit has been discussed by Olsen and Goodrich, who proposed six interrelated metrics, most important of which are:

    Robot Attention Demand (RAD) is a measure of how much time of total task time must be spent by the user interacting with the robot.

    Fan-out (FO), which is a measure of many robots with similar capabilities the user can operate simultaneously and effectively.

    Common Toolkit for HMI

    Drawbacks

    Two essential human related criteria are ignored: Human trust in automation, and human reliability.

    Metrics do not generalize to accommodate multi-robot scenarios.

  • 58

    Towards a Generalized Performance Metric for HRI

    FO is augmented to include a representative index for human reliability.

    RAD is divided as: direct interaction time (DIT), and indirect interaction time (IIT)

    Where NT is the time in which the robot is being ignored.

    Proposed Augmented Metric

    Proposed trust Model

    Two-level fuzzy temporal model is proposed. Level II is a finite fuzzy state machine that relates trust to some second order perceptions.

    Level I infers second order perceptions from first order ones, e.g. fault sized is inferred from fault frequency, cruciality, and recovery.

  • 59

    Towards a Generalized Performance Metric for HMI

    Three membership functions are used to model first and second order perceptions.

    Five states are used to model the human trust in automation factor.

    Max-Min rule of inference is used to infer the states activation levels.

    Proposed trust model

    State activation levels are defuzzified using Sugeno-Like Zero order consequences (0.1, 0.3, 0.5, 0.7, 0.9).

  • 60

    Towards a Generalized Performance Metric for HMI

    Sequential Scenario: Only one robot is active at a time. Idle robots are assigned zero FO. System FO is calculated as:

    Parallel and Task Independent Scenario: Multiple robots are active simultaneously, executing independent tasks.

    wi is the percent contribution of the ith robot toward the final goal completion.

    Generalization to Multi-robot Scenarios

  • 61

    Towards a Generalized Performance Metric for HMI

    Parallel and task dependent Scenario: Starting with 2 robots that are active simultaneously, executing independent tasks.

    System FO falls between the weighted average and the minimum FO.

    Where d is the percent task dependency between the two robots.

    Generalizing this results to N-Robots, pair-wise dependencies are considered. An upper bound to practical system FO is calculated as:

    Generalization to Multi-robot Scenarios

  • 62

    Towards a Generalized Performance Metric for HMI

    Some Experimental Results

  • 63

    ApplicationsPart5 : Challenges

    Part1 : Toward Cognition in RoboticsPart2 : Major Aspects of Cognitive RoboticsPart3 : Recent Advances

    Part4 :

  • 64

    Applications

    Health-care and Assistive robotics

    Entertainment robotics

    Military and Space exploration

    Humanoid/social robotics

  • 65

    Applications

    Trends in percentage of the elderly (over age 85) in the world

    Source: Adriana Tapus, Maha J. Mataric, and Brian Scassellati, “Socially Assistive Robotics: The Grand Challenges in Helping Humans Through Social Interaction,” IEEE Robotics & Automation Magazine, MARCH 2007

    Health Care and Assistive Robotics

  • 66

    Applications

    Domo (MIT robot): A household assistant for the elderly or wheelchair-bound. It grasps objects and places them on shelves or counters.

    ASIBOT: A portable assistive robot for elderly and disease people bringing more freedom in daily tasks as eating, drinking, shaving, make up wearing, tooth cleaning, etc.

    Huggable: A robotic companion for health care, education, and social communication.

    Health Care and Assistive Robotics

  • 67

    Applications

    Sparky project on AIBO platform: alleviating loneliness and causing to form attachments for nursing home residents.

    CiceRobot: A museum guide cognitive robot.

    Soccer robot

    Flutist robot: The first member of a humanoid robot orchestra.

    Entertainment Robotics

    Video : Violinist Robot

  • 68

    Applications

    Robonaut: A robotic system that can function as an EVA astronaut equivalent.

    BEAR: Designed to find, pick up and rescue people in harm's way.

    Military and Space Exploration

    Video : Bear Robot

  • 69

    Applications

    Maggie: research platform to study human robot interaction and robot intelligence and autonomy.

    Humanoid/social robotics

    Video : Partner/ Cooperative Dancing with Maggie CB2: child robot with

    biomimetic body. It is able to develop a behavior similar to a 1 or 2 years old baby.

    PresenterPresentation NotesDuring the CampusPartyTM 2005, Maggie has demonstrated its ability for closely cooperative dancing with humans. In this interaction scenario, Maggie andher partner stay together for the duration of the dance. During this duration, Maggie changes its movements as respond for events detected by tactile sensors as results ofpartner touching.

  • 70

    Challenges

    Part1 : Toward Cognition in RoboticsPart2 : Major Aspects of Cognitive RoboticsPart3 : Recent AdvancesPart4 : Applications

    Part5 :

  • 71

    Major progress made in understanding human cognition, but no

    complete theory exists yet on how biological system naturally

    develop learning functions

    Even if a theoretical model is provided, it is not straightfoward to

    implement a working system on machines/robots

    Need for performance metric to evaluate the cognitive abilities in a

    robotic system. Difficulty in providing objective measures

    Issue I: Need for Metrics and Evaluation Measure for Cognition

    Challenges

  • 72

    Need for a cross-disciplinary collaborative work to integrate the

    most understood aspects of human cognition in order to come up

    with a comprehensive theory for robot cognition.

    Lack of comprehensive architecture on which to impart cognitive

    skills for robots

    Robustness and adaptability of robot cognition from one

    environment to the next.

    Difficulty in dealing with uncertainty in perception

    Issue II: Integration of cross-disciplinary knowledge

    Challenges

  • 73

    Tighter cooperation involving roboticists, AI specialists, Neuroscientists

    and psychologists is much needed

    Avoid for now the goal of designing a general theory for robot cognition

    and focus on designing robots with cognitive traits that are task specific

    More work needed on understanding the dynamics of cognitive paths in

    human brains

    Further work is needed on the mathematical modeling of developmental

    cognition in humans

    Final Thoughts

  • 74

    References

    [Koch 1985] C. Koch and S. Ullman, “Shifts in selective visual attention: toward the underlying neural circuitry," Human Neurobiology, vol. 4, pp. 219 -227, 1985.[Itti 1998] L. Itti, C. Koch, and E. Niebur, “A model of saliency-based visual attention for rapid scene analysis," IEEE Trans. Pattern Analysis and Machine Intelligence., vol. 20, pp. 1254-1259, 1998[Tsotsos 1995] J. K. Tsotsos, S. Culhane, Y. Winky, L. Yuzhong, N. Davis, and F. Nuo, “Modeling visual attention via selective tuning," Artificial Intelligence, vol. 78, pp. 507-545, 1995[Frintrop 2006] S. Frintrop, “VOCUS: A Visual Attention System for Object Detection and Goal-directed Search,” Lecture Notes in Articial Intelligence (LNAI), Vol. 3899, Springer Berlin/Heidelberg. ISBN: 3-540-32759-2, 2006.[Vijayakumar 2001] S. Vijayakumar, J. Conrad, T. Shibata, and S. Schaal, “Overt visual attention for a humanoid robot,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2001, pp. 2332 – 2337.[Dankers 2007] A. Dankers, N. Barnes, and A. Zelinsky, “A reactive vision system: Active-dynamic saliency,” in Proceedings of International Conference on Computer Vision Systems, 2007.[Vitay 2005] J. Vitay, N. P. Rougier, and F. Alexandre, A Distributed Model of Spatial Visual Attention. Springer-Verlag, 2005, pp. 54 –72.

  • 75

    References

    M. Begum, F. Karray, G. K. I. Mann, and R. G. Gosine probabilistic Model of Overt VisualAttention for Cognitive Robots, accepted for publication in IEEE SMC Part B

    M. Begum and F. Karray, omputational Intelligence Techniques in Bio-inspired Robotics,Computational Intelligence in Autonomous Robotic Systems, pp. 1-29, Springer 2008

    M. Begum, F. Karray, G. K. I. Mann, and R. G. Gosine, Probabilistic Approach forAttention-Based Multi-Modal Human-Robot Interaction, IEEE International Symposiumon Robot and Human Interactive Communication (IEEE RO-MAN), 2009

    M. Begum, F. Karray, G. K. I. Mann, and R. G. Gosine, e-mapping of Visual Saliency in Overt Attention: A Particle Filter Approach for Robotic Systems, IEEE International Conference on Robotics and Bio-mimetic, pp. 425-430, 2008

    M. Begum, G. K. Mann, R. Gosine, and F. Karray, Object- and space-based visual attention: An Integrated Framework for Autonomous Robots, IEEE/RSJ Intl. Conference on Intelligent Robots and Systems, pp. 301-306, 2008

    M. Begum, G. K. I. Mann, and R. G. Gosine, Biologically inspired Bayesian model of vsual attention for humanoid robots, Proceedings of IEEE-RAS International Conference on Humanoid Robots, pp. 587-592, 2006

  • 76

    References

    [Orabona 2005] F. Orabona, G. Metta, and G. Sandini, “Object-based visualattention: A model for a behaving robot,” in IEEE Conference on Computer Vision and Pattern Recognition, 2005.[Metta 2001] G. Metta, “An attentional system for humanoid robot exploitingspace variant vision,” in IEEE-RAS International Conference onHumanoid Robots, 2001.[Ruesch 2008] J. Ruesch, M. Lopes, A. Bernardino, J. Hornstein, J. S. Victor, and R. Pfeifer, “Multi modal saliency-based bottom-up attention: A framework for the humanoid robot icub,” in IEEE International conference on Robotics and automation, 2008, pp. 962 – 967.[Fleming 2006] K. A. Fleming and R. E. B. R. A. Peter II, “Image mapping and visual attention on a sensory ego-sphere,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2006, pp. 241 – 246.[Aryananda 2006] L. Aryananda, “Attending to learn and learning to attend for a social robot,” in IEEE International conference on Humanoid robots, 2006, pp. 618 – 623.[Canas 2008] J. M. Canas, M. M. Casa, and T. Gonzalez, “An overt visual attention mechanism based on saliency dynamics,” International Journal of Intelligent Computing in Medical Sciences and Image Processing, vol. 2, pp. 93 – 100, 2008.[Crespo 2009] J. L. Crespo, A. Faina, and R. J. Duro, “An adaptive detection/ attention mechanism for real time robot operation,” Neurocomputing, vol. 72, pp. 850 – 860, 2009.

  • Acknowledgements

    Dr. Alaa Khamis (PAMI)

    Dr. Momotaz Begum (PAMI)

    Dr. Malek Baklouti (PAMI, Thales)

    Mr. Jamil Abu Saleh (PAMI)

    Mr. jeremy Sun (PAMI)

    Mr. Ahmed ElMougy (PAMI)

    77

  • 78

    Campus

    University of Waterloo

  • 79

    Thank you

    Slide Number 1Slide Number 2Slide Number 3Slide Number 4OutlineSlide Number 6Toward Cognition in RoboticsToward Cognition in RoboticsToward Cognition in RoboticsToward Cognition in RoboticsToward Cognition in RoboticsToward Cognition in RoboticsToward Cognition in RoboticsToward Cognition in RoboticsSlide Number 15Major Aspects of Cognitive RoboticsMajor Aspects of Cognitive RoboticsMajor Aspects of Cognitive RoboticsMajor Aspects of Cognitive RoboticsMajor Aspects of Cognitive RoboticsMajor Aspects of Cognitive RoboticsMajor Aspects of Cognitive RoboticsMajor Aspects of Cognitive RoboticsMajor Aspects of Cognitive RoboticsMajor Aspects of Cognitive RoboticsMajor Aspects of Cognitive RoboticsMajor Aspects of Cognitive RoboticsMajor Aspects of Cognitive RoboticsMajor Aspects of Cognitive RoboticsMajor Aspects of Cognitive RoboticsMajor Aspects of Cognitive RoboticsMajor Aspects of Cognitive RoboticsPart 3: Aspects, Components and Architectures of Cognitive RoboticsSlide Number 34Recent AdvancesRecent AdvancesRecent AdvancesRecent AdvancesRecent AdvancesRecent AdvancesVisual Attention Model for robotic systemsRecent AdvancesVisual Attention Model for robotic systemsVisual Attention Model for robotic systemsVisual Attention Model for robotic systemsVisual Attention Model for robotic systemsVisual Attention Model for robotic systemsSlide Number 48Recent AdvancesToward Cognition in RoboticsRecent AdvancesRecent AdvancesRecent AdvancesRecent AdvancesRecent AdvancesTowards a Generalized Performance Metric for HRITowards a Generalized Performance Metric for HRITowards a Generalized Performance Metric for HRITowards a Generalized Performance Metric for HMITowards a Generalized Performance Metric for HMITowards a Generalized Performance Metric for HMITowards a Generalized Performance Metric for HMISlide Number 63ApplicationsApplicationsApplicationsApplicationsApplicationsApplicationsSlide Number 70ChallengesChallengesFinal ThoughtsReferencesReferencesReferencesAcknowledgementsUniversity of WaterlooThank you