Contact info [email protected] Controlling Mobile Robots with Distributed Neuro-Biological Systems...

Contact [email protected]

Controlling Mobile Robots with Distributed

Neuro-Biological SystemsSebastian Gutierrez-Nolasco (UCI)

Nalini Venkatasubramanian (UCI)

Alfredo Weitzenfeld (ITAM)

Biologically Inspired Robotic Systems

Nature has always been a source of inspiration in the development of autonomous robotic systems Ethology

Animal behavior-based simulation Interaction with the environment is usually oversimplified

Lack of strong biological basis for their working assumptions Lack of any formal underpinnings for the simulation results

Neuroethology behavior related to neurobiological structure

Replicate brain models to provide credible and general animal behavior Provide inspiration for further robotics architectures

More complex and accurate than ethology systems Enable experimentation Experimentation requires real-time performance

Neuroethological robotic systems

Super Robots Incorporate extensive

processing capabilities Bulky Expensive

Inexpensive Robots Smaller and inexpensive robots

connected to a network of processing nodes

Concerns Real-time performance Unpredictable communication

environment affects robot performance

A neural model may take hours of processing time Simulation of multiple neural networks require a distributed

processing environment A typical retina model may consist of more than 100,000 neurons and

500,000 interconnections

Biologically inspired robotics demand sophisticated image processing techniques Communication intensive tasks are required

Autonomous robotic agents have real-time and processing restrictions, as well as power awareness requirements Battery usage is a major concern in mobile robots

Challenges of Biologically Inspired Autonomous Robots

Developing software for autonomous mobile robots is complex Highly heterogeneous methods for capturing and processing

sensor information Multiple sensory input devices Sensory input multi-granularity

Communication is error-prone due to unpredictable interference and failures partial and complete failures Unreliability and disconnection Varying available bandwidth

Developing Biologically Inspired Robot Architectures

Our Approach

Develop an embedded architecture capable of conducting neuroethological robotic experimentation

Inexpensive small robots communicate (wireless) with distributed computational resources

Neural models are distributed in multiple processing nodes

Adaptive robotic middleware optimizes robot communication in response to varying network conditions

Structure of the Talk

1. Neuroethological Modeling Study animal behavior and corresponding neural structure

as inspiration to robotic architectures

2. Embedded Mobile Robots Develop distributed wireless robot architectures capable of

efficient neural processing

3. Adaptive Middleware Achieve real-time computation and adapt embedded

architecture to varying network conditions

4. Internet Based Robotics Enable remote robot task development and

experimentation








experimentation

PSMate

Mate-PairS

PSPrey

Prey-AcqS

PSPred

Pred-AvS

Moving-ObjectS

PSNMO

Find-LocS

Non-Moving-ObjectS

PSMO

+ ++

-

--

Perceptual Schema (PS)

Main Schema (S)

Behavior:Frog and Toad - Rana Computatrix [Arbib 1987, Cervantes 1990]

Mobile visual stimulus in lateral visual field(monocular perception)

Mobile visual stimulus in binocular visual field(short distance)

Mechanic stimulusin mouth and pharynxreceptors

Orientation

Binocularfixation

Attack

Snap

Clean

Stimulus Response

Behavior:Toad Prey Acquisition [Cervantes 1985]

Behavior:Toad Prey Acquisition with Detour Behavior Before and After Learning [Corbacho and Arbib 1995]

20cm Barrier Before learning 20cm Barrier After learning

10cm Barrier

Schema Level 1 data indata out

Other Processes

Schema Level 2

Schema Level

Neural Level......... ...

din1

dinn

dout1

doutm

Schema

Neural

Schema Computational Model

R4

Visual

R1-R2

R3

Retina

Tectum

PreTectum/Thalamus

Motor Heading Map

Static ObjectRecognizer

Prey Recognizer

TH10

T5_2

Static ObjectAvoidance

Prey Approach

Forward

Orient

Sidestep

Backward

Tactile Schema Level

Neural Level

Moving Stimulus Selector

R1-R2

R3

R4MaxSelector

PredatorRecognizer Predator Avoid

R1-R2

R3

R4

Depth

Stereo

Neural based Behavior : Toad Prey Acquisitions and Predator Avoidance

1

2

3

4

5

67

1189

1012

1314

1516

10cm barrier 20cm barrierBefore learning

20cm barrierAfter learning

Toad Prey Acquisition with Detour:Simulation Results








experimentation

Sensors Actuators Vision Tact

Legs Wheels

LEGO

OOPIC

Embedded Mobile Robots:Robot Hardware

Autonomous Robot 1

Autonomous Robot N

Inte

rnet

Wireless

InternetServer

... ......

RemoteComputaional

System

Instance 1

Instance N

RemoteComputational

System

Embedded Mobile Robots:Distributed Embedded Architecture

Time consuming processes are carried out in the (neural) computational system Neural processing Image processing

Limited task are carried out in the robot hardware Sensory input Motor output Default behavior

Communication and data transformation is managed by the adaptive middleware


motormotor

servo

camera

CPU(OOPic)

Transceiver

Power stage

Frame Grabber

Sensors(tact)

TransceiverPC

Remote Computaional System

Robot

Wireless


NSL NSL

NSL NSL

NSL NSL

NSL NSL

ASL ASL

ASLASL

VideoServer

Video/Image

Processingcamera

Robot

Wireless

Remote Computational System

TactileServer

Motor Server

tactile

motortran

scei

ver transceiver

NSL – Neural Simulation LanguageASL – Abstract Schema Language


Video capture

Video processing

Model simulation

Model output

Navigation control

(d , r , c)

Embedded Mobile Robots:Processing cycle








experimentation

Distributed Systems Middleware

Enables the modular interconnection of distributed software Abstract over low level mechanisms used to implement resource

management services

Concurrent Object Oriented Model Separation of concerns and reuse of services

Customizable, Composable Middleware Frameworks Provide for dynamic network and system customizations,

dynamic invocation/revocation/installation of services Concurrent execution of multiple resource management policies

Core Resource Management Services

Core Services - basic services where interactions between the application and system can occur. Building blocks for other services Reduce interactions among many services to interactions

between a few simple services

Choosing core services - commonly observed patterns Recreation of data/services at a remote site Capturing approximation of distributed state at multiple sites Interactions with a global repository

TLAM: The Two Level Meta-architecture

DistributedSnapshot

Remote Creation

Directory Services

Replication

Migration

DGC

Check-pointing

AccessControl

System (Meta) Level

Application (Base) Level

Adaptive Robotic Middleware (ARM)

Extends the TLAM to Optimize information flow between robots and the computational

system Determine how, when and what information should be modified in

order to match fluctuations in the communication environment

Compose communication protocols to obtain the combined benefits - conflicting requirements Explicit knowledge of how communication protocols compose and

interact is required

Adapt protocols and mechanisms to changing communication and power constraints

ARM

ARM

NSL NSL

NSL NSL

NSL NSL

NSL NSL

ASL ASL

ASLASL

VideoServer

Video/Image

processingcamera

Wireless

TactileServer

Motor Server

tactile

motortran

scei

ver transceiver

Robot

Remote Computational System

NSL – Neural Simulation LanguageASL – Abstract Schema Language

ARM: Distributed Embedded Architecture

Communication manager Provide and enforce application level requirements Components

Oracle determine most suitable protocol implementation in terms of coverage and efficiency

Set of communication protocols Protocol installer/uninstaller Resident ARM module running in the robot (resident evil)

Adaptation manager Provide adaptation and monitor mechanisms operating at different levels of

abstraction Reactive

Triggered when failure to achieve intended communication goal is detected Proactive

Triggered when a more efficient communication can be achieved under the current environment conditions

Adaptation Repository Determine most suitable adaptation strategy to be applied

ARM: Components

Protocol Oracle

ARM manager

Protocol(un)installer

Protocolloader

Secure compressed video capture

SecurityProtocol

CompressionProtocol

TimelinessProtocol

set of protocols

Robot’sresident

ARM

AdaptationProfiler

Battery Monitors

Communication Performance Monitor

AdaptationRepository

Communication Manager

Adaptation Manager

Communication Requirement

Protocol Oracle

Protocol Oracle

ARM manager

Protocol(un)installer

Protocolloader



SecurityProtocol

CompressionProtocol

TimelinessProtocol

set of protocols

Robot’sresident

ARM

AdaptationProfiler

Battery Monitors

Communication Performance Monitor

AdaptationRepositoryAdaptationRepository

Communication Manager

Adaptation Manager

Communication Requirement

ARM: Example








experimentation

Interned based Robotics: Web Access

Experimental Results: 2 Preys

Experimental Results: 2 Preys and Predator

(A) (B)

(C) (D)

Embedded Mobile Robots:Experimental Results: Prey Acquisition with 10 cm Barrier

(A) (B) (C) (D)

(E) (F) (G) (H)

Embedded Mobile Robots:Experimental Results: Prey Acquisition with 20 cm Barrier

Neural based Behavior: Prey Acquisition (10cm barrier)

• Barrier (PreTectum)

• Prey (Tectum)

• Integrated (MHM)

• Heading (MHM)

•Tactile

Visual Fields

Neural based Behavior: Prey Acquisition (20cm barrier before bumping)


• Prey (Tectum)


• Heading (MHM)

•Tactile

Visual Fields

Neural based Behavior: Prey Acquisition (20cm barrier after bumping)


• Prey (Tectum)


• Heading (MHM)

•Tactile

Visual Fields

Neural based Behavior: Prey Acquisition (20cm barrier after learning)


• Prey (Tectum)


• Heading (MHM)

•Tactile

Visual Fields

Future Work

Complete Internet based System Develop middleware adaptation capabilities Build smaller robotic systems Extend to multiple robot tasks Extend vision system to “true” moving forms Extend biological models

Bonus Section

Neuroscience(Experiments)

RoboticsBrain Theory(Modeling)

New HypothesisGaps in Knowledge

Data, Hypotheses Formal Models

New Ideas(Results from

Experiments withPhysical Devices)

New Hypotheses

Research Cycle

T - Temporal, D - Dorsal, N - Nasal, V - VentralO - Optic Tectum, B - Nucleus of BelonciC - Lateral Geniculate Nucleus,P - Thalamic Pretectal NeuropilX - Basal Optic Root[Scalia and Fite 1974]

Neural Maps

mp

neuron

s mf

input output

mp - membrane potential : dmp(t)/dt = f(s,mp,t) mf - firing rate : mf(t) =(mp(t)) Leaky Integrator : dm(t)/dt = -m(t) + s

Neuron Model

TP – ThalamusPreTectumGL – GlomerelusSN – Stellate NeuronsSP – Small PearLP – Large PearPY - Pyramidal

TP

LP

SN

SP

GL

PY

R4R3R2

++

++

+

+-

+

++ +

+

-

+

+-

+

-

+

Output

Retina

+-

+

-

Excitation

Inhibition

Synapsis

Input

Retina-Thalamus-Tectum

vp

vf

up0up1 upi upn-1

ufn-1uf0 uf1

w0

ufi

w1wi

wn-1

s0 s1si sn-1

........

wu0wu1 wui

wun-1

wm

........

imiuii

u shvgwufwudt

tdu 1

v

dv

dt v wn f ui

i 1

n

h2

f (ui ) 1 ui 0

0 ui 0

g(v) v v 0

0 v 0

Max Selector [Didday 1976]

uf

Ulayer

vf

Vlayer

u_inv_in

s_inin

outMaxSelector

s_out

MaxSelectorStimulus

MaxSelectorModel

MaxSelectorOutput

nslModel MaxSelectorModel () extends NslModel(){

private MaxSelector maxselector(10);private MaxSelectorStimulus stimulus(10);private MaxSelectorOutput output();

public void initSys() {system.setRunTime(10.0);system.setRunDelta(0.1);

}public void makeConn() {

nslConnect(stimulus.s_out,maxselector.s_in);nslConnect(stimulus.s_out,output.s_in);nslConnect(maxselector.out, output.uf);

}}

uf s_in

Max Selector Model

uf

Ulayer

vf

Vlayer

u_in

v_in

s_inin

out

MaxSelector

nslModule MaxSelector (int size) extends NslModule() {

public Ulayer u1(size);public Vlayer v1(size);public NslDinDouble1 in(size);public NslDoutDouble1 out(size);

public void makeConn(){nslRelabel(in,u1.s_in);nslConnect(v1.vf,u1.v_in);nslConnect(u1.uf,v1.u_in);nslRelabel(u1.uf,out);

}}

Max Selector Module

nslModule Ulayer(int size) extends NslModule () {

public NslDinDouble1 s_in(size); public NslDinDouble0 v_in();public NslDoutDouble1 uf(size);private NslDouble1 up(size);private NslDouble0 hu();private double tau;

public void simRun() { up =0; uf = 0; hu = 0.1; tau =1.0;

} public void simRun() {

up = nslDiff(up,tau, -up + uf - v_in – hu + s_in); uf = nslStep(up,0.1,0.1.0);

}}

nslModule Vlayer(int size) extends NslModule () {

public NslDinDouble1 u_in(size);public NslDoutDouble0 vf();private NslDouble0 vp();private NslDouble0 hv();private double tau;

public void initRun() { vp =0; vf = 0; hv=0.5; tau=1.0;

} public void simRun() {

vp = nslDiff(vp,tau,-vp+nslSum(u_in) – hv); vf = nslRamp(vp);

}}

uf

Ulayer

v_in

s_invf

Vlayer

u _in

Ulayer and Vlayer Modules

Axon

Neuron

Dendrite

Soma

Synapse

Spine

Synaptic Cleft

AxonTerminal

Receptor

InteracellularElement

Vescicle

CalciumMechanism

NMDAReceptor

AMPAReceptor

PumpDiffusionChannel Buffer

DendritesAxon

Synapses

Soma

Neuron (detailed)

Neuroscience:Autonomous Biological Agents

Sensors Actuators Vision Sound Smell Touch

Legs Wings Fins

Robotics:Autonomous Robotic Agents

Sensors Actuators Vision Sound Smell Touch

Legs Wings FinsWheels

Contact info [email protected] Controlling Mobile Robots with Distributed Neuro-Biological Systems...

Documents

Transcript of Contact info [email protected] Controlling Mobile Robots with Distributed Neuro-Biological Systems...