Modelling the Division of Labor:

A Spiking Neuron Net Approach

Michele Sebag

TAOJoint work with Sylvain Chevallier, Helene Paugam-Moisy

SocPAR 2010

Framework: Swarm Robotics

Swarm-bot (2001-2005) Swarm Foraging, UWE

Symbrion IP, 2008-2013; http://symbrion.org/

Swarm Robotics: Why and What

WHAT

I Simple agentssimple micro-motives for macro-behaviors

I No pacemakersdecentralized, distributed, randomized systems

I More is different

An alternative to complex robots

I Inexpensive → Many → Reliable

I The “invisible hand“(Hayek’s inheritage ?)

Swarms: HOW

PrinciplesLocal information I` → estimates global quantities I

Local information → individual behaviour b(I`)Aggregate b(I`) = Behaviour[I ]

Examples

I Sounds & clusters of birds and frogs; Melhuish 99

I Bees & air-conditioning of the hive Auman 08

From observing to designing emergenceMain Issues

I Communication feasibility, cost

I Convergence individual and collective safety

I Reality Gap in simulation vs in-situ

I Bootstrapping how to prime the pump

Swarms: HOW

PrinciplesLocal information I` → estimates global quantities I

Local information → individual behaviour b(I`)Aggregate b(I`) = Behaviour[I ]

Examples

I Sounds & clusters of birds and frogs; Melhuish 99

I Bees & air-conditioning of the hive Auman 08

From observing to designing emergenceMain Issues

I Communication feasibility, cost

I Convergence individual and collective safety

I Reality Gap in simulation vs in-situ

I Bootstrapping how to prime the pump

This talk focuses on

Division of labor

Synchronization

How do social agents proceed to synchronize their activities?

Overview

I Swarm Robotics

I Biological / Artificial modelsI SpikeAnts

I Spiking NeuronsI Network Architecture

I Analysis

I Discussion and Perspectives

Biological/Artificial models

BatteryMotors

Software

The hardware perspective

Division of laborthe social stomach ?

Biological/Artificial models

BatteryMotors

Software

The hardware perspectiveDivision of labor

the social stomach ?

Biological/Artificial models, 2

J. Halloy et al., 2010

Social stomach: Macro-modelling

X Foraging robots β rate of energy stocking

S Stocking robots µ rate of energy consumption

Y Other robots (θ + Xt) rate of recruitment∂X∂t = (θ + Xt)(N − X − S)− βX∂S∂t = βX − µS

Y: empty robots

X: foraging robotsrate of recruitment

rate of stocking

S: stocking robots

rate of consumption

Division of labor, 2

Winfield & Liu 08Finding food/resting

I Finding food delivers energy

I Searching costs energy

I bumping into other robots costs energy

Goal

I Allocate time between search and resthttp://www.brl.uwe.ac.uk/projects/swarm/index.html

Adaptive Foraging in Swarm Robotic Systems, 2

Probabilistic Finite State Machine

Design

I Find transition probabilities

I Rest and Search thresholdsI Input:

I internal cues (food retrieved)I environment cues (bumping into other

robots)I social cues (success/failure of relatives)

Controller design in Swarm Robotics

Constraints design of emergence

I Spatially distributed

I Decentralized (no pacemaker)

I Asynchronous

Available information

I Cues from relatives

I Internal time (hunger-like)

Can it be avoided ?

I Random generator probabilistic model

I Sophisticated skills counting ability

Overview

I Swarm Robotics

I Biological / Artificial modelsI SpikeAnts

I Spiking NeuronsI Network Architecture

I Analysis

I Discussion and Perspectives

Spiking neurons vs std neurons

Standard neurons

I Directed graph G = (E ,V) and weights W

I An activation function: (linear, sigmoid, RBF)

ei (t + 1) =← f (∑j

wijej(t))

Spiking neurons Hodgkin Huxley 52

I Internal state (membrane potential)

I Activation function ≡ differential equation

∂e(t)∂t = f (e(t),Excitations, Inhibitions) if e(t) < ϑ

else fires a spike and e(t) is set to Vreset

Spiking neurons, 2

What is new

I An asynchronous process

I What matters is the dynamics of the input

Modelling/studying dynamics

I Synchrony in cell assembliesHebb, 49

I Complete synchronyMirollo, 90

I Transient synchronyHopfield, 01

I Order-chaos phase transitionSchrauwen, 08

I PolychronizationIzhikevich, 06

I Rhythmic oscillationsBrunel, 03

Synchronization

In biological systems

I Fireflies

I Cricket chirping

I Pacemaker cells of the heart

I Neural cells

Questions

I Why (synchronized patterns are more efficient ?)

I How ?

ClaimEmergence/Dynamics results from individual interactions

Cole 91, Gordon 92

Division of labor among foraging ants

PrincipleThe ant goes foraging

iff she does not see sufficiently many ants foraging

Related problems

I The Dying seminar Schelling 1978

I The El Farol bar Arthur 1994

The Dying Seminar

Schelling, 1978; Nadal et al. 2009

Individual Utility Function

I N scientists are asked to go to a seminar:

I ... scientist i will go if #attendees > n(i)

The Dying Seminar

Schelling, 1978; Nadal et al. 2009

Individual Utility Function

I N scientists are asked to go to a seminar:

I ... scientist i will go if #attendees > n(i)

The Dying Seminar

Schelling, 1978; Nadal et al. 2009

Individual Utility Function

I N scientists are asked to go to a seminar:

I ... scientist i will go if #attendees > n(i)

The El Farol bar

Arthur 1994

Individual Utility Function 100 scientists

I The best option is to go to El Farol bar

I ... if not too many people go to the bar... < 60

I otherwise, better stay at home...

Devising a policy

I Random draw: go to the bar with probability .6

I Find rules to predict the attendance, based on the history

Division of labor among foraging ants

PrincipleThe ant goes foraging

iff she does not see sufficiently many ants foraging

Related problems

I The Dying seminar Schelling 1978

I The El Farol bar Arthur 1994

Differences

I Not an imitation game survival of the colony

I No synchronization

The foraging colony

A 4 state agent model

I S leep

I Observe

I Forage

I General Interest

S O

F

G

Ant policy

1. If I don’t see “sufficiently many” foraging ants,I go foraging (then sleeping)

2. Otherwise, I go for General Interest tasks

3. After any task, back to Observation

The ant model: Two spiking neurons

Passive neuron Leaky Integrate-and-Fire (LIF)

dVp

dt = −λ(Vp(t)− Vrest) + Iexc(t), if Vp < ϑelse fires a spike and Vp is set to V p

reset

Excitation: signal of working ants

Active neuron Quadratic Integrate-and-Fire (QIF)

dVa

dt = −λ(Va(t)− Vrest)(Va(t)− Vthres) + Iinh(t) + Iclock(t), if Va < ϑelse fires a spike and Va is set to V a

reset

Inhibition: signal of working ants

Excitation: internal clock

Bistable:

{> Vrest bursting regime< Vrest goto V a

reset

The ant model: Two spiking neurons, foll’d

During the observation state,Decision making: Competition of active and passive neuron

I if Active wins, goto F (and emits spikes, sent to neighborants)

I If Passive wins, goto GI if none wins before tO, goto F .

wins= emits a spikeOther states

I Passive and Active neurons are not excited/inhibited.

Microscopic Scale

Sleep state

PA

0

0.5

1

1.5

0 20 40 60 80 100 120

Active neuronPassive neuron

ϑ

Time (ms)

Microscopic Scale

Observation state

PA

0

0.5

1

1.5

0 20 40 60 80 100 120

Active neuronPassive neuron

ϑ

Time (ms)

Microscopic Scale

Foraging state

PA

0

0.5

1

1.5

0 20 40 60 80 100 120

Active neuronPassive neuron

ϑ

Time (ms)

Microscopic Scale

Sleep state

PA

0

0.5

1

1.5

0 20 40 60 80 100 120

Active neuronPassive neuron

ϑ

Time (ms)

Microscopic Scale

Observation state

PA

0

0.5

1

1.5

0 20 40 60 80 100 120

Active neuronPassive neuron

ϑ

Time (ms)

Microscopic Scale

General Interest state

PA

0

0.5

1

1.5

0 20 40 60 80 100 120

Active neuronPassive neuron

ϑ

Time (ms)

Microscopic Scale

Observation state

PA

0

0.5

1

1.5

0 20 40 60 80 100 120

Active neuronPassive neuron

ϑ

Time (ms)

Microscopic Scale

Foraging state

PA

0

0.5

1

1.5

0 20 40 60 80 100 120

Active neuronPassive neuron

ϑ

Time (ms)

Macroscopic Scale

Ant colony

I M ants = M (active, passive) neurons

I A spiking neuron network

I Sparsely connected (connectivity ρ)

What happens ?n(t): number of foraging ants at time t

Foraging effort F =∑t

n(t)

Parameters of the model

Parameter type Symbol Description Value (units)

Neural λ Membrane relaxation constant 0.1 mV−1

Vrest Resting potential 0.0 mVϑ Spike firing threshold 1.0 mV

V preset Passive neuron reset potential -0.1 mV

Vthres Active neuron bifurcation threshold 0.5 mVV a

reset Active neuron reset potential 0.55 mVIclock Active neuron constant input current 0.1 mVw Synaptic weight 0.01 mV−1

Agent tF Foraging duration 47.1 mstO Maximum observation duration 10.5 mstS Sleeping duration 45.7 mstG General I. duration 16.7 ms

Population ρ Connection probability 0.3 %M Population size 150 agents

Initializationevery ant sleeps and wakes up after U[0, 2tS ]

Foraging effort: Sensitivity analysis

Average on 10 independent runs times 100,000 time steps

200

400

600

800

1000

0 200 400 600 800 1000200

300

400

500

600

700

0 0.2 0.4 0.6 0.8 1

FF

M ρ

0

500

1000

1500

0 0.05 0.1 0.15 0.2

Fw

200

240

280

0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95

F

V areset

vs population size M, connectivity ρ,active neuron reset potential V a

reset and synaptic weight w .

Influence of connectivity ρ

ρ = 0.1

ρ = 0.2

0

10

20

30

40

50

60

0 500 1000 1500 2000 2500 3000

nF

(t)

01020304050607080

0 500 1000 1500 2000 2500 3000

nF

(t)

t

Emergence of workshift as ρ increases.

Influence of population size M

0102030405060708090

0 500 1000 1500 2000 2500 3000

nF

(t)

300 agents

050

100150200250300350

0 500 1000 1500 2000 2500 3000

nF

(t)

t

1000 agents

Variance of workshift size increases with M

Macroscopic study, foll’d

First indicator: Foraging effort FI Behaves as expected

I high variance in some regions.

Second indicator: Entropy of synchronization HI Consider n(t) number of foraging agents at t

I Discard orphan time steps t s.t. n(t − 1) 6= n(t) 6= n(t + 1)

I LetN = {n(t), t = 1 . . .T , n(t) = n(t + 1) or n(t) = n(t − 1)}

I Let pn ∝ |{t, n(t) = n, n ∈ N}|H = −

∑n∈N

pn log pn

Three different regimes

0

10

20

30

40

50

60

70

80

0 500 1000 1500 2000

n F(t

)

Simulated time t

0

50

100

150

200

250

0 500 1000 1500 2000

n F(t

)Simulated time t

0100200300400500600700800900

1000

0 500 1000 1500 2000

n F(t

)

Simulated time t

Asynchronous Synchronous aperiodic Synchronous periodicA B C

0

10

20

30

40

50

60

70

80

0 10 20 30 40 50 60 70 80

n F(t

+1)

nF(t)

50

100

150

200

250

50 100 150 200 250

n F(t

+1)

nF(t)

200

400

600

800

1000

200 400 600 800 1000

n F(t

+1)

nF(t)

H = 0 High Log2

Three different regimes

0

10

20

30

40

50

60

70

80

0 500 1000 1500 2000

n F(t

)

Simulated time t

0

50

100

150

200

250

0 500 1000 1500 2000

n F(t

)

Simulated time t

0100200300400500600700800900

1000

0 500 1000 1500 2000

n F(t

)

Simulated time t

Asynchronous Synchronous aperiodic Synchronous periodicA B C

0

50

100

150

200

0 500 1000 1500 2000Simulated time t

0

50

100

150

200

0 500 1000 1500 2000Simulated time t

0

50

100

150

200

0 500 1000 1500 2000Simulated time t

Raster plot: Active = red, passive = blue

SpikeAnts: Emergent synchronization

Control parameters

I Sociability ρ√M

I Receptivity w|ϑ−Vrest|

Phase diagram

C

B

AC

B

A

0 0.2 0.4 0.6 0.8

0.05

0.1

0.15

0.2

Rec

epti

vity

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

0 0.2 0.4 0.6 0.8

0.05

0.1

0.15

0.2

Rec

epti

vity

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

H(m

ean)

H(s

tand

ard

devi

atio

n)

Sociability Sociability

SpikeAnts: Emergent synchronization

Control parameters

I Sociability ρ√M

I Receptivity w|ϑ−Vrest|

Phase diagram

C

B

AC

B

A

0 0.2 0.4 0.6 0.8

0.05

0.1

0.15

0.2

Rec

epti

vity

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

0 0.2 0.4 0.6 0.8

0.05

0.1

0.15

0.2

Rec

epti

vity

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

H(m

ean)

H(s

tand

ard

devi

atio

n)

Sociability Sociability

SpikeAnts: A representative run

at the triple point

B A B A B C

0

50

100

150

200

250

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000

n F(t

)

t

Stable regime: synchronous periodic.

Overview

I Swarm Robotics

I Biological / Artificial modelsI SpikeAnts

I Spiking NeuronsI Network Architecture

I Analysis

I Discussion and Perspectives

SpikeAnts

The model

I Frugal, deterministic model

I Biological plausibility / no counting abilities

I Accounts for the emergence of synchronization

Further extensions

I Comparisons with probabilistic models

I Stochastic parameters

Further extensions

Reconsidering excitation/inhibitionFrom SpikeAnts to an Ising model

The environment handling perturbationsWhat can be learned/optimized within SpikeAnts ?

Going realImplementing SpikeAnts

Thanks

I Sylvain Chevallier, Helene Paugam Moisy TAO, LRI

I Jose Halloy, Jean-Louis Deneubourg VUB

I Symbrion IP

More in NIPS 2010.