Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector 1 Laurent Itti:...

1Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence

Lecture 7. Didday Model of Winner-Take-All

Reading Assignments:

TMB2:4.3, pp. 194-197. Prey Selection - or Winner Takes All4.4. A Mathematical Analysis of Neural Competition


The Prey-Selector Model of Didday

Consider how the frog's brain might select one of several visually presented prey objects.

The task: to design a distributed network (not a serial scan strategy) that could take a position-tagged "foodness array" and ensure that usually the strongest region of activity would influence the motor control system.

The prey-selector: an early, biological example of what connectionists call a winner-take-all network.

Didday’s model adds a relative foodness layer of cells in topographic correspondence to the retinotopic foodness layer.

The new layer yields the input to the motor circuitry.


Winner-take-all Networks

Goal: given an array of inputs, enhance the strongest (or strongest few) and suppress the others

No clear strong input yieldsglobal suppression

Strongest input is enhancedand suppresses other inputs


Winner-Take-All Networks

Basic architecture:- topographically organized

layer of neurons- each neuron receives

excitatoryinput (e.g., sensory)

- each neuron also sends inhibition

to neighboring neurons,

possibly via an inhibitory

inter-neuron

a

b

Example of 2-neuron WTAwith spiking neurons

strong stimulus

weak stimulus


Didday’s Model

= copy of input

= inhibitory inter-neurons

= receives excitationfrom foodness layerand inhibition fromS-cells

retinotopicinput


Didday's transformation scheme from foodness to relative-foodness

It uses a population of S “sameness“ cells in topographic correspondence with the other layers.

Each S-cell inhibits the activity that cells in its region of the relative-foodness layer receive from the corresponding cells in the foodness layer by an amount that augments with increasing activity outside its particular region.

This ensures that high activity in a region of the foodness layer penetrates only if the surrounding areas do not contain sufficiently high activity to block it.


Hysteresis

How does the network respond to a change in the maintained stimulus pattern?

Once in equilibrium, one may increase a non-maximal stimulus s2 so that it becomes larger than the previously largest stimulus s1, yet not switch activity to the corresponding element.

In neural networks with loops - an internal state resists dependence on input: buildup of excitation and inhibition precludes the system's quick response to new stimuli.

For example, if one of two very active regions were to suddenly become more active, then the deadlock should be broken quickly.

In the network so far described, however, the new activity cannot easily break through the excitation of its competitor and the global inhibition: there is hysteresis.


Hysteresis in a Ferromagnet

Not a single functionmagnetic field directionbut a history-dependent relation.

Direction of External Magnetic

Field

In a ferromagnetic material, the atoms act like magnets which tend to align themselves with their neighbors -- this cooperation is opposed by thermal noise.

A ferromagnet becomes magnetized when the atomic magnets have high probability of pointing in the same direction.


Didday's Transformation Scheme From Foodness to Relative-Foodness

Didday uses a local mechanism, to combat hysteresis, introducing a Newness cell - "N-cell" for each S-cell to monitor temporal changes in the activity of its region.

For a dramatic increase in the region's activity, it overrides the inhibition on the S-cell and permits this new level of activity to enter the relative foodness layer.


Formalizing: An "N-cell" for each S-cell

Introducing an "N-cell" for each S-cell

Formally: neurons Ni have activity ki proportional to dSi/dt.

We then postulate that theinput to the ith element toour array is no longer si

but si + ki.

If ki is large enough for

long enough, the relativefoodness Ei can gain

ascendancy.

Si

dSi/dt


Formalizing Didday's Model

The rest of this lecture provides a mathematical analysisfrom TMB2, Section 4.4, and is based on the paper:

Arbib, M.A., and Amari, S.I., (1977) Competition and Cooperation in Neural Nets, in Systems Neuroscience (J. Metzler, Ed.), New York: Academic Press, pp. 119-165.

To see the mathematics, switch to the Word file:

7W. Amari-Arbib Mathematics

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector 1 Laurent Itti:...

Documents

Transcript of Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector 1 Laurent Itti:...