CSCI 252: Neural Networks Prof. Levy

“Architecture” #4: Neuromorphic Vision Chips

(Koch & Mathur 1996;Barrows, Srinivasan, & Chahl 2003;

Conradt 2009)

Topics

● Basic Circuit Concepts– Ohm’s Law– Digital vs. Analog– Common Circuit Components

● Image Processing– Convolution– Optical flow

Part I: Basic Circuit Concepts

Understanding Signals: Ohm's Law

• Electricity is like water going through a hose:

• Pressure : Voltage

• Constriction: Resistance

• Flow: Current

• I = V / R : Current = Voltage / Resistance

• Voltage: Volts; Current: Amp(ere)s;

Resistance: Ohms (Ω)

Ohm's Law Example

Question: A 120 Volt lamp is powering a lightbulb that has a resistance of 240 Ohms. How much current is passing through the lightbulb?

Answer: I = V / R = 120 V / 240 Ω = 0.5 A

Ohm's Law• Very high voltage, very low current

(10000 V * .0001 A = 1 Wat)

• Low voltage, low current

(3.7V * 1A = 3.7 Wats)

• High voltage, high current

(1000V * 1000A =

1000000 Wats!!!)

Analog vs. Digital

Analog

Continuous

Measuring

Ruler

Real numbers

Smooth

“Natural”

Digital

Discrete

Counting

Abacus

Integers

Step-like

Arbitrary

House Mountain House Mountain

0,2,4,3,3,3,4,1,0

Analog vs. Digital Circuits

● Electricity is inherently analog: we measure voltage, current, resistance; we don’t “count” it.

● Electronic components can be assembled to make circuits.

● When engineers say “digital circuit”, they mean “electronic circuit that approximates discrete voltages”

● Both digital and analog circuits involve three major components

● Let’s look at each in turn ….

Resistor: Reduce current or voltage

• Recall Ohm’s Law: I = V / R

• So more resistance means less current

• We can also use two resistors to build a voltage divider:

V out=V in

R2

R1+R2

Capacitor: Slow down fluctuations

• Like a fast-charging/draining batery

• Can be used to “damp down” rapid changes in voltage (noise reduction):

Transistor: Use one signal tocontrol another

• Analog usage: amplifier

• Digital usage: on/off switch for Boolean logic gates

• The basis of all modern digital computers

• Integrated circuit shrinks billions of these down to a tiny die (chip) you can conceal under your finger

Analog vs. Digital Example: SOM Update

# Adjust weights of winner neighbors

for p in self._neighbors(winner, d):

self.u[p] += alpha*(e-self.u[p])

Part II: Image Convolution

http://wernerstudio.typepad.com/.a/6a00d83549adb153ef01053629c0cf970c-500wi

Convolution:

Convolution is a mathematical way of combining two signals to form a third signal. It is the single most important technique in Digital Signal Processing.*

* http://www.analog.com/media/en/technical-documentation/dsp-book/dsp_book_Ch6.pdf

Convolution: “The Dot Product on Steroids”

Let's look dot product in terms of vectors (arrays):

∑i=1

n

x i w i

x1

x2

x3

xn

...

w1

w2

w3

wn

...* * * *

+

Convolution: “The Dot Product on Steroids”

To get convolution, we “slide” w across x:

x1

x2

x3

xn

...

w1

w2

w3

wn

...* * * *

+y

1

x1

x2

x3

xn

...

w2

w3

w4

wn+1

...* * * *

+y

2

x1

x2

x3

xn

...

w3

w4

w5

wn+2

...* * * *

+y

3

Do you see a problem?

Convolution: “The Dot Product on Steroids”

Usual definition of convolution assumes infinite vectors:

yn= ∑k=−∞

∞

xn−k w k

x3

x2

x1

...

w-1

w0

w1

...* * *

+y

2

...

...x2

x1

x0

...

w-1

w0

w1

...* * *

+y

1

...

... x4

x3

x2

...

w-1

w0

w1

...* * *

+y

3

...

...

Of course, this is unrealistic, so ...

Convolution: “The Dot Product on Steroids”

We treat w as a finite convolution kernel, and switch to non-negative indices: e.g.,:

yn=∑k=0

2

xn−k w k

x3

x2

x1

...

w0

w1

w2

* * *

+y

3

...x2

x1

x0

...

w0

w1

w2

* * *

+y

2

... x4

x3

x2

...

w0

w1

w2

* * *

+y

4

...

Do you notice another (small) problem?

w

x

y

w

x

y

w

x

y

w

x

y

w

x

y

w

x

y c c

w

x

y c c c

w

x

y c c c c

w

x

y c c c c c

Convolution: What's It Goodfer?

• Many useful operations can be expressed as a convolution

• E.g., smoothing, a.ka. moving average, a.k.a. low-pass filtering ….

w

x

y

.33 .33.33

1 1 1 1 10 0 0 0 0

w

x

y

.33 .33.33

1 1 1 1 10 0 0 0 0

.66

w

x

y

.33 .33.33

1 1 1 1 10 0 0 0 0

.66 .33

w

x

y

.33 .33.33

1 1 1 1 10 0 0 0 0

.66 .33 .66

w

x

y

.33 .33.33

1 1 1 1 10 0 0 0 0

.66 .33 .66 .33

w

x

y

.33 .33.33

1 1 1 1 10 0 0 0 0

.66 .33 .66 .33 .66

w

x

y

.33 .33.33

1 1 1 1 10 0 0 0 0

.66 .33 .66 .33 .66 .33

w

x

y

.33 .33.33

1 1 1 1 10 0 0 0 0

.66 .33 .66 .33 .66 .33 .66

w

x

y

.33 .33.33

1 1 1 1 10 0 0 0 0

.66 .33 .66 .33 .66 .33 .66 .33

Convolution: What's It Goodfer?

• Many useful operations can be expressed as a convolution

• E.g., edge detection, a.k.a. high-pass filtering

w

x

y

-1 -12

1 1 1 1 10 0 0 0 0

w

x

y

-1 -12

1 1 1 1 10 0 0 0 0

-2

w

x

y

-1 -12

1 1 1 1 10 0 0 0 0

-2 2

w

x

y

-1 -12

1 1 1 1 10 0 0 0 0

-2 2 -2

w

x

y

-1 -12

1 1 1 1 10 0 0 0 0

-2 2 -2 2

w

x

y

-1 -12

1 1 1 1 10 0 0 0 0

-2 2 -2 2 -2

w

x

y

-1 -12

1 1 1 1 10 0 0 0 0

-2 2 -2 2 -2 2

w

x

y

-1 -12

1 1 1 1 10 0 0 0 0

-2 2 -2 2 -2 2 -2

w

x

y

-1 -12

1 1 1 1 10 0 0 0 0

-2 2 -2 2 -2 2 -2 2

Two-Dimensional Convolution

w

x

Two-Dimensional Convolution

x y

Two-Dimensional Convolution

x y

Two-Dimensional Convolution

x y

Two-Dimensional Convolution

x y

Common Kernels for Image Manipulation

https://en.wikipedia.org/wiki/Kernel_(image_processing)

Common Kernels for Image Manipulation

https://en.wikipedia.org/wiki/Kernel_(image_processing)

Homebrew Convolution with NumPy

Optical Flow

• A basic question: how do we (and other sighted creatures) move around without bumping into things?

• Ecological insight (J.J. Gibson): It can’t involve a complicated computational algorithm in the brain* As much as possible, an animal must be have its senses / nervous system tightly coupled with its environment.

* But, because of the lack of analog optical-flow circuits, nearly everyone still does it with an algorithm / package (OpenCV)!

Hardware Progress

PX4Flow2012$150Inaccurate / difficult to use

PWM39012018$36Accurate / easy to use

The Cutting Edge

https://studywolf.wordpress.com/tag/spiking-camera/