Post on 18-Aug-2020
CSCI 252: Neural Networks Prof. Levy
“Architecture” #4: Neuromorphic Vision Chips
(Koch & Mathur 1996;Barrows, Srinivasan, & Chahl 2003;
Conradt 2009)
Topics
● Basic Circuit Concepts– Ohm’s Law– Digital vs. Analog– Common Circuit Components
● Image Processing– Convolution– Optical flow
Part I: Basic Circuit Concepts
Understanding Signals: Ohm's Law
• Electricity is like water going through a hose:
• Pressure : Voltage
• Constriction: Resistance
• Flow: Current
• I = V / R : Current = Voltage / Resistance
• Voltage: Volts; Current: Amp(ere)s;
Resistance: Ohms (Ω)
Ohm's Law Example
Question: A 120 Volt lamp is powering a lightbulb that has a resistance of 240 Ohms. How much current is passing through the lightbulb?
Answer: I = V / R = 120 V / 240 Ω = 0.5 A
Ohm's Law• Very high voltage, very low current
(10000 V * .0001 A = 1 Wat)
• Low voltage, low current
(3.7V * 1A = 3.7 Wats)
• High voltage, high current
(1000V * 1000A =
1000000 Wats!!!)
Analog vs. Digital
Analog
Continuous
Measuring
Ruler
Real numbers
Smooth
“Natural”
Digital
Discrete
Counting
Abacus
Integers
Step-like
Arbitrary
House Mountain House Mountain
0,2,4,3,3,3,4,1,0
Analog vs. Digital Circuits
● Electricity is inherently analog: we measure voltage, current, resistance; we don’t “count” it.
● Electronic components can be assembled to make circuits.
● When engineers say “digital circuit”, they mean “electronic circuit that approximates discrete voltages”
● Both digital and analog circuits involve three major components
● Let’s look at each in turn ….
Resistor: Reduce current or voltage
• Recall Ohm’s Law: I = V / R
• So more resistance means less current
• We can also use two resistors to build a voltage divider:
V out=V in
R2
R1+R2
Capacitor: Slow down fluctuations
• Like a fast-charging/draining batery
• Can be used to “damp down” rapid changes in voltage (noise reduction):
Transistor: Use one signal tocontrol another
• Analog usage: amplifier
• Digital usage: on/off switch for Boolean logic gates
• The basis of all modern digital computers
• Integrated circuit shrinks billions of these down to a tiny die (chip) you can conceal under your finger
Analog vs. Digital Example: SOM Update
# Adjust weights of winner neighbors
for p in self._neighbors(winner, d):
self.u[p] += alpha*(e-self.u[p])
Part II: Image Convolution
http://wernerstudio.typepad.com/.a/6a00d83549adb153ef01053629c0cf970c-500wi
Convolution:
Convolution is a mathematical way of combining two signals to form a third signal. It is the single most important technique in Digital Signal Processing.*
* http://www.analog.com/media/en/technical-documentation/dsp-book/dsp_book_Ch6.pdf
Convolution: “The Dot Product on Steroids”
Let's look dot product in terms of vectors (arrays):
∑i=1
n
x i w i
x1
x2
x3
xn
...
w1
w2
w3
wn
...* * * *
+
Convolution: “The Dot Product on Steroids”
To get convolution, we “slide” w across x:
x1
x2
x3
xn
...
w1
w2
w3
wn
...* * * *
+y
1
x1
x2
x3
xn
...
w2
w3
w4
wn+1
...* * * *
+y
2
x1
x2
x3
xn
...
w3
w4
w5
wn+2
...* * * *
+y
3
Do you see a problem?
Convolution: “The Dot Product on Steroids”
Usual definition of convolution assumes infinite vectors:
yn= ∑k=−∞
∞
xn−k w k
x3
x2
x1
...
w-1
w0
w1
...* * *
+y
2
...
...x2
x1
x0
...
w-1
w0
w1
...* * *
+y
1
...
... x4
x3
x2
...
w-1
w0
w1
...* * *
+y
3
...
...
Of course, this is unrealistic, so ...
Convolution: “The Dot Product on Steroids”
We treat w as a finite convolution kernel, and switch to non-negative indices: e.g.,:
yn=∑k=0
2
xn−k w k
x3
x2
x1
...
w0
w1
w2
* * *
+y
3
...x2
x1
x0
...
w0
w1
w2
* * *
+y
2
... x4
x3
x2
...
w0
w1
w2
* * *
+y
4
...
Do you notice another (small) problem?
w
x
y
w
x
y
w
x
y
w
x
y
w
x
y
w
x
y c c
w
x
y c c c
w
x
y c c c c
w
x
y c c c c c
Convolution: What's It Goodfer?
• Many useful operations can be expressed as a convolution
• E.g., smoothing, a.ka. moving average, a.k.a. low-pass filtering ….
w
x
y
.33 .33.33
1 1 1 1 10 0 0 0 0
w
x
y
.33 .33.33
1 1 1 1 10 0 0 0 0
.66
w
x
y
.33 .33.33
1 1 1 1 10 0 0 0 0
.66 .33
w
x
y
.33 .33.33
1 1 1 1 10 0 0 0 0
.66 .33 .66
w
x
y
.33 .33.33
1 1 1 1 10 0 0 0 0
.66 .33 .66 .33
w
x
y
.33 .33.33
1 1 1 1 10 0 0 0 0
.66 .33 .66 .33 .66
w
x
y
.33 .33.33
1 1 1 1 10 0 0 0 0
.66 .33 .66 .33 .66 .33
w
x
y
.33 .33.33
1 1 1 1 10 0 0 0 0
.66 .33 .66 .33 .66 .33 .66
w
x
y
.33 .33.33
1 1 1 1 10 0 0 0 0
.66 .33 .66 .33 .66 .33 .66 .33
Convolution: What's It Goodfer?
• Many useful operations can be expressed as a convolution
• E.g., edge detection, a.k.a. high-pass filtering
w
x
y
-1 -12
1 1 1 1 10 0 0 0 0
w
x
y
-1 -12
1 1 1 1 10 0 0 0 0
-2
w
x
y
-1 -12
1 1 1 1 10 0 0 0 0
-2 2
w
x
y
-1 -12
1 1 1 1 10 0 0 0 0
-2 2 -2
w
x
y
-1 -12
1 1 1 1 10 0 0 0 0
-2 2 -2 2
w
x
y
-1 -12
1 1 1 1 10 0 0 0 0
-2 2 -2 2 -2
w
x
y
-1 -12
1 1 1 1 10 0 0 0 0
-2 2 -2 2 -2 2
w
x
y
-1 -12
1 1 1 1 10 0 0 0 0
-2 2 -2 2 -2 2 -2
w
x
y
-1 -12
1 1 1 1 10 0 0 0 0
-2 2 -2 2 -2 2 -2 2
Two-Dimensional Convolution
w
x
Two-Dimensional Convolution
x y
Two-Dimensional Convolution
x y
Two-Dimensional Convolution
x y
Two-Dimensional Convolution
x y
Common Kernels for Image Manipulation
https://en.wikipedia.org/wiki/Kernel_(image_processing)
Common Kernels for Image Manipulation
https://en.wikipedia.org/wiki/Kernel_(image_processing)
Homebrew Convolution with NumPy
Optical Flow
• A basic question: how do we (and other sighted creatures) move around without bumping into things?
• Ecological insight (J.J. Gibson): It can’t involve a complicated computational algorithm in the brain* As much as possible, an animal must be have its senses / nervous system tightly coupled with its environment.
* But, because of the lack of analog optical-flow circuits, nearly everyone still does it with an algorithm / package (OpenCV)!
Hardware Progress
PX4Flow2012$150Inaccurate / difficult to use
PWM39012018$36Accurate / easy to use
The Cutting Edge
https://studywolf.wordpress.com/tag/spiking-camera/