Biologically Inspired Computing: The DARPA SyNAPSE Program ...

Portland State University Portland State University

PDXScholar PDXScholar

Systems Science Friday Noon Seminar Series Systems Science

2-26-2010

Biologically Inspired Computing: The DARPA Biologically Inspired Computing: The DARPA

SyNAPSE Program & The Hierarchical Temporal SyNAPSE Program & The Hierarchical Temporal

Memory Memory

Dan Hammerstrom Portland State University, [email protected]

Follow this and additional works at: https://pdxscholar.library.pdx.edu/systems_science_seminar_series

Part of the Computer Sciences Commons

Let us know how access to this document benefits you.

Recommended Citation Recommended Citation Hammerstrom, Dan, "Biologically Inspired Computing: The DARPA SyNAPSE Program & The Hierarchical Temporal Memory" (2010). Systems Science Friday Noon Seminar Series. 36. https://pdxscholar.library.pdx.edu/systems_science_seminar_series/36

This Book is brought to you for free and open access. It has been accepted for inclusion in Systems Science Friday Noon Seminar Series by an authorized administrator of PDXScholar. Please contact us if we can make this document more accessible: [email protected].

https://pdxscholar.library.pdx.edu/

https://pdxscholar.library.pdx.edu/systems_science_seminar_series

https://pdxscholar.library.pdx.edu/sysc

https://pdxscholar.library.pdx.edu/systems_science_seminar_series?utm_source=pdxscholar.library.pdx.edu%2Fsystems_science_seminar_series%2F36&utm_medium=PDF&utm_campaign=PDFCoverPages

http://network.bepress.com/hgg/discipline/142?utm_source=pdxscholar.library.pdx.edu%2Fsystems_science_seminar_series%2F36&utm_medium=PDF&utm_campaign=PDFCoverPages

http://library.pdx.edu/services/pdxscholar-services/pdxscholar-feedback/

https://pdxscholar.library.pdx.edu/systems_science_seminar_series/36?utm_source=pdxscholar.library.pdx.edu%2Fsystems_science_seminar_series%2F36&utm_medium=PDF&utm_campaign=PDFCoverPages

mailto:[email protected]

1

Maseeh College of Engineering and Computer Science 2/26/10 1

Biologically Inspired Computing:

- The DARPA SyNAPSE Program & - The Hierarchical Temporal Memory

Dan Hammerstrom Electrical And Computer Engineering Portland State University

Maseeh College of Engineering and Computer Science Hammerstrom 2/26/10 2

Intelligent Computing

  There is one class of problems that we still do not solve well

  These problems involve the interaction of a computing system with the real world

  Which, in part, involves a transformation and understanding of data at the boundary between the real world and the digital world

  These problems occur wherever a computer is interacting with the real world – which includes almost every embedded application

  An interesting opportunity for specialized hardware

2


Our Focus: Intelligent Signal Processing (ISP)   ISP augments and enhances traditional DSP (Digital Signal Processing) by

incorporating contextual and higher level knowledge of the application domain into the data transformation process

Front End Signal

Processing

Feature Extraction Classification

Contextual Semantic Inference

Motor Control

Motor Control

Subprogram

Motor Control

Programs

Decision Making

The “Front End” - DSP The “Back End” - ISP


The “Front End”

  Front end processing is well understood, it is the realm of traditional digital signal and image processing

  Front end algorithms generally apply the same computation over large arrays of elements, they are data parallel, and communication tends to be local

  An excellent example of such an architecture is the CNN (Cellular Non-linear Network) developed by Chua, Roska et al.

  Most “neuromorphic” VLSI operates at the front end

3


But Then There’s The “Back-End” …

  In the early days of computing, “Artificial Intelligence” focused on the representation and use of contextual and semantic information

  Knowledge was generally represented by a set of rules

  However, these systems were “brittle,” exhibiting limited flexibility, generalization, and graceful degradation

  They did not scale

  And they were unable to adapt dynamically (i.e., learn) within the context of most real world applications


The ISP Toolbox – Still mostly empty after all these years …

4


Some Desirable ISP Characteristics

  Solving the problem   Massively parallel and low precision   Self-organizing – in fact, system design becomes more the provisioning of

organizing principles (Prof. Christoph von der Malsburg), than the specification of all operational aspects of the models   www.organic-computing.org

  Generalization, and graceful degradation   Low power - the processing power of the brain is roughly 1015 operations

per second which it accomplishes at a power dissipation of about 25 watts   Scales - The scaling limitations of both symbolic and traditional neural

network approaches constitute one of their biggest shortcomings   Adaptive - Consequently another important characteristic of real systems is

incremental, integrative adaptation or learning during system operation

Maseeh College of Engineering and Computer Science Hammerstrom

The Scope of Our Project

  Conceptually one can think of “computational intelligence” as a spectrum   And though not universally accepted, it has been hypothesized that this

spectrum is more or less continuous from one end to the other

2/26/10 8

Big “C” Cognition: humans

Small “c” cognition: most mammals

Computing Is currently

here

An even longer way!

ISP

Increasing Intelligence

Our Goal

A long way!

5


  There is increasing interest in using models from Computational Neuroscience, in particular cortical models as inspiration for new models of computation in general and intelligent computing in particular

  In Europe there is FACETS   The goal of the FACETS (Fast Analog Computing with Emergent Transient

States) project is to create a theoretical and experimental foundation for the realisation of novel computing paradigms which exploit the concepts experimentally observed in biological nervous systems.

  And in the US there is the DARPA SyNAPSE Program

2/26/10 9

Approved for Public Release, Distribution Unlimited 10

Systems of Neuromorphic Adaptive Plastic Scalable Electronics

Dr. Todd Hylton, Program Manager DARPA DSO

6


IC Transistor µProcessor & memory

Programmable machines

• End of scaling • Defect intolerant • Architectural bottleneck • Software limited • No path to biologically

competitive intelligence 60 years

•  Increased component density •  Increased component function • Defect tolerant • Neuromorphic information,

learning, cognition, understanding architecture

• Path to biologically competitive intelligence

<<60 years

The SyNAPSE program seeks to extend the development of modern electronics into a new revolutionary new era using a similar paradigm.

“Cortical” Microcircuit

Electronic Synapse

“Cortex” Fabric

Intelligent machines

DARPA SyNAPSE

Historical Evolution of Modern Electronics

Vision for the Future


Program Approach

Measure Make

Model

Employ theoretical and empirical approaches constrained by practicality.

Hardware

Architecture

Simulation

Environment

Sponsor a suite of complementary capabilities to build, train, and evaluate devices.

Attack the problem “bottom-up” and “top-down” and force disciplinary integration with a common set of objectives.

Top-down (simulation)

Bottom-up (devices)

Biological Scale Machine Intelligence

Materials (e.g. memristors)

Components (e.g. synapse / neuron)

Circuits (e.g. center-surround)

Networks (e.g. cortical column)

Modules (e.g. visual cortex)

System (SyNAPSE)

7


Program Outline Phase 1 Phase 2 Phase 3 Phase 4

Emul

atio

n &

Sim

ulat

ion Simulate large

neural subsystem dynamics

“Mouse” level benchmark (~ 106 neuron)

Har

dwar

e Component synapse (and neuron) development

CMOS process and core circuit development

~106 neuron single chip implementation “Mouse” level

Arc

hite

ctur

e &

Too

ls

Microcircuit architecture development

~106 neuron design for simulation and hardware layout

~108 neuron multi-chip robot at “Cat” level

~108 neuron design for simulation and hardware layout

“Cat” level benchmark (~ 108 neuron)

Build Sensory, Planning and Navigation environments “Small mammal” complexity En

viro

nmen

t

Comprehensive design capability

Phase 0

CMOS process integration

System level architecture development

Add Audition, Proprioception and Survival “All mammal” complexity

Add Touch and Symbolic environments

Sustain

Preparatory studies only

Preparatory studies only

Currently only Phases 0 and 1 have been funded We are now In Phase 1


  There are three contractors:   IBM - Dharmendara Modha   HP – Greg Snider   HRL (formerly the Hughes Research Lab) – Narayan Srinivasa

  The ultimate goal is to build a low-power, compact electronic chip combining a novel analog circuit design and a neuroscience-inspired architecture that can address a wide range of cognitive abilities—perception, planning, decision making and motor control

  "Our research progress in this area is unprecedented," says DARPA program manager Todd Hylton, Ph.D. "No suitable electronic synaptic device that can perform critical functions of a biological brain like spike-timing-dependent plasticity [an indicator of the capability to learn] has ever before been demonstrated or even articulated.”

2/26/10 14

8


HRL Team

  Hardware:   Analog circuits, HRL   Nano-devices, Wei Lu, Univ. of Michigan   Floating gate devices, Paul Hasler, Georgia Tech   Systems integration and global communication, Dan Hammerstrom, Portland State

  Neuroscience   Steve Grossberg, Boston University   Eugene Izhikevich, Jason Fleischer, et al., Neurosciences Institute   Jeff Krichmar, UC Irvine   Phil Goodman, University of Nevada, Reno   Giorgio Ascoli and Alexei Samsonovich, George Mason

  All three teams have completed phase 0 and are now in the middle of phase 1, which is scheduled to end in December 2010

2/26/10 15

16

NEUROMORPHIC NETWORKS (“CROSSNETS”)

wjk = {-1, 0, +1}

Generic structure of a feedforward CrossNet S. Fölling et al. (2001) O. Turel et al. (2004)

Basic idea: CMOS “somas” + nanowire “axons” and “dendrites” + nanodevice “synapses”

soma j

soma k

jk+

jk- +

+ -

-

j i wij

Likharev

9

17

Structural View of Mixed-Signal CMOL Design (Each CP) - Gao

C. Gao

2/26/10 Work performed by HRL under DARPA

contract HRL0011-09-C-001 18

10

Maseeh College of Engineering and Computer Science

However …

2/26/10 19


The “Gap”   So do neural techniques lead to

advanced ISP?

  Most Computational Neuroscience is weak in making the jump from spiking neurons with learning rules such as STDP to Cognition   The SyNAPSE program has this

problem

  Even solutions to the more narrowly defined ISP back-end problem are not obvious

2/26/10 20

Neurons

Cognition

“Then a Miracle Occurs”

11


Modular Intermediate Form

  One way to possibly bridge the gap is to define a computational model that “spans” the gap

  A candidate has been proposed by Albus and many others:

  The Cortical Computation Unit (CCU)

2/26/10 21

Neurons

Intelligent Computation

Intermediate Model


Desirable Characteristics

  Modular   Distributed representation   Hierarchical, bi-directional information flow   Massively parallel   Scales   Learning / Self-organizing   Does a kind of Bayesian inference

  Solves the problem …

2/26/10 22

12

•  overall system level (central nervous system)

•  arrays of macro-computational units (e.g., cortical regions)

•  macro-computational units (e.g., cortical hypercolumns & loops)

•  micro-computational units (e.g., cortical microcolumns & loops)

•  neural clusters (e.g., spinal and midbrain sensory-motor nuclei)

•  neurons (elemental computational units) – input/output functions

•  synapses (electronic gates, memory elements) – synaptic phenomena

•  membrane mechanics (ion channel activity) – molecular phenomena

Albus: What is the path to success for reverse engineering the brain?

Pick the right level of resolution

AI and Cognitive Neuroscience

Mainstream Neuroscience & Neural Nets

CCUs

Slide courtesy James Albus, “Reverse Engineering the Visual System” From the PSU / Intel / ONR / NSF “Massively Parallel, Adaptive Computing” Workshop, March 2009


Modular Hierarchies

  Computer engineers make extensive use of modular, hierarchical design, can we assume the same for these models?

  In neocortex the fundamental unit of computation appears to be the cortical minicolumn (Mountcastle)   A minicolumn is a vertically organized group of about 80-100 neurons which

traverses the thickness of the gray matter (~3 mm) and is about 50µ in diameter   Neurons in a column tend to communicate mostly vertically with other neurons in

the different layers in the same column

  These are subsequently organized into larger columns variously called just “columns”, “cortical columns”, “hypercolumns”, or sometimes “modules”   Note, columnar organization is not universally accepted in the neuroscience

community

13


“Bayesian Memory” (BM) Building Block

  An approximation to a CCU   A BM sees only a subset of its

input BMs and each BM’s subset is slightly different

  Inference is performed over small sub-blocks

  The number of blocks increases linearly

  Relies heavily on sparse, distributed representations

2/26/10 25

BM BM BM BM

BM BM BM BM

BM BM BM BM

BM BM BM BM


From Big Brain by Gary Lynch and Rick Granger (Palgrave McMillan 2008):

  “Although the ‘front end’ circuits of the brain, with their point-to-point circuit designs, specialize in their own particular visual and auditory inputs, the rest of the brain converts these to random-access encodings in association areas throughout cortex. … these areas take initial sensory information and construct grammars

  “These are not grammars of linguistic elements, they are grammatical organizations (nested, hierarchical, sequences of categories) of percepts – visual, auditory, and other

  “Processing proceeds by incrementally assembling these constructs … these grammars generate successively larger ‘proto-grammatical fragments,’ eventually constituting full grammars

2/26/10 26

14


  “They thus are not built in the manner of most hand-made grammars; they are statistically assembled, to come to exhibit rule-like behavior, of the kind expected for linguistic grammars

  “Proto-grammatical fragments capture regularities that are empirically found to suffice both for recognizing and generating grammatical sequences

  “Auditory pathways in our brains grew and lengthened building voice-sounds into words, words into phrases, phrases into sentences”

2/26/10 27

Maseeh College of Engineering and Computer Science Hammerstrom 28

  Information needs to flow both ways   Assume that conditional probabilities / priors - model the world   Bi-Directional Belief propagation – e.g., visual cortex model   Inference as the basic computation

Lee and Mumford Visual cortex model

15


A BM module then has two parts:

  An input that approximately learns the probability distribution of its inputs   A table of vectors, which is called a codebook and implements Vector functionality   Approximates the input probability distribution   Learns in an unsupervised manner by allocating new vectors and/or moving

existing vectors

  A Vector Quantizer is an example of such a function – an “entropy” maximizing data reducer

2/26/10 29


  An output that creates a new representation of the codebook vectors to send up the hierarchy   A table of vectors, one for each codebook vector   The output vectors are sparse and are of a higher dimension than the space they

span   In some implementations they are random, in Numenta’s HTM they just pass up

the index of the winning codebook vector   Ideally they would self-organize, as in a Self-Organizing Map (SOM) to capture

the one or two dimensions of highest invariance of the input space

2/26/10 30

16


  The winning codebook vector is the most likely given some input

  W is some vector weight

  In VQ terms it specifies the width of the region surrounding the codebook vector

  It can be thought of as the “prior” probablities

Input Vector W i Output Vector

1000100111000 0.12 0 10010101000

0001101001010 0.05 1 01011100110

.

.

.

.

.

.

.

.

.

.

.

. 0111010101000 0.001 n 10001000100

Evidence for Input Vectors

Winning Output Vector


Belief Propagation

  The model looks good, but something missing

  It is generally assumed that biological systems perform a kind of inference over the knowledge they have learned

  If so, then perhaps Bayesian Inference   Probabilistic Reasoning in Intelligent Systems: Networks of Plausible

Inference, by Judea Pearl, Morgan Kaufmann, 1988, ISBN-10: 1558604790

  Assume that our modular hierarchy is a directed Bayesian network   Vertices are objects which have local information and carry out local

computations by updating of probability distribution via message passing

2/26/10 32

17


Most Likely Computation – Influence From Above

  OK, I lied, the computation of the most-likely codebook vector is actually more complicated

  The reason is that it is a result of the influence of “belief” propagated both from above and from below

Input Vector W i Output Vector

1000100111000 0.12 0 10010101000

0001101001010 0.05 1 01011100110

.

.

.

.

.

.

.

.

.

.

.

. 0111010101000 0.001 n 10001000100

Input Vector Evidence Winning Input Vector

Winning Output Vector Evidence for Output Vectors

Vector from upper BM

Input Vector


A Simple Bayesian Network

P(d|b,c) d1 d2

b1, c1 0.5 0.5

b2, c1 0.3 0.7

b1, c2 0.9 0.1

b2, c2 0.8 0.2

CPT for node D, there are similar tables for B and C

CPT is “Conditional Probability Table”

18


Pearl’s Belief Propagation

V

U2

V1 V2

U1

π(U2)

π(V1) π(V2)

π(U1)

λ(U1)

λ(V2)

λ(V1)

λ(U2)

Singliar Slides


The Evidence   Evidence – values of observed nodes

  V3 = T, V6 = 3   Our belief in what the value of Vi

‘should’ be changes   This belief is propagated   As if the CPTs became

V1

V5

V2

V4

V3

V6

V3=T 1.0 V3=F 0.0

P V2=T V2=F V6=1 0.0 0.0

V6=2 0.0 0.0

V6=3 1.0 1.0

Slides by Tomas Singliar, CMU

19


The π Messages   What are the messages?   For simplicity, let the nodes be binary

V1

V2

V1=T 0.8 V1=F 0.2

P V1=T V1=F V2=T 0.4 0.9

V2=F 0.6 0.1

The message passes on information

What information? Observe:

P(V2| V1) = P(V2| V1=T)P(V1=T)

+ P(V2| V1=F)P(V1=F)

The information needed is the CPT of V1 = πV(V1)

π Messages capture information passed from parent to child

Singliar Slides


The λ Messages   What about λ?

  The messages are π(V)=P(V|E+) and λ(V)=P(E-|V)

V1

V2

Assume E = { V2 } and compute by Bayes rule:

The information not available at V1 is the P(V2|V1). To be passed upwards by a λ-message. Again, this is not in general exactly the CPT, but the belief based on evidence down the tree.

Singliar Slides

20


Combination of evidence

  α is the normalization constant   normalization is not necessary (can do it at the end)   but may prevent numerical underflow problems

Singliar Slides


Messages to pass

  We need to compute πXY(x)

  Similarly, λXY(x), X is parent, Y child   Symbolically, group other parents of Y into V = V1, … , Vq

Singliar Slides

21


The Pearl Belief Propagation Algorithm

  Iterate until no change occurs   (For each node X) if X has received all the π messages from its parents, calculate π(x)   (For each node X) if X has received all the λ messages from its children, calculate λ(x)   (For each node X) if π(x) has been calculated and X received all the λ-messages from all its

children (except Y), calculate πXY(x) and send it to Y.   (For each node X) if λ(x) has been calculated and X received all the π-messages from all

parents (except U), calculate λXU(x) and send it to U.

  Compute BEL(X) = λ(x)π(x) and normalize

Singliar Slides


Most Graphs are not Polytrees

  Cutset conditioning   Instantiate a node in cycle, absorb the value in child’s CPT   Do it with all possible values and run belief propagation   Sum over obtained conditionals   Hard to do

  Need to compute P(c)   Exponential explosion - minimal cutset desirable (also NP-complete)

  Clustering algorithm   Approximate inference

  Sampling methods   Loopy BP

Singliar Slides

22


Now To Add BBP to The BM

  A node in our hierarchy then represents a variable and is part of a larger, acyclic graph

  Child regions Y1 and Y2, parent U

Voilá - A “Bayesian Memory”

23

Neural Network Equivalent of BBP-PA   For 4K CB entries   No. Neurons ~ 32e3   Synapses ~ 34e6   NN derived from Hawkins’ paper

Maseeh College of Engineering and Computer Science

HTM – Hierarchical Temporal Memory – Version 2

2/26/10 46

24


Numenta   The HTM algorithm is the work of Jeff Hawkins and Dileep

George

  Jeff (Palm Pilot inventor) founded the Redwood Neuroscience Institute, http://redwood.berkeley.edu

  From which has emerged a synthesis of a number of existing and new ideas of cortical operation

  These are highlighted in his book, “On Intelligence”

  The models have worked so well that he has now spun out a company, Numenta, Inc., www.numenta.com

  Our work has borrowed heavily from Jeff and Dileep


  Based on neuroscience principles, Jeff proposed that Cortex performs the following:

1.  Learns sequences of patterns 2.  Operates auto-associatively 3.  Captures invariants 4.  Is organized hierarchically, and 5.  Based on fundamental Bayesian principles

  The George / Hawkins model starts with a fairly general Bayesian module, very similar to the BM presented earlier   “A Hierarchical Bayesian Model of Invariant Pattern Recognition in the Visual

Cortex,” D. George and J. Hawkins, Proceedings of the ICJNN 2005

  These modules then are combined into a hierarchy to form the Numenta Hierarchical Temporal Memory (HTM)

25


  Hierarchical -- HTMs are organized as a tree-shaped hierarchy of nodes. Each node implements a learning and memory function, that is, it encapsulates an algorithm   Lower-level nodes receive large amounts of input and send processed input up to the next

level   In that way, the HTM Network abstracts the information as it is passed up the hierarchy

  Temporal -- During training, the HTM application must be presented with objects as they change over time   For example, during training of the Pictures application, the images are presented first top to

bottom, then left to right as if the image were moving over time   Note that the temporal element is critical: The algorithm has been written to expect input that

changes gradually over time

2/26/10 49


  Memory -- An HTM application works in two stages, which can be thought of as training memory and using memory   During training, the HTM Network learns to recognize patterns in the input it receives. Each

level in the hierarchy is trained separately   In the fully trained HTM Network, each level in the hierarchy knows -- has in memory -- all the

objects in its world   During inference, when the HTM Network is presented with new objects, it can determine the

likelihood that an object is one of the already known objects.

2/26/10 50

26

2/26/10 51 Courtesy Dileep George 2009

George D, Hawkins J, 2009, “Towards a Mathematical Theory of Cortical Micro-circuits,” PLoS Comput Biol 5(10):e1000532 doi:10.1371/journal.pcbi.1000532


27



28



29



  Hierarchy in space and time   Evidence for biology abstracting in space and time as signals proceed up the

hierarchy   Feed-forward and feedback connections   Common cortical algorithm   Inference using Bayesian belief propagation   Sparse Distributed Representations   Prediction using temporal context   Biologically accurate

  Several computational vision applications   www.numenta.com

2/26/10 58

30


Kanisza Square Illusion …


Mapping To Biology …

  In the PLOS paper, “Towards a Mathematical Theory of Cortical Micro-circuits,” they also speculate on mapping the algorithms to cortical circuitry   Use known facts about cortical organization to map belief propagation to cortical

layers   “The vertical dimension of the cortical rectangle is only a few layers deep, the

horizontal dimension is variable”   The states of the region are represented by neurons along the horizontal

dimension of the cortical region   They then divide the horizontal dimension of the cortical region into a number of

Compartments, where each compartment corresponds to a particular state of the region

  This subdivision corresponds to a columnar organization of the cortex

31


2/26/10 62 Courtesy Jeff Hawkins 2009

32


  It is an interesting time!

  SyNAPSE probably won’t meet its original goals, but it will push the field forward – assuming there is no catastrophic failure – or too much hype …   IEEE Tech Blog, “Cat Fight Brews Over Cat Brain”

  I personally believe that Jeff is on the right track   And he is in this for the long haul, like the Terminator he will just keep

on attacking this problem …   And for the work I do, there are all kinds of interesting hardware

possibilities – especially with nano and molecular electronics!

2/26/10 63


A Path From Nanowires to ISP

2/26/10 64

Intelligent Signal Processing

Hierarchical Bayesian Network

Modular Bidirectional Spiking Associative Memory

Mixed Signal Nano-Scale Devices

  Our approach is top-down, not bottom up

  There is a large range of implementation options   1000 Atom processors   Neuromorphic VLSI   Nano-grids   Other nano …

33

2/26/10 65

Our Goal – A Commercial Product: The Field Adaptable Bayesian Array (FABA)

Nanoscale Analog

Associative Memory

Nanoscale Analog

Associative Memory

Nanoscale Analog

Associative Memory

Nanoscale Analog

Associative Memory

Each Square is a single Bayesian Memory Node

CMOS provides sparse inter-module connectivity, I/O, signal amplification

Thousands of of nodes with full connectivity

Bayesian Memory Inside!


FABA – Long Term Goal

  A roughly 1 inch die containing several billion CMOS transistors and close to a trillion molecular devices

  Operating at over 10 Tera-Ops   Extensive fault / defect tolerance   Performs real-time, adaptive bayesian inference over very complex spatial

and temporal knowledge structures   Available in a portable, hand-held, low power devices

Biologically Inspired Computing: The DARPA SyNAPSE Program ...

Documents

Transcript of Biologically Inspired Computing: The DARPA SyNAPSE Program ...