Ed Safford III MetroCon 2015 Verification, Validation, and Deployment of Hybrid Neuromorphic Systems...
-
Upload
edward-l-s-safford-iii -
Category
Documents
-
view
186 -
download
0
Transcript of Ed Safford III MetroCon 2015 Verification, Validation, and Deployment of Hybrid Neuromorphic Systems...
1
“Verification, Validation, and
Deployment of Hybrid
Neuromorphic Systems”
E.L. (Ed) Safford III
10-22-2015
2
Some Old Ideas Revisited
Robotics
Drones
Pulse Codes – Spikes
Nonlinear Stochastic Theory
Cybernetics
NOT NEW
3
Past
Limited by our machines, we have obtained operational
capability by:
a) Constrained Inputs (Information Loss)
b) Simplified Computational Models
c) Tasks Assigned to Machine
V&V Has Been Relatively Mature for These Systems:
a) Safety
b) Security
c) Mission Critical
We put “Man-in-the-Loop” for Additional Adaptability
and Assurance
4
Future
New Machines Will Provide Better Performance
Systems Will Become Larger and More Complex
Operational Demands Will Increase:
a) Improved System Adaptability
b) Greater Machine Autonomy
c) Better Coverage of “Must Not”
5
V&V Problem
Assurance Methods Have Not
Adequately Progressed to
Support The Full Potential of
These Systems For
Deployment
Therefore We Have Not Been
Able to Delegate Certain
Desired Levels of Operational
Autonomy
6
Idealized Brain Neural Network
- from m1.behance.net
7
Simplified Brain Neuron
SYNAPSES are where Axon termini
form junctions with Dendrites
8
Neuron Model
Weighted Inputs Outputs
Activation Function
Summation
Neuron
9
Artificial Neural Networks
1. Composed of one or more “layers” of
“neurons”, i.e. neuron models, with nodes and
connections
2. There is an input layer, an output layer, and
one or more additional layers called “hidden”
layers. May have feedback loops.
3. Mathematically can be represented by
composition of functions with known or
unknown parameters. DEs for feedback.
Run Modes: Estimation and Approximation
10
Diagram
y = f(x) = WTσ(VTx) + ε(x)
x0 = 1
x1
x2
xn
yn
y1
.
.
.
.
.
.
.
.
.
11
Neuromorphic (NM) Systems
“ … hardware specifically designed for
machine learning or inspired by biological
neural networks” (1)
“Brain Inspired” or “brain-like” in its
function (2)
1) Appavoo et al, “programmable Smart Machines: A Hybrid Neuromorphic
approach to General Purpose Computation”,
http://people.bu.edu/schuye/files/appavoo-neuroarch-2014.pdf
2) Mohda, D. S., “Introducing A Brain Inspired Computer”, refers to article
published in Science 8 August 2014,
http://www.research.ibm.com/articles/brain-chip.shtml
12
Hybrid Neuromorphic Systems Can be very complex combinations of digital and
analog devices (hardware and software) that
include components which simulate or emulate
neurological function in order to learn and perform
the required tasks.
A broader definition for neuromorphic computing,
covering those systems which include both von
Neumann (VN) and non-VN architectures.
Calimera, Andrea et al, “The Human Brain Project and neuromorphic
computing”,http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3812737/pd
f/191-196.pdf
13
Research
And Technology Advances
14
• Henry Markram EPFL
Lausanne, Switzerland An HBP Pilot (downselect 2012)
Patch clamp study for model data
• Neuroscience Modeling Uses IBM Blue Gene Computer
Michael Hines’s NEURON sw
Blue Brain 2005-2012
15
SyNAPSE is backronym – Neuromorphic
Adaptive Plastic Scalable Electronics
DARPA Program to develop
neuromorphic machine technology that
scales to biological levels
Substantial Funding went to HRL & IBM
SyNAPSE
16
Hewlett-Packard Research Lab
(HRL) Dec 2011 CMOS chip
- Simultaneous memory storage and
logic processing
- Multi-bit fully-addressable memory
with up to 30 Gbits/sq cm
MEMRISTOR Arrays
17
Massively Parallel Cortical
Simulation on Blue Gene/P (Dawn)
Supercomputer at LLNL 2009
- 1.6 Billion Neurons, 8.87 Trillion
Synapses
- Spiking Neurons (STDP) and axon
delay (0.1 ms simulated time step)
IBM “Cat Brain” Sim C2
18
Lab Prototype Digital Neurosynaptic
Core Processor and Integration
Board Announced Aug 2011
- 256 leaky integrate and fire (LIF)
neurons on 45 nm SOI CMOS hw
- deterministic hw with ~1 kHz chip
clock (emulates ~1ms bio time-step)
IBM New uProcessor Core
19
Neuromorphic Architecture Using
Simple LIF Spiking Neurons and
“Binary” Synapses - May 2012
And arranged in layers like brain, BUT
- Synaptic homeostatic renormalization
- Burst spiking time dependent plasticity
(Burst STDP)
New Implementation and Models
25
System Capabilities
• Robots
• - Baxter
• - Brett
• Big-Data-Mining Engines
• Self-Completing Software
• - PLINY, Brain####
• Autonomous Vehicles
• - Google Cars
• - Quadcopters
• Natural Language Processing
• Feature Recognition and Tracking
26
Model-Based V&V “was” Good Enough
Systems were simpler in their complexity
and autonomy
They could be white and black box tested at
points of homeostasis
And …
“A Meta Model could be constructed
that defined acceptable system states and
behaviors”
- Blackburn
27
Then What Happened?
Very Complex Learning Systems
Dramatic Emergent Behavior
28
Autonomous Robot Learning
29
A “Disruptive” System
Technology Announcement
And COMPASS
TN Simulator
June 2014
30
IBM TrueNorth
Video
And/or
Deep Dive
Summer 2015
31
• They Learn From a Teacher
• They Learn From Each Other
• They Learn From Their Environment
• Deep Learning and Reservoir Computing
Emergent Behaviors Must Be Controlled
• New Methods of V&V are Needed to
Achieve Desired Assurance Levels
The Machines Learn
32
Problem Re-Statement
The (Stochastic) Complexity of the Systems
We Are Now (and Will Continue to Be)
Constructing and Deploying (e.g. Reservoir
Computing and Liquid State Machines) Is
Rapidly Exceeding Our Ability To
Adequately and Feasibly Verify, Validate,
and Control Them
33
A neural network can be trained as a test
“oracle” or model of the original system to
determine whether a given test case exposes
a fault or not. - Dale Boren
A firm foundation for the use of neural
networks in feedback control systems has
been developed over the years by many
researchers. - F. L. Lewis
There Are Concepts
34
1. Blackburn, M., Denno, P., Virtual Design and Verification
of Cyber-Physical Systems: Industrial Process Plant
Design, Procedia Computer Science, Volume 28, 2014,
Pages 883-890,
http://www.sciencedirect.com/science/article/pii/S187705
0914000696
There Are Studies
35
There Are Experiments
36
There Are Solicitations
37
Neurogrid
- Stanford
SpiNNaker (Spiking NN Arch.)
- University of Manchester
TrueNorth (Compass, SyNAPSE)
- IBM
There Are Platforms
38
Trainable Testbed Concept
39
Trainable Testbed Benefits
40
IT IS TIME
For the Trainable Testbed