Building a Brain with Raspberry Pi and Zulu Embedded JVM
-
Upload
simon-ritter -
Category
Software
-
view
766 -
download
3
Transcript of Building a Brain with Raspberry Pi and Zulu Embedded JVM
© Copyright Azul Systems 2017
© Copyright Azul Systems 2015
@speakjava
Building A Brain WithRaspberry Pi And
Zulu Embedded JVM
Simon Ritter
Deputy CTO, Azul Systems
1
© Copyright Azul Systems 2017
Hierarchical Temporal Memory
Theoretical framework for how the neocortex works
– Models neurons and synapses and how they interact
– Also includes dendrites and precise synapses
This is different to basic neural network topologies
HTM simulated neuron far more complex
– More adaptable than AI neuron
– links decay over time if not used (temporal)
Potentially much better at learning
5
© Copyright Azul Systems 2017
Learning For Computers
Machine Learning
– Giving “computers the ability to learn without being
explicitly programmed” - Arthur Samuel
– Requires training data to predict future values
7
© Copyright Azul Systems 2017
Learning For Computers
Deep Learning
– “Neural networks with more than two layers”
– Not necessary to have training data
Deep Reinforcement Learning
– No training data
– Learn by experience
– Much like a human: trial and error
8
© Copyright Azul Systems 2017
Markov Decision Process
Mathematical model of decision making
– Outcome partly random, partly controlled by decision
5-tuple
– S: State
– A: Action
– P: Probablility that action a leads to state s
– R: Reward for transitioning
– γ: Discount factor
Difference in importance between future and present rewards
9
© Copyright Azul Systems 2017
Markov Decision Process
10
By waldoalvarez - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=59364518
© Copyright Azul Systems 2017
Deep Q Network
Q: Quantity of a state-action combination
– Learning rate
– Discount factor
– Initial conditions
Used by Deep Mind
to learn to play Atari
games with no previous
experience
© Copyright Azul Systems 2017
Raspberry Pi
Fantastically successful development board
– Cheap, powerful, great connectivity
– Original boards $25, Pi Zero only $5!
Officially launched on Febuary 29th 2012
– First production run was 10,000 boards
– RS reported over 100,000 pre-orders in one day
Crashed servers
– Over 10 million boards have been sold (Sept, 2016)
13
© Copyright Azul Systems 2017
Raspberry Pi Variations
14
Model A Model B Pi Zero
ARM v6
700 MHz
ARMv6/7/8
700/900/1200 MHz
ARM v6
1 GHz
1 core 1 or 4 core 1 core
256 Mb
512 Mb
512 Mb
1 Gb
512 Mb
1 x USB
No Network
2/4 USB
100 Mb Ethernet
WiFi/Bluetooth (v3)
1 micro USB
No ethernet
WiFi/Bluetooth (Zero W)
© Copyright Azul Systems 2017
Pi Brain Architecture
Network
Client
Laptop
Raspberry Pi
Cluster(Emulating HTM)
© Copyright Azul Systems 2017
Cluster Hat
18
USB GPIO
Raspberry Pi 3
Master
PORT 1
PORT 2
PORT 3
PORT 4
Pi Zero W
Slaves
GPIO 22
GPIO 23
GPIO 24
GPIO 25
Alert LED
GPIO 21
© Copyright Azul Systems 2017
Why Java?
“Write once, run anywhere”
– Easy to move code between different platforms
Wide range of libraries and frameworks
– Takes a lot of hard work out of app development
Simple interfacing
– Pi4J and DIO libraries for Raspberry Pi
Simple concurrency
– Comprehensive multi-threading support
Native code integration for complex situations
– JNI20
© Copyright Azul Systems 2017
Azul Zulu Java For Embedded
Built from OpenJDK code base
Passes all TCK tests
Ports for Intel, ARM, PowerPC
ARM v6, v7, v8
– Soft and hard float
– 64 bit close to release
– C2 and C1+C2 compiler enhancements
Drop in replacement for other JVMs
No licensing restrictions (FoU)
21
© Copyright Azul Systems 2017
Deep Learning Basics
Neural network nodes
– Neuron takes weighted inputs that can change
Organised as layers
– Non-linear processing
– Different levels of problem abstraction
– Good analogy to HTM model
Unsupervised learning
– No initial data set
– Learn by experience
23
© Copyright Azul Systems 2017
Neural Network Elements: Layers
25
Output Layer
Hidden Layer
Input Layer
© Copyright Azul Systems 2017
DeepLearning4J (DL4J)
Toolkit for building, training and deploying Neural Networks
– configure neural networks and build computation graphs
ND4J: N-dimensional arrays for Java
– Core technology for deep learning
DataVec
– Transform data into feature vectors
Arbiter
– Used to evaluate and tune ML models
– Hyperparameter optimisation functionality
26
© Copyright Azul Systems 2017
RL4J Basics
Low-dimensional (state) and high-dimensional (pixels)
processing
– MDP class (Markov Decision Process)
– DQN class (Deep Q Network)
– Policy abstract class
Choose the next action given a state
ACpolicy (Actor Critic)
BoltzmannQ (Probablilty distribution)
DQNPolicy (Action with maximum Q value)
EpsGreedy (Epsilon Greedy, partially random)
28
© Copyright Azul Systems 2017
Minecraft
Immensly popular ‘construction’ game
– Especially amongst children
– Different modes: creative, survival, hardcore
PC (first) version written in Java
– 121 million copies sold worldwide (all versions)
– Probably most successful Java desktop app ever
– Mojang aquired by Microsoft in 2014
30
© Copyright Azul Systems 2017
Project Malmo
Microsoft Labs project
AI experimentation platform built on top of Minecraft
– Mod for Java version of Minecraft
– Libraries to write AI agents to act as player in game
Including Java
32
Image courtesy of Microsoft
© Copyright Azul Systems 2017
Malmo Example Code (1)
33
AgentHost host = new AgentHost();ClientPool clientPool = new ClientPool();clientPool.add(new ClientInfo("10.0.0.2"));
MissionSpec mission = new MissionSpec();mission.timeLimitInSeconds(120);mission.requestVideo(640, 480);
© Copyright Azul Systems 2017
Malmo Example Code (2)
34
MissionRecordSpec missionRecord = new MissionRecordSpec("./saved_data.tgz");
missionRecord.recordCommands();missionRecord.recordMP4(20, 400000);missionRecord.recordRewards();missionRecord.recordObservations();
host.startMission(mission, clientPool,missionRecord, 0, "TEST");
© Copyright Azul Systems 2017
Minecraft Interaction
MissionSpec
– setModeToCreative
– startAt/endAt
– drawBlock/drawLine/drawSphere
AgentHost
– startMission
– peekWorldState
– sendCommand
35
© Copyright Azul Systems 2017
Demo Software Architecture
37
LaptopMinecraft with
Malmo mod
Pi Zero 1
(Input)SS RL4J
Pi Zero 2
(Fliter 1)RL4JSS
Pi Zero 3
(Fliter 2)RL4JSS
Pi Zero 4
(Output)RL4JSS
Raspberry Pi 3
Demo Controller
Malmo Library
TCP/I
P
© Copyright Azul Systems 2017
Demo Scenario
Original idea:
– Have agent build a house
– Turns out that’s really hard
Demo revision 1
– Have the agent build a wall
– Simpler, but not much, than building a house
Demo revision 2
– Have agent walk around without getting killed
– Easier reinforcement learning, reward is to stay alive
© Copyright Azul Systems 2017
Applying ML To Minecraft
DL4J/RL4J uses vectors to represent data
– Initially use Minecraft world state
– Long term goal is to use video frames
Have neural network extract state from this
Start with random moves
– Record success/failure (live/die)
Use reinforcement to (hopefully) learn how to navigate
minecraft world safely
39
© Copyright Azul Systems 2017
Conclusions
Embedded hardware can be used for AI/ML/DL
– Model on the structure of the human brain
– Cheap to build multi-core, multi-CPU cluster
Java is an ideal language for this
– Easy to learn
– Simple threading model
– Many, many useful libraries
Minecraft is a fun tool for AI testing and development
41
© Copyright Azul Systems 2017
Future Directions
More work on ML/DL for Minecraft
– Make a better player!
Investigation of jCUDA
– Use NVidia Jetson TK1 board
Use of machine vision
– OpenCV Java bindings
42
© Copyright Azul Systems 2017
Further Information
www.raspberrypi.org
clusterhat.com
deeplearning4j.org
minecraft.net
www.microsoft.com/en-us/research/project/project-malmo
zulu.org
Code to be published on Github after some cleaning up
– Will post a blog about this
43