PyBrain Slides
-
Upload
pablo-loste-ramos -
Category
Documents
-
view
29 -
download
0
Transcript of PyBrain Slides
-
PyBrain
An introduction
-
What's in PyBrain
Neural networks
Reinforcement learning
Black-box optimization
Supervised learning
...
-
General RL framework:
Agent interacts with an EnvironmentTask provides rewardsExperiment connects them
-
An agent generally chooses its actions according to a parametrized Policy .
(by default parameters is a continuous vector)
-
A Learner adapts the parameters of the policy in order to maximize reward.
-
Examples of Environments that work with Tasks:
FlexCubeEnvironmentRollingUpTask
WalkDirectionTaskJumpTask
CaptureGameGoMokuGame
MaximizeWinTaskHandicapTask
MazeEnvironment CheeseMazeTaskTMazeTask
-
A different perspective on the same scenario:Black-box optimization .
-
Class hierarchy for Learners
-
Policies use Modules :black box transform (vector-) input into (vector-) outputtransformation depends on parameters
Modules can be differentiable :
back-propagate output-error towards the inputcompute the partial derivatives w.r.t. each parameter
-
Example Modules:
ActionValueTable SigmoidLayerBernoulliLayerBiasUnitLSTMLayerGateLayerLinearLayer SoftmaxLayer
-
Modules can be combined in Networks :linked with Connections (which can have parameters)Networks are Modules themselvesNetwork-parameters are the concatenation of the component parameters.
-
Example Networks:
FeedForwardNetworkRecurrentNetworkConvolutionalNetworkBidirectionalNetwork
Example Connections:
FullConnectionIdentityConnectionPermutationConnectionSharedConnection
-
Example: A simple Recurrent Neural Network
4 Module components (1 in, 1 out) without parameters4 Connection components with parameters1 of which is a recurrent Connection (pink)
-
# initialize a new networkfrom pybrain import *n = RecurrentNetwork() # add all modules (in any order) n.addOutputModule(SigmoidLayer(2, name='out'))n.addModule(TanhLayer(10, name='h'))n.addModule(BiasUnit(name='bias'))n.addInputModule(LinearLayer(4, name='in')) # add all connections (in any order) n.addConnection(FullConnection(n['h'], n['out'])) n.addConnection(FullConnection(n['bias'], n['h'])) n.addConnection(FullConnection(n['in'], n['h'])) n.addRecurrentConnection(FullConnection(n['h'], n['h'])) # build the network structure n.sortModules() # use it to map an input to an outputoutput = n.activate([0,1,2,-2])
-
General supervised learning framework:
DataSet provides training examples: inputs with associated known targetsModule provides model that estimates targets from inputsTrainer tries to improve the model
-
Another view on the problem shows the analogy to reinforcement learning
-
A DataSet essentially consists of a collection of 2d-arrays (=fields ), holding the patterns to learn from.
Each ML problem solved by PyBrain has a custom DataSet .
Simplest case: SupervisedDataSet
DataSets can be saved/read assembled pattern by pattern constructed from arrays transformed, split, scaled extended with custom fields ...
-
The Trainer's function is to adjust parameters of the Module directly or (most of the time) iteratively, to produce the best model for the data. Schematically, this works as follows:
Module with initial parameters
Module is activated with an input pattern and produces output
Error is calculated by comparing with target, and back-propagated through the module
Parameters are adjusted by some gradient descend or other scheme
-
Further information
Web page: http://www.pybrain.org
Source: http://github.com/pybrain/pybrain
Mailing list: http://groups.google.com/group/pybrain/
Documentation: http://www.pybrain.org/docs/