Test Driven Neural

Networks

Matthew Kirk - Modulus 7

automatic playlists

The Challenge

“Big Data” Language

sJavaPythonRJuliaClojure

Ruby has tools too

Ruby is not Complex Math

Today we’ll cover

• What feed forward neural networks are

• How to classify strings to languages using Neural Nets

• How to do it in a TDD fashion

• Demonstration

Neural Networks!

aka the sledge hammer of functional relationships

Neural Networks

Input layer

Hidden layer

How many Neurons?

•2/3 * Input layer count + output count is a good start

•Aggregation over expansion so less neurons in the hidden layer than on the input layer.

Output layer

Neurons

Digital Logic

Fuzzy Logic

Activation Functions

•Sigmoid

•Elliott

•Gaussian

•Linear

•Threshold

•Cosine and Sine

Activation Functions

•Sigmoid => Learning Curve

•Elliott => Learning Curve

•Gaussian => Bell curve

•Linear => Line

•Threshold => Yes or No

•Cosine and Sine => Periodic

Training Algorithms

•Quickprop

•RProp => Use this

•Back propagation

Visually What they do

Neural NetsJust the tip of the iceberg

Specifically

•English

•German

•Polish

•Swedish

•Finnish

•Norwegian

Data Collection

•Using the most translated book in the world “The Bible” to collect sentences used in each of these languages.

Now What?

Character Distribution

TDD Neural Nets

Test the Seams

describe Language do it 'has the proper keys for each vector'

it 'sums to 1 for all vectors'

it 'returns characters that is a unique set of characters used'end

Cross Validation

describe Network do %w[English Finnish German Norwegian Polish Swedish].each do |lang| it "Trains and cross-validates with an error of 5% for #{lang}" endend

Ockham’s Razor

Demo

modulus7.com/rubyconf

@mjkirk

Conclusion

This is just the beginning

Go learn more become more adept at data analysis

http://rickmanelius.com/article/do-you-dread-emails

http://www.flickr.com/photos/irisheyes/8469160004/

http://www.flickr.com/photos/andy_bernay-roman/2206610268/

http://www.flickr.com/photos/kev_walsh/2216144544/sizes/o/in/photostream/

http://www.flickr.com/photos/clover_1/2926385130/

http://www.flickr.com/photos/andy_bernay-roman/2206610268/sizes/o/in/photostream/

http://translate.google.com

http://www.flickr.com/photos/epistemographer/68200471

http://www.allaboutcircuits.com/vol_4/chpt_3/5.html

http://www.flickr.com/photos/brunobord/3987593006/

Photo Credits