Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data...
-
Upload
data-science-milan -
Category
Data & Analytics
-
view
761 -
download
0
Transcript of Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data...
![Page 1: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/1.jpg)
An introduction to neural nets
by Valentino [email protected]
[email protected]+39 333 789 2692+1 202 640 1381
![Page 2: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/2.jpg)
Deep Learning in AIDigitalisation
Features
Classification/Prediction
AnalysisInteraction
The World
Sensor
Machine Learning
![Page 3: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/3.jpg)
1642• Pascaline: first mechanical adder, invented by
French mathematician Blaise Pascal, using a system of gears and wheels could add and subtract numbers.
![Page 4: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/4.jpg)
1694• Machine by Gottfried Wilhelm Von Leibniz
who also developed calculus and invented the binary system. His machine could also multiplicate and divide.
![Page 5: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/5.jpg)
1801• Joseph Marie Charles invents the Jacquard
loom to weave different patterns using cards punched with holes. Precursor for modern computers and data storage.
![Page 6: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/6.jpg)
1890• Herman Hollerith, founder of the Tabulating
Machine Company (later merged into IBM), creates a mechanical tabulator using punched cards to store data to calculate statistics. He is regarded as the father of modern machine data processing.
![Page 7: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/7.jpg)
1957• Frank Rosenblatt invents the perceptron
algorithm, the first neural networks implementation. It was later proved by Marvin Minsky and Seymour Papert in 1969 that it could not learn the XOR function.
![Page 8: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/8.jpg)
1974• Paul Werbos’s Ph.D. thesis describes the
process of training neural nets through back-propagation.
![Page 9: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/9.jpg)
Supervised Learning
1. Input Data2. Process the information3. Check the output4. Learn new rule5. New rule is applied to better performance
![Page 10: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/10.jpg)
Neural Networks
The theory of neural networks arises from the attempt to mimic our biological brain in order to create machines that can learn or perform pattern recognition in order to make predictions.
![Page 11: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/11.jpg)
Neural Networks
Neural networks are systems comprised of many ”neurons”, which are the units of the neural net.
![Page 12: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/12.jpg)
Neural Networks
In neural networks, a space of ”weights” is defined alongside the input space. The weights and the input together define the activity rules that in turn will define the output according to specified activation functions.
![Page 13: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/13.jpg)
Neural Networks
The weights can change with time as the neural network learns and their change may be specified by some learning rule which will generally be depending on the activities of the neurons.
![Page 14: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/14.jpg)
Perspective
• Biggest artificial neural network to-date: Over 11 billion parameters. (1.1 * 10ˆ10).
• Number of neurons in brain 10ˆ11, each with about 10ˆ4 connections for a total of 10ˆ15 parameters.
![Page 15: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/15.jpg)
A model for a neuron
Images from upcoming book "Python Deep Learning"
![Page 16: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/16.jpg)
Neural Networks
x w wx
![Page 17: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/17.jpg)
Neural Networks
x w1
w1x+w2y
y w2
![Page 18: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/18.jpg)
Neural Networks
x w1
w1x+w2y+b
y w2
1
b
w1x+w2y+b > 0
w1x+w2y > -b
![Page 19: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/19.jpg)
Neural Networks
x w1 σ(w1x+w2y+b)
y w2
1
b
![Page 20: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/20.jpg)
Neural Networks
![Page 21: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/21.jpg)
The Universal Approximation Theorem
Neural networks with a single hidden layer can be used to approximate any continuous function to any desired precision.
![Page 22: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/22.jpg)
A logistic sigmoid: 1/(1+exp(-x))
In general: 1/(1+exp(-wx-b))
![Page 23: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/23.jpg)
Why the Universal Approximation theorem is true
![Page 24: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/24.jpg)
A step function
![Page 25: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/25.jpg)
Picture by John Kaufhold
Classic Neural Networks
![Page 26: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/26.jpg)
Deep neural networks in the 80’s
Recent Developments in Deep Learning by Geoff Hintonhttps://www.youtube.com/watch?v=vShMxxqtDDs
![Page 27: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/27.jpg)
Learning Representations
Neural Networks for Machine Learning by Geoff Hintonhttps://class.coursera.org/neuralnets-2012-001/lecture
![Page 28: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/28.jpg)
Learning Representations
Neural Networks for Machine Learning by Geoff Hintonhttps://class.coursera.org/neuralnets-2012-001/lecture
![Page 29: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/29.jpg)
Building High-level Features Using Large ScaleUnsupervised Learning
http://research.google.com/archive/unsupervised_icml2012.html
![Page 30: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/30.jpg)
http://ai.stanford.edu/~ang/papers/icml09-ConvolutionalDeepBeliefNetworks.pdf
![Page 31: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/31.jpg)
http://ai.stanford.edu/~ang/papers/icml09-ConvolutionalDeepBeliefNetworks.pdf
![Page 32: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/32.jpg)
http://ai.stanford.edu/~ang/papers/icml09-ConvolutionalDeepBeliefNetworks.pdf
![Page 33: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/33.jpg)
http://ai.stanford.edu/~ang/papers/icml09-ConvolutionalDeepBeliefNetworks.pdf
![Page 34: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/34.jpg)
http://ai.stanford.edu/~ang/papers/icml09-ConvolutionalDeepBeliefNetworks.pdf
![Page 35: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/35.jpg)
http://ai.stanford.edu/~ang/papers/icml09-ConvolutionalDeepBeliefNetworks.pdf
![Page 36: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/36.jpg)
http://ai.stanford.edu/~ang/papers/icml09-ConvolutionalDeepBeliefNetworks.pdf
![Page 37: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/37.jpg)
Picture by John Kaufholdhttp://www.cs.toronto.edu/~rsalakhu/ISBI1_pdf_version.pdf
![Page 38: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/38.jpg)
Next Talk (Part II)
• Ising Models• Restricted Boltzmann
Machines• Convolutional Networks
![Page 39: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/39.jpg)
Various Resources
Courses • Geoff Hinton Coursera course-http://www.coursera.org/course/neuralnets)• Andrew Ng Coursera course-http://www.coursera.org/course/ml
![Page 40: Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines](https://reader036.fdocuments.in/reader036/viewer/2022070510/58ac4deb1a28ab99028b6385/html5/thumbnails/40.jpg)
An introduction to neural nets
An Introduction by Valentino [email protected]
[email protected]+39 333 789 2692+1 202 640 1381