Backpropagation: A modular approach (Torch NN)...Backpropagation: A modular approach (Torch NN)...
Transcript of Backpropagation: A modular approach (Torch NN)...Backpropagation: A modular approach (Torch NN)...
![Page 1: Backpropagation: A modular approach (Torch NN)...Backpropagation: A modular approach (Torch NN) Nando de Freitas Outline of the lecture This lecture describes modular ways of formulating](https://reader036.fdocuments.in/reader036/viewer/2022071022/5fd698f90a369d35d5581370/html5/thumbnails/1.jpg)
Backpropagation: A modular approach(Torch NN)
Nando de Freitas
![Page 2: Backpropagation: A modular approach (Torch NN)...Backpropagation: A modular approach (Torch NN) Nando de Freitas Outline of the lecture This lecture describes modular ways of formulating](https://reader036.fdocuments.in/reader036/viewer/2022071022/5fd698f90a369d35d5581370/html5/thumbnails/2.jpg)
Outline of the lectureThis lecture describes modular ways of formulating and learningdistributed representations of data. The objective is for you to learn:
How to specify models such as logistic regression in layers. How to formulate layers and loss criterions. How well formulated local rules results in correct global rules. How back-propagation works. How this manifests itself in Torch.
![Page 3: Backpropagation: A modular approach (Torch NN)...Backpropagation: A modular approach (Torch NN) Nando de Freitas Outline of the lecture This lecture describes modular ways of formulating](https://reader036.fdocuments.in/reader036/viewer/2022071022/5fd698f90a369d35d5581370/html5/thumbnails/3.jpg)
![Page 4: Backpropagation: A modular approach (Torch NN)...Backpropagation: A modular approach (Torch NN) Nando de Freitas Outline of the lecture This lecture describes modular ways of formulating](https://reader036.fdocuments.in/reader036/viewer/2022071022/5fd698f90a369d35d5581370/html5/thumbnails/4.jpg)
Derivative using the chain rule
![Page 5: Backpropagation: A modular approach (Torch NN)...Backpropagation: A modular approach (Torch NN) Nando de Freitas Outline of the lecture This lecture describes modular ways of formulating](https://reader036.fdocuments.in/reader036/viewer/2022071022/5fd698f90a369d35d5581370/html5/thumbnails/5.jpg)
Layer specification
![Page 6: Backpropagation: A modular approach (Torch NN)...Backpropagation: A modular approach (Torch NN) Nando de Freitas Outline of the lecture This lecture describes modular ways of formulating](https://reader036.fdocuments.in/reader036/viewer/2022071022/5fd698f90a369d35d5581370/html5/thumbnails/6.jpg)
Derivative via layer-specification
![Page 7: Backpropagation: A modular approach (Torch NN)...Backpropagation: A modular approach (Torch NN) Nando de Freitas Outline of the lecture This lecture describes modular ways of formulating](https://reader036.fdocuments.in/reader036/viewer/2022071022/5fd698f90a369d35d5581370/html5/thumbnails/7.jpg)
Back-propagation algorithm
![Page 8: Backpropagation: A modular approach (Torch NN)...Backpropagation: A modular approach (Torch NN) Nando de Freitas Outline of the lecture This lecture describes modular ways of formulating](https://reader036.fdocuments.in/reader036/viewer/2022071022/5fd698f90a369d35d5581370/html5/thumbnails/8.jpg)
Derivatives wrt to the input
![Page 9: Backpropagation: A modular approach (Torch NN)...Backpropagation: A modular approach (Torch NN) Nando de Freitas Outline of the lecture This lecture describes modular ways of formulating](https://reader036.fdocuments.in/reader036/viewer/2022071022/5fd698f90a369d35d5581370/html5/thumbnails/9.jpg)
Logit Regression Model in Torch
![Page 10: Backpropagation: A modular approach (Torch NN)...Backpropagation: A modular approach (Torch NN) Nando de Freitas Outline of the lecture This lecture describes modular ways of formulating](https://reader036.fdocuments.in/reader036/viewer/2022071022/5fd698f90a369d35d5581370/html5/thumbnails/10.jpg)
Loss criterion in Torch
![Page 11: Backpropagation: A modular approach (Torch NN)...Backpropagation: A modular approach (Torch NN) Nando de Freitas Outline of the lecture This lecture describes modular ways of formulating](https://reader036.fdocuments.in/reader036/viewer/2022071022/5fd698f90a369d35d5581370/html5/thumbnails/11.jpg)
Derivatives closure in Torch
![Page 12: Backpropagation: A modular approach (Torch NN)...Backpropagation: A modular approach (Torch NN) Nando de Freitas Outline of the lecture This lecture describes modular ways of formulating](https://reader036.fdocuments.in/reader036/viewer/2022071022/5fd698f90a369d35d5581370/html5/thumbnails/12.jpg)
Optimization in Torch
![Page 13: Backpropagation: A modular approach (Torch NN)...Backpropagation: A modular approach (Torch NN) Nando de Freitas Outline of the lecture This lecture describes modular ways of formulating](https://reader036.fdocuments.in/reader036/viewer/2022071022/5fd698f90a369d35d5581370/html5/thumbnails/13.jpg)
Next lecture
In the next lecture, we consider a generalization of logistic regression,with many logistic units, called multi-layer perceptron (MLP) or feed-forward neural network.