CS230: Lecture 3cs230.stanford.edu/files/lecture-notes-3.pdf · CS230: Lecture 3 The mathematics of...

14
CS230: Lecture 3 The mathematics of deep learning Backpropagation, Initializations, Regularization Kian Katanforoosh

Transcript of CS230: Lecture 3cs230.stanford.edu/files/lecture-notes-3.pdf · CS230: Lecture 3 The mathematics of...

Page 1: CS230: Lecture 3cs230.stanford.edu/files/lecture-notes-3.pdf · CS230: Lecture 3 The mathematics of deep learning Backpropagation, Initializations, Regularization Kian Katanforoosh.

CS230:Lecture3ThemathematicsofdeeplearningBackpropagation,Initializations,Regularization

KianKatanforoosh

Page 2: CS230: Lecture 3cs230.stanford.edu/files/lecture-notes-3.pdf · CS230: Lecture 3 The mathematics of deep learning Backpropagation, Initializations, Regularization Kian Katanforoosh.

I– BackpropagationII– InitializationsIII– Regularization

Page 3: CS230: Lecture 3cs230.stanford.edu/files/lecture-notes-3.pdf · CS230: Lecture 3 The mathematics of deep learning Backpropagation, Initializations, Regularization Kian Katanforoosh.

I- Backpropagation

Page 4: CS230: Lecture 3cs230.stanford.edu/files/lecture-notes-3.pdf · CS230: Lecture 3 The mathematics of deep learning Backpropagation, Initializations, Regularization Kian Katanforoosh.

Problemstatement

Page 5: CS230: Lecture 3cs230.stanford.edu/files/lecture-notes-3.pdf · CS230: Lecture 3 The mathematics of deep learning Backpropagation, Initializations, Regularization Kian Katanforoosh.

A– LogisticRegressionbackpropagationforonetrainingexample

Page 6: CS230: Lecture 3cs230.stanford.edu/files/lecture-notes-3.pdf · CS230: Lecture 3 The mathematics of deep learning Backpropagation, Initializations, Regularization Kian Katanforoosh.

B– LogisticRegressionbackpropagationforabatchofmexamples

Page 7: CS230: Lecture 3cs230.stanford.edu/files/lecture-notes-3.pdf · CS230: Lecture 3 The mathematics of deep learning Backpropagation, Initializations, Regularization Kian Katanforoosh.

Question:Youhavetrainedananimalclassifier.Canyoutellwhatpartoftheinputledtothisprediction?

Page 8: CS230: Lecture 3cs230.stanford.edu/files/lecture-notes-3.pdf · CS230: Lecture 3 The mathematics of deep learning Backpropagation, Initializations, Regularization Kian Katanforoosh.

II- Initializations

Page 9: CS230: Lecture 3cs230.stanford.edu/files/lecture-notes-3.pdf · CS230: Lecture 3 The mathematics of deep learning Backpropagation, Initializations, Regularization Kian Katanforoosh.

Problemstatement

Page 10: CS230: Lecture 3cs230.stanford.edu/files/lecture-notes-3.pdf · CS230: Lecture 3 The mathematics of deep learning Backpropagation, Initializations, Regularization Kian Katanforoosh.

Inclass,you’veseenthat:

Page 11: CS230: Lecture 3cs230.stanford.edu/files/lecture-notes-3.pdf · CS230: Lecture 3 The mathematics of deep learning Backpropagation, Initializations, Regularization Kian Katanforoosh.

Thegoal:

Page 12: CS230: Lecture 3cs230.stanford.edu/files/lecture-notes-3.pdf · CS230: Lecture 3 The mathematics of deep learning Backpropagation, Initializations, Regularization Kian Katanforoosh.

Let’sprovethat: !"# "[%&'] = !"# "[%] → !"# +[%] = 1-[%&']

Page 13: CS230: Lecture 3cs230.stanford.edu/files/lecture-notes-3.pdf · CS230: Lecture 3 The mathematics of deep learning Backpropagation, Initializations, Regularization Kian Katanforoosh.

Let’sprovethat: !"# "[%&'] = !"# "[%] → !"# +[%] = 1-[%&']

!"# "[%] = !"# .[%] = !"# / +',1% "1%&'2 345

16'Checkpoint:

Page 14: CS230: Lecture 3cs230.stanford.edu/files/lecture-notes-3.pdf · CS230: Lecture 3 The mathematics of deep learning Backpropagation, Initializations, Regularization Kian Katanforoosh.

We’veshownthatforeverylayerl: !"# "[%] = -[%&']!"# +[%] !"# "[%&']


cs230.stanford.educs230.stanford.edu/projects_winter_2019/reports/15811869.pdf · realistic personalised letters, formulating digital signatures, etc. In order to preserve information

cs230.stanford.educs230.stanford.edu/projects_winter_2019/reports/15811869.pdf · realistic personalised letters, formulating digital signatures, etc. In order to preserve information

A of sites that exchange informa- - Donald Bren School of ...cs230/reading/00043525.pdf · of sites that exchange informa-

A of sites that exchange informa- - Donald Bren School of ...cs230/reading/00043525.pdf · of sites that exchange informa-

Generating Probability Distributions for Future Stock …cs230.stanford.edu/projects_winter_2020/reports/32144605.pdfGenerating Probability Distributions for Future Stock Prices Jeffrey

Generating Probability Distributions for Future Stock …cs230.stanford.edu/projects_winter_2020/reports/32144605.pdfGenerating Probability Distributions for Future Stock Prices Jeffrey

Module: CS230 Web Information Processing Assignment: Presentation Facilitators: Mr. Adam Winstanley Mr. David Kaneswaren Team: C3 Student Names and Numbers:

Module: CS230 Web Information Processing Assignment: Presentation Facilitators: Mr. Adam Winstanley Mr. David Kaneswaren Team: C3 Student Names and Numbers:

CS230: Lecture 2 Deep Learning Intuitioncs230.stanford.edu/files/Week2_slides_old.pdf · 2019. 1. 7. · 2 2+Style S−Style G 2 2 Content C Style S We are not learning parameters

CS230: Lecture 2 Deep Learning Intuitioncs230.stanford.edu/files/Week2_slides_old.pdf · 2019. 1. 7. · 2 2+Style S−Style G 2 2 Content C Style S We are not learning parameters

CS230: Lecture 9 Deep Reinforcement Learningcs230.stanford.edu/files/Lecture9.pdf · Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri I. Motivation Human Level Control through

CS230: Lecture 9 Deep Reinforcement Learningcs230.stanford.edu/files/Lecture9.pdf · Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri I. Motivation Human Level Control through

CS230: Lecture 3 Various Deep Learning Topicscs230.stanford.edu/files_fall2017/CS230_Handout4.pdfy = 0 (it’s not you) Bertrand Goal: A school wants to use Face Verification for

CS230: Lecture 3 Various Deep Learning Topicscs230.stanford.edu/files_fall2017/CS230_Handout4.pdfy = 0 (it’s not you) Bertrand Goal: A school wants to use Face Verification for

cs230.stanford.educs230.stanford.edu/projects_winter_2019/reports/15766721.pdf · and representations of the results), media monitoring, newsletters, social media marketing, question

cs230.stanford.educs230.stanford.edu/projects_winter_2019/reports/15766721.pdf · and representations of the results), media monitoring, newsletters, social media marketing, question

cs230.stanford.educs230.stanford.edu/projects_spring_2019/reports/18675538.pdfconvolutional neural networks. " Convolutional Neural Networks for Visual Recognition 2 (2016). [3] Sharma,

cs230.stanford.educs230.stanford.edu/projects_spring_2019/reports/18675538.pdfconvolutional neural networks. " Convolutional Neural Networks for Visual Recognition 2 (2016). [3] Sharma,

17 - 1 Binary Trees PS4 due 1:30pm Tuesday, April 10 Wellesley College CS230 Lecture 17 Thursday, April 5 Handout #28.

17 - 1 Binary Trees PS4 due 1:30pm Tuesday, April 10 Wellesley College CS230 Lecture 17 Thursday, April 5 Handout #28.

CS230: Lecture 5cs230.stanford.edu/fall2017/files/CS230_Handout6.pdf · CS230: Lecture 5 Advanced topics in Object Detection Kian Katanforoosh, Andrew Ng Stanford University. Kian

CS230: Lecture 5cs230.stanford.edu/fall2017/files/CS230_Handout6.pdf · CS230: Lecture 5 Advanced topics in Object Detection Kian Katanforoosh, Andrew Ng Stanford University. Kian

CS230: Lecture 9 Deep Reinforcement Learningcs230.stanford.edu/spring2020/lecture9.pdfCS230: Lecture 9 Deep Reinforcement Learning Kian Katanforoosh Kian Katanforoosh I. Motivation

CS230: Lecture 9 Deep Reinforcement Learningcs230.stanford.edu/spring2020/lecture9.pdfCS230: Lecture 9 Deep Reinforcement Learning Kian Katanforoosh Kian Katanforoosh I. Motivation

CS230: Lecture 2 Deep Learning IntuitionLeon A. Gatys, Alexander S. Ecker, Matthias Bethge: A Neural Algorithm of Artistic Style, 2015 We are not learning parameters by minimizing

CS230: Lecture 2 Deep Learning IntuitionLeon A. Gatys, Alexander S. Ecker, Matthias Bethge: A Neural Algorithm of Artistic Style, 2015 We are not learning parameters by minimizing

CS230-Fall 2019 Final Project Report Painting Genre ...cs230.stanford.edu/projects_fall_2019/reports/26180781.pdf · CS230-Fall 2019 Final Project Report Painting Genre Classification

CS230-Fall 2019 Final Project Report Painting Genre ...cs230.stanford.edu/projects_fall_2019/reports/26180781.pdf · CS230-Fall 2019 Final Project Report Painting Genre Classification

CS230: Lecture 9 Deep Reinforcement Learning · CS230: Lecture 9 Deep Reinforcement Learning Kian Katanforoosh Menti code: 80 24 08. Kian Katanforoosh, Andrew Ng, ... Today’s outline.

CS230: Lecture 9 Deep Reinforcement Learning · CS230: Lecture 9 Deep Reinforcement Learning Kian Katanforoosh Menti code: 80 24 08. Kian Katanforoosh, Andrew Ng, ... Today’s outline.

CS230: Lecture 10 Class wrap-up › fall2018 › lecture10.pdfCS 229A: Applied Machine Learning CS 224N: Natural Language Processing with Deep Learning (LINGUIST 284) CS 224U: Natural

CS230: Lecture 10 Class wrap-up › fall2018 › lecture10.pdfCS 229A: Applied Machine Learning CS 224N: Natural Language Processing with Deep Learning (LINGUIST 284) CS 224U: Natural

ColorEdge CS230/CX240 User's Manualcdn.cnetcontent.com/50/44/5044c719-ee59-4aeb-a2a0-fab75fe81fb3.pdf · As it takes about 30 minutes (CS230) / 7 minutes (CS240: under our measurement

ColorEdge CS230/CX240 User's Manualcdn.cnetcontent.com/50/44/5044c719-ee59-4aeb-a2a0-fab75fe81fb3.pdf · As it takes about 30 minutes (CS230) / 7 minutes (CS240: under our measurement

cs230.stanford.educs230.stanford.edu/projects_winter_2019/posters/15811897.pdfdeep reinforcement learning networks to play simple cooperative games. This project utilizes a simulated

cs230.stanford.educs230.stanford.edu/projects_winter_2019/posters/15811897.pdfdeep reinforcement learning networks to play simple cooperative games. This project utilizes a simulated

Midterm Review CS 230 – Distributed Systems (cs230) Nalini Venkatasubramanian nalini@ics.uci.edu.

Midterm Review CS 230 – Distributed Systems (cs230) Nalini Venkatasubramanian [email protected].

CS230 : Computer Graphicsshinar/courses/cs230-winter-2012/Lecture12.pdf · Rise of the Planet of the Apes, 2011 Avatar, 2009 Andy Serkis - Gollum, Lord of the Rings ... - animations

CS230 : Computer Graphicsshinar/courses/cs230-winter-2012/Lecture12.pdf · Rise of the Planet of the Apes, 2011 Avatar, 2009 Andy Serkis - Gollum, Lord of the Rings ... - animations