Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)
-
Upload
rajiv-shah -
Category
Engineering
-
view
196 -
download
1
Transcript of Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)
![Page 1: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/1.jpg)
August 31, 2016
Rajiv shahRajivShah.com
rshahATpobox.comgithub.com/rajshah4/
rajcs4
Teaching Recurrent Neural
Networks using Tensorflow
![Page 2: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/2.jpg)
• why• theory/app - deep learning• theory/app - recurrent neural
networks
![Page 3: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/3.jpg)
why?
![Page 4: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/4.jpg)
http
://g
oogl
eres
earc
h.bl
ogsp
ot.c
om/2
015/
11/c
ompu
ter-
resp
ond-
to-t
his-
emai
l.htm
l
smart reply
![Page 5: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/5.jpg)
http
s://
rese
arch
.goo
gleb
log.
com
/201
6/08
/tex
t-su
mm
ariz
atio
n-w
ith-
tens
orflo
w.ht
ml
text summarization
![Page 6: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/6.jpg)
http
://b
log.
kagg
le.c
om/2
016/
01/0
4/ho
w-m
uch-
did-
it-r
ain-
ii-w
inne
rs-i
nter
view
-1st
-pla
ce-p
upa-
aka-
aaro
n-si
m/
kaggle
![Page 7: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/7.jpg)
http
s://
mag
enta
.ten
sorfl
ow.o
rg/2
016/
07/1
5/lo
okba
ck-r
nn-a
tten
tion
-rnn
/
music
![Page 8: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/8.jpg)
theory: deep learning
![Page 11: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/11.jpg)
http
://c
s231
n.gi
thub
.io/n
eura
l-ne
twor
ks-1
/
deep learning
![Page 12: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/12.jpg)
theory: operation
![Page 13: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/13.jpg)
Recurrent
recurrent neural network
![Page 14: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/14.jpg)
http
://c
olah
.git
hub.
io/p
osts
/201
5-08
-Und
erst
andi
ng-L
STM
s/
rnn: rolled out
10 22310+3+22
![Page 15: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/15.jpg)
demo: char-rnn
http
://c
s.st
anfo
rd.e
du/p
eopl
e/ka
rpat
hy/r
ecur
rent
js/
![Page 16: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/16.jpg)
theory ii: cells
![Page 17: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/17.jpg)
http
://c
olah
.git
hub.
io/p
osts
/201
5-08
-Und
erst
andi
ng-L
STM
s/
rnn cell
![Page 18: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/18.jpg)
vanish gradients
![Page 19: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/19.jpg)
http
://g
oogl
eres
earc
h.bl
ogsp
ot.c
om/2
015/
11/c
ompu
ter-
resp
ond-
to-t
his-
emai
l.htm
l
![Page 20: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/20.jpg)
http
://c
olah
.git
hub.
io/p
osts
/201
5-08
-Und
erst
andi
ng-L
STM
s/
long short term memory (LSTM)
![Page 21: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/21.jpg)
long short term memory
![Page 22: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/22.jpg)
http
s://
gith
ub.c
om/r
ajsh
ah4/
tens
orflo
w_s
hiny
demo: sine wave
![Page 23: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/23.jpg)
theory iii: architecture
![Page 24: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/24.jpg)
rnn architecture
![Page 25: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/25.jpg)
rnn architecture:one to many
![Page 26: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/26.jpg)
rnn architecture:one to many
![Page 27: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/27.jpg)
rnn architecture:many to many
![Page 28: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/28.jpg)
(array([[7], [4], [1], [8], [0], [3], [1], [5], [6], [9], [2], [0], [0], [0], [0]]), 46.0)
demo: addition
http
://p
roje
cts.
rajiv
shah
.com
/blo
g/20
16/0
4/05
/rnn
_add
itio
n/
![Page 29: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/29.jpg)
generative
![Page 31: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/31.jpg)
demo: handwriting
http
://w
ww.
cs.t
oron
to.e
du/~
grav
es/h
andw
riti
ng.h
tml
![Page 32: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/32.jpg)
• rnn’s have value• lots of good examples you can
use as a starting point in building an rnn
![Page 33: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/33.jpg)
http
s://
rese
arch
.goo
gleb
log.
com
/201
6/08
/tex
t-su
mm
ariz
atio
n-w
ith-
tens
orflo
w.ht
ml
text summarization
![Page 34: Teaching Recurrent Neural Networks using Tensorflow (Webinar: August 2016)](https://reader030.fdocuments.in/reader030/viewer/2022033020/587d15cc1a28abae148b6963/html5/thumbnails/34.jpg)
August 31, 2016
Rajiv shahRajivShah.com
rshahATpobox.comgithub.com/rajshah4/
rajcs4
Teaching Recurrent Neural
Networks using Tensorflow