A Tutorial on Deep Learning Part 1: Nonlinear Classifiers and The ...
Deep learning tutorial (i)
-
Upload
guan-wang -
Category
Engineering
-
view
68 -
download
1
Transcript of Deep learning tutorial (i)
![Page 1: Deep learning tutorial (i)](https://reader035.fdocuments.in/reader035/viewer/2022062503/587d162f1a28abae148b6a69/html5/thumbnails/1.jpg)
Deep Learning Tutorial (I)IntroductionGuan Wang
![Page 2: Deep learning tutorial (i)](https://reader035.fdocuments.in/reader035/viewer/2022062503/587d162f1a28abae148b6a69/html5/thumbnails/2.jpg)
Contents
Overview
Industry Landscape
System architectures
Bleeding edges
![Page 3: Deep learning tutorial (i)](https://reader035.fdocuments.in/reader035/viewer/2022062503/587d162f1a28abae148b6a69/html5/thumbnails/3.jpg)
Overview
![Page 4: Deep learning tutorial (i)](https://reader035.fdocuments.in/reader035/viewer/2022062503/587d162f1a28abae148b6a69/html5/thumbnails/4.jpg)
Deep neural networks learn to do the following
![Page 5: Deep learning tutorial (i)](https://reader035.fdocuments.in/reader035/viewer/2022062503/587d162f1a28abae148b6a69/html5/thumbnails/5.jpg)
Yearbook• prior 2012
• Machine learning was more about SVM, Graphical Models, non-parametric baysian, and simple hacks, e.g., decision tree, naïve bayesian, regressions, etc
•2012~2013
• Deep Neural Network, big data, mature distributed computing architectures
• Refreshing accuracy record in image recognition tasks
• Feed forward Neural Networks, CNN, RBM
•2014~2015
• Bridging CV, NLP and expanding to other domains
• New architectures and new ML tasks
• RNN, LSTM, RL
•2016~future
• Larger models on CV, NLP, keep expanding to other domains
• Larger systems, larger models on CNN, LSTM, etc
![Page 6: Deep learning tutorial (i)](https://reader035.fdocuments.in/reader035/viewer/2022062503/587d162f1a28abae148b6a69/html5/thumbnails/6.jpg)
Industry Landscape
![Page 7: Deep learning tutorial (i)](https://reader035.fdocuments.in/reader035/viewer/2022062503/587d162f1a28abae148b6a69/html5/thumbnails/7.jpg)
Hardware Optimized for Neural Networks
Google TPUNVIDIA
![Page 8: Deep learning tutorial (i)](https://reader035.fdocuments.in/reader035/viewer/2022062503/587d162f1a28abae148b6a69/html5/thumbnails/8.jpg)
Deep Learning Service on cloud
![Page 9: Deep learning tutorial (i)](https://reader035.fdocuments.in/reader035/viewer/2022062503/587d162f1a28abae148b6a69/html5/thumbnails/9.jpg)
Computer Vision (crowded market)Generic Algorithms & API providers•Special object recognition (face, etc)•General object recognition (image search, etc)•Moving object detection & recognition (pedestrian detection, etc)•Image understanding (visual QA, artify, etc)•Video understanding (video search, etc)Verticals•Satellite image analysis (understanding civil developments)•Home & office place security & surveillance•User interest analysis and high precision targeting (ads)•ADAS & Autonomous Driving•Robotics and drones•The list goes on and on
![Page 10: Deep learning tutorial (i)](https://reader035.fdocuments.in/reader035/viewer/2022062503/587d162f1a28abae148b6a69/html5/thumbnails/10.jpg)
Natural Language Understanding (crowded market)
Generic Algorithms & API providers•Personal assistance (x.ai, api.ai, etc)•Chat bots (facebook ecosystem, viv.ai, etc)•Knowledge understanding (IBM watson, etc)
Verticals•Customer service•Travel management•Financial service•Smart homes•Connected cars•The list goes on and on
![Page 11: Deep learning tutorial (i)](https://reader035.fdocuments.in/reader035/viewer/2022062503/587d162f1a28abae148b6a69/html5/thumbnails/11.jpg)
System Architecture
Every good machine learning algorithm deserves its own system architecture. -- one of my mentors
![Page 12: Deep learning tutorial (i)](https://reader035.fdocuments.in/reader035/viewer/2022062503/587d162f1a28abae148b6a69/html5/thumbnails/12.jpg)
Distributed ArchitecturesGeneric solvers•Stochastic Gradient Descent•Coordinate Descent•MCMC•ADMM•…Design Choices•sync or async•CPU or GPU cluster•Online training or offline training•...Examples•Parameter Server•DistBelief•Tensorflow
Flexible solutions•CoreOS, etcd, Docker•Kubernetes•Pachyderm•Mesos•etcBig data ecosystem•Spark & tachyon•yarn•hdfs•etcDeep learning tools•Tensorflow•Torch-IPC•DistBelief•Petuum•GraphLab
![Page 13: Deep learning tutorial (i)](https://reader035.fdocuments.in/reader035/viewer/2022062503/587d162f1a28abae148b6a69/html5/thumbnails/13.jpg)
Standalone ToolkitsCPU with GPU speedup •Torch•Caffe•Theano•Tensorflow
Good for research or small applications
Embedded Support•Tiny-CNN•Tensorflow-embedded
Good for phone apps, raspberry Pi, cars, drones, robotics
![Page 14: Deep learning tutorial (i)](https://reader035.fdocuments.in/reader035/viewer/2022062503/587d162f1a28abae148b6a69/html5/thumbnails/14.jpg)
Bleeding edge directions
![Page 15: Deep learning tutorial (i)](https://reader035.fdocuments.in/reader035/viewer/2022062503/587d162f1a28abae148b6a69/html5/thumbnails/15.jpg)
CNN
CNN on Text ClassificationR-CNN, fast R-CNNAttention Model
![Page 16: Deep learning tutorial (i)](https://reader035.fdocuments.in/reader035/viewer/2022062503/587d162f1a28abae148b6a69/html5/thumbnails/16.jpg)
LSTMSeq2seq model•Translation•Dialogue system•Time series analysis•Text classification
![Page 17: Deep learning tutorial (i)](https://reader035.fdocuments.in/reader035/viewer/2022062503/587d162f1a28abae148b6a69/html5/thumbnails/17.jpg)
Memory NetworksExternal memory on RNN•Translation•Dialogue system•QA
![Page 18: Deep learning tutorial (i)](https://reader035.fdocuments.in/reader035/viewer/2022062503/587d162f1a28abae148b6a69/html5/thumbnails/18.jpg)
Reinforcement LearningDeep Q-LearningFunction approximation on•Policy function•Value function
![Page 19: Deep learning tutorial (i)](https://reader035.fdocuments.in/reader035/viewer/2022062503/587d162f1a28abae148b6a69/html5/thumbnails/19.jpg)
Deep Generative ModelAdversarial networks
![Page 20: Deep learning tutorial (i)](https://reader035.fdocuments.in/reader035/viewer/2022062503/587d162f1a28abae148b6a69/html5/thumbnails/20.jpg)
The End
Thank You!