Tensorflow vs MxNet
-
Upload
ashish-bansal -
Category
Data & Analytics
-
view
8.135 -
download
0
Transcript of Tensorflow vs MxNet
![Page 1: Tensorflow vs MxNet](https://reader034.fdocuments.in/reader034/viewer/2022052116/5a6478df7f8b9a3b568b468f/html5/thumbnails/1.jpg)
DEEP LEARNING IN THE CLOUD: TENSORFLOW VS MXNET
ASHISH BANSAL, SNR DIR DATA SCIENCE @ CAPITAL ONE
@ASH_BANS | LINKEDIN.COM/IN/BANSALASHISH
![Page 2: Tensorflow vs MxNet](https://reader034.fdocuments.in/reader034/viewer/2022052116/5a6478df7f8b9a3b568b468f/html5/thumbnails/2.jpg)
AGENDA FOR TODAY
• What is deep learning
• Pre-requisites for deep learning
• Libraries landscape
• TensorFlow vs MxNet
• Programming model
• Popularity, developer support, cloud, deployment
• Summary
Deep Learning in the cloud Ashish Bansal 1
![Page 3: Tensorflow vs MxNet](https://reader034.fdocuments.in/reader034/viewer/2022052116/5a6478df7f8b9a3b568b468f/html5/thumbnails/3.jpg)
WHAT IS DEEP LEARNING?• Variant of neural networks
• Inspired by human brain, but only very loosely mimics it (very very simple compared to the brain)
• Saves a ton of data scientist time in feature engineering–spend more time in tuning
• GPUs make it very efficient. Thank you Gamers!
• Requires large amounts of data
• Given enough data, will generally outperform other algorithms
• All popular references to AI that you read refer to deep learning advances
Deep Learning in the cloud Ashish Bansal 2
![Page 4: Tensorflow vs MxNet](https://reader034.fdocuments.in/reader034/viewer/2022052116/5a6478df7f8b9a3b568b468f/html5/thumbnails/4.jpg)
WHAT MAKES DEEP LEARNING DEEP: FEATURE HIERARCHY OF TRAINED REPRESENTATIONS
Deep Learning in the cloud Ashish Bansal 3
• 2 layer models are not deep because there is no feature
hierarchy
• Neural networks with 1 hidden layer are not considered
deep for the same reason
• Classification trees are not deep as all decisions are
made in the input space
Source: Unreasonable Effectiveness of Deep Learning by Yann LeCun (http://www4.ncsu.edu/~pcombet/data2015/lecun2015.pdf)
![Page 5: Tensorflow vs MxNet](https://reader034.fdocuments.in/reader034/viewer/2022052116/5a6478df7f8b9a3b568b468f/html5/thumbnails/5.jpg)
PRE-REQUISITES FOR DEEP LEARNING
• Data, and lots of it
• A way to express or code neural networks
• A fast processor adept at matrix computations
• High throughput between storage and compute for several passes during
training
Note: Training, by far, is the most time taking activity. Once trained, networks can classify fairly quickly
Deep Learning in the cloud Ashish Bansal 4
![Page 6: Tensorflow vs MxNet](https://reader034.fdocuments.in/reader034/viewer/2022052116/5a6478df7f8b9a3b568b468f/html5/thumbnails/6.jpg)
PRE-REQUISITES FOR DEEP LEARNING
• Data, and lots of it: Is it on the cloud? Key challenge here is getting data to the cloud
• A way to express or code neural networks: Can be solved with TensorFlow or MxNet
• A fast processor adept at matrix computations: GPU based cloud units like AWS P2
instances
• High throughput between storage and compute for several passes during training:
Large memory on the cloud instances, like p2.8xlarge, and parallelization
Deep Learning in the cloud Ashish Bansal 5
![Page 7: Tensorflow vs MxNet](https://reader034.fdocuments.in/reader034/viewer/2022052116/5a6478df7f8b9a3b568b468f/html5/thumbnails/7.jpg)
LIBRARIES LANDSCAPE
Deep Learning in the cloud Ashish Bansal 6
![Page 8: Tensorflow vs MxNet](https://reader034.fdocuments.in/reader034/viewer/2022052116/5a6478df7f8b9a3b568b468f/html5/thumbnails/8.jpg)
TENSORFLOW & MXNET OVERVIEW
Deep Learning in the cloud Ashish Bansal Source: https://svds.com/getting-started-deep-learning/ 7
![Page 9: Tensorflow vs MxNet](https://reader034.fdocuments.in/reader034/viewer/2022052116/5a6478df7f8b9a3b568b468f/html5/thumbnails/9.jpg)
TENSORFLOW VS MXNET: GITHUB VIEW
Deep Learning in the cloud Ashish Bansal Source: Github. Data as of Apr 18, 2017
8
0
10000
20000
30000
40000
50000
60000
Stars Forks Commits Contributors
Deep Learning Libraries: Github Metrics
TensorFlow Keras MxNet Qix Torch Caffe Theano DL4J CNTK DSSTNE
![Page 10: Tensorflow vs MxNet](https://reader034.fdocuments.in/reader034/viewer/2022052116/5a6478df7f8b9a3b568b468f/html5/thumbnails/10.jpg)
TENSORFLOW VS MXNET: STACK OVERFLOW VIEW
Deep Learning in the cloud Ashish Bansal Source: Stack Overflow, Data as of Apr 18, 2017 9
0
5000
10000
15000
20000
25000
TensorFlow Keras MxNet Qix Torch Caffe Theano DL4J CNTK DSSTNE
Stack Oveflow Activity
![Page 11: Tensorflow vs MxNet](https://reader034.fdocuments.in/reader034/viewer/2022052116/5a6478df7f8b9a3b568b468f/html5/thumbnails/11.jpg)
TENSORFLOW VS MXNET: PERFORMANCE VIEW
Deep Learning in the cloud Ashish Bansal Source: https://arxiv.org/pdf/1608.07249.pdf 10
AlexNet Performance (5 layers, 81K params) ResNet-56 Performance (56 layers, 850K params)
![Page 12: Tensorflow vs MxNet](https://reader034.fdocuments.in/reader034/viewer/2022052116/5a6478df7f8b9a3b568b468f/html5/thumbnails/12.jpg)
TENSORFLOW VS MXNET: CLOUD SUPPORT VIEW
• TensorFlow is fully supported on AWS EC2, Google Cloud & Microsoft Azure
• MxNet is fully supported on AWS EC2 (preferred) and Microsoft Azure
• Your mileage may vary for MxNet on Google Cloud
Deep Learning in the cloud Ashish Bansal 11
![Page 13: Tensorflow vs MxNet](https://reader034.fdocuments.in/reader034/viewer/2022052116/5a6478df7f8b9a3b568b468f/html5/thumbnails/13.jpg)
TENSORFLOW VS MXNET: DEPLOYMENT VIEW
TensorFlow MxNet
Android Yes OOB Yes – needs additional libs & compilation
iOS Yes OOB Yes – needs additional libs & compilation
Raspberry Pi Yes OOB Yes OOB
Windows YMMV Yes
Unix/Linux Yes OOB Yes OOB
Deep Learning in the cloud Ashish Bansal 12
![Page 14: Tensorflow vs MxNet](https://reader034.fdocuments.in/reader034/viewer/2022052116/5a6478df7f8b9a3b568b468f/html5/thumbnails/14.jpg)
SUMMARY
• Lots of momentum and support behind TensorFlow
• TensorFlow has better RNN capabilities
• TensorFlow has better tutorials and online guides. It also has more
supporting material like Stack Overflow questions etc.
• TensorFlow enjoys greater support on the cloud, and has more
deployment options
• MxNet has more language bindings, and is usually faster
Deep Learning in the cloud Ashish Bansal 13
![Page 15: Tensorflow vs MxNet](https://reader034.fdocuments.in/reader034/viewer/2022052116/5a6478df7f8b9a3b568b468f/html5/thumbnails/15.jpg)
QUESTIONS@ASH_BANS | LINKEDIN.COM/IN/BANSALASHISH
Deep Learning in the cloud Ashish Bansal 14