Kaz Sato, Evangelist, Google at MLconf ATL 2016
76
Machine Intelligence at Google Scale ML APIs, TensorFlow and Cloud ML
-
Upload
mlconf -
Category
Technology
-
view
375 -
download
1
Transcript of Kaz Sato, Evangelist, Google at MLconf ATL 2016
Machine Intelligence at Google ScaleML APIs, TensorFlow and Cloud ML
+Kazunori Sato@kazunori_279
Kaz Sato
Staff Developer AdvocateTech Lead for Data & AnalyticsCloud Platform, Google Inc.
What we’ll cover
What is Neural Network and Deep Learning
Machine Learning use cases at Google services
Externalizing the power with ML APIs
TensorFlow: the open source library for ML
TensorFlow in the Wild
Distributed training and prediction with Cloud ML
What is Neural Network and Deep Learning
Neural Network is a function that can learn
xn
> b?
w1
wn
x2
x1
Inspired by the behavior of biological neurons
How do you classify them?
weights
bias(threshold)
Programmers need to specify the parameters
Let’s see how neural network solves the problem
The computer tries to find the best parameters
A neuron classifies a data point into two kinds
Gradient Descent: adjusting the params gradually to reduce errors
From: Andrew Ng
How do you classify them?
What we see What the computer “sees”
28 x 28 gray scale image =
784 numbers
input vector (pixel data)
output vector (probability)
How do you classify them?
More neurons = More features to extract
Hidden Layers:
mapping inputs to
a feature space,
classifying with
a hyperplaneFrom: Neural Networks, Manifolds, and Topology, colah's blog
How about this?
More hidden layers = More hierarchies of features
How about this?
We need to go deeper neural network
From: Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations, Honglak Lee et al.
From: mNeuron: A Matlab Plugin to Visualize Neurons from Deep Models, Donglai Wei et. al.
Machine Learning use casesat Google services
25
signalfor Search ranking,
out of hundreds
improvementto ranking quality
in 2+ years
#3 #1
Search
machine learning for search engines
RankBrain: a deep neural network for search ranking
WaveNet by Google DeepMind
28
[glacier]
Google Photos
28
29
Smart reply in Inbox by Gmail
10%of all responses sent on mobile
http://googleresearch.blogspot.com/2016/05/announcing-syntaxnet-worlds-most.html
Saved Data Center cooling energy for 40%Improved Power Usage Effectiveness (PUE) for 15%
32
AndroidAppsGmailMapsPhotosSpeechSearchTranslationYouTubeand many others ...
Used across products:
2012 2013 2014 2015
Deep Learning usage at Google
Externalizing the powerwith ML APIs
TensorFlow Cloud Machine Learning ML API
Easy-to-Use, for non-ML engineers
Customizable, for Data Scientists
Machine Learning products from Google
Image analysis with pre-trained models
No Machine Learning skill required
REST API: receives an image and returns a JSON
General Availability
Cloud Vision API
Confidential & ProprietaryGoogle Cloud Platform 36
FacesFaces, facial landmarks, emotions
OCRRead and extract text, with support for > 10 languages
LabelDetect entities from furniture to transportation
LogosIdentify product logos
Landmarks & Image PropertiesDetect landmarks & dominant color of image
Safe SearchDetect explicit content - adult, violent, medical and spoof
3737
Demo
Pre-trained models. No ML skill required
REST API: receives audio and returns texts
Supports 80+ languages
Streaming or non-streaming
Public Beta - cloud.google.com/speech
Cloud Speech API
Confidential & ProprietaryGoogle Cloud Platform 39
Features
Automatic Speech Recognition (ASR) powered by deep learning neural networking to power your applications like voice search or speech transcription.
Recognizes over 80 languages and variants with an extensive vocabulary.
Returns partial recognition results immediately, as they become available.
Filter inappropriate content in text results.
Audio input can be captured by an application’s microphone or sent from a pre-recorded audio file. Multiple audio file formats are supported, including FLAC, AMR, PCMU and linear-16.
Handles noisy audio from many environments without requiring additional noise cancellation.
Audio files can be uploaded in the request and, in future releases, integrated with Google Cloud Storage.
Automatic Speech Recognition Global Vocabulary Inappropriate Content Filtering
Streaming Recognition
Real-time or Buffered Audio Support Noisy Audio Handling Integrated API
4040
Demo
Pre-trained models. No ML skill required
REST API: receives text and returns analysis results
Supports English, Spanish and Japanese
Public Beta - cloud.google.com/natural-language
Cloud Natural Language API
Confidential & ProprietaryGoogle Cloud Platform 42
Features
Extract sentence, identify parts of speech and create dependency parse trees for each sentence.
Identify entities and label by types such as person, organization, location, events, products and media.
Understand the overall sentiment of a block of text.
Syntax Analysis Entity Recognition
Sentiment Analysis
4343
Demo
TensorFlow:An open source library forMachine Intelligence
Google's open source library for
machine intelligence
tensorflow.org launched in Nov 2015
Used by many production ML projects
What is TensorFlow?
Sharing our tools with researchers and developers around the world
repositoryfor “machine learning”
category on GitHub
#1
Released in Nov. 2015
From: http://deliprao.com/archives/168
Before
Hire Data Scientists↓
Understand the math model↓
Impl with programming code↓
Train with single GPU↓
Build a GPU cluster↓
Train with the GPU cluster↓
Build a prediction serveror Impl mobile/IoT prediction
After
Easy network design and impl
↓
Train with single machine
↓
Train on the cloud
↓
Prediction on the cloud
or mobile/IoT devices
many peoplestuck here
# define the networkimport tensorflow as tfx = tf.placeholder(tf.float32, [None, 784])W = tf.Variable(tf.zeros([784, 10]))b = tf.Variable(tf.zeros([10]))y = tf.nn.softmax(tf.matmul(x, W) + b)
# define a training stepy_ = tf.placeholder(tf.float32, [None, 10])xent = -tf.reduce_sum(y_*tf.log(y))step = tf.train.GradientDescentOptimizer(0.01).minimize(xent)
TensorBoard: visualization tool
Portable and ScalableTraining on:
Mac/Windows
GPU server
GPU cluster / Cloud
Prediction on:
Android and iOS
RasPi and TPU
Distributed Training with TensorFlow
TensorFlow in the Wild(or democratization of deep learning)
TensorFlow
powered
Fried Chicken
Nugget Server
From: http://www.rt-net.jp/karaage1/
TensorFlow powered Cucumber Sorter
From: http://workpiles.com/2016/02/tensorflow-cnn-cucumber/
Autonomous Driving
of RasPi carwith Inception 3on TensorFlow
From: https://github.com/zxzhijia/GoPiGo-Driven-by-Tensorflow
TV popstar classifierwith 95% accuracy
From: http://memo.sugyan.com/entry/2016/06/14/220624
TensorFlow＋
RasPi forsorting garbages
From: https://techcrunch.com/2016/09/13/auto-trash-sorts-garbage-automatically-at-the-techcrunch-disrupt-hackathon/
TensorFlow +
Drones
for counting trucks
From: http://www.brainpad.co.jp/news/2016/09/02/3454
Distributed Training and Prediction with Cloud ML
From: Andrew Ng
The Bigger, The Better
The Challenge: Computing Power
DNN requires large training datasets
Large models doesn't fit into a GPU
Requires try-and-errors to find the best design, configs and params
↓Need to spend a few days or
weeks to finish a training
GPUs run at nanosecondsGPU cluster needs microsec network
Enterprise
Google Cloud is
The Datacenter as a Computer
Jupiter network
10 GbE x 100 K = 1 Pbps
Consolidates servers with
microsec latency
Borg
No VMs, pure containers
10K - 20K nodes per Cell
DC-scale job scheduling
CPUs, mem, disks and IO
Distributed Training with TensorFlow
by data parallelism
split data,
share model
● CPU/GPU scheduling
● Communications
○ Local, RPC, RDMA
○ 32/16/8 bit quantization
● Cost-based optimization
● Fault tolerance
Distributed Systems for Large Neural Network
What's the scalability of Google Brain?
"Large Scale Distributed Systems for Training Neural
Networks", NIPS 2015
○ Inception / ImageNet: 40x with 50 GPUs
○ RankBrain: 300x with 500 nodes
Fully managed distributed training and prediction
Supports custom TensorFlow graphs
Integrated with Cloud Dataflow and Cloud Datalab
Limited Preview - cloud.google.com/ml
Cloud Machine Learning (Cloud ML)
7272
Ready to use Machine Learning models
Use your own data to train models
Cloud Vision API
Cloud Speech API
Cloud Translate API
Cloud Machine Learning
Develop - Model - Test
Google BigQuery
Stay Tuned….
Cloud Storage
Cloud Datalab
NEW
Alpha
GA BetaGA
AlphaGA
GA
Tensor Processing Unit
ASIC for TensorFlow
Designed by Google
10x better perf / watt
latency and efficiency
bit quantization
TPU on Production
RankBrain
AlphaGo
Google Photos
Speech
and more
Thank you!
Links & Resources
Large Scale Distributed Systems for Training Neural Networks, Jeff Dean and Oriol Vinals
Cloud Vision API: cloud.google.com/vision
Cloud Speech API: cloud.google.com/speech
TensorFlow: tensorflow.org
Cloud Machine Learning: cloud.google.com/ml
Cloud Machine Learning: demo video
![Special Assistance Project (Loans 1337-KAZ[SF] & 1338-KAZ)](https://static.fdocuments.in/doc/165x107/577ce66d1a28abf10392ca7d/special-assistance-project-loans-1337-kazsf-1338-kaz.jpg)
