Gradoop: Scalable Graph Analytics with Apache Flink @ Flink Forward 2015
TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1....
Transcript of TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1....
![Page 1: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/1.jpg)
Eron Wright @eronwright
TensorFlow & Apache FlinkTM
An early look at a community project
Apache®, Apache Flink™, Flink™, and the Apache feather logo are trademarks of The Apache Software Foundation. TensorFlow, the TensorFlow logo and any related marks are trademarks of Google Inc.
https://github.com/cookieai/flink-tensorflow
![Page 2: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/2.jpg)
Background
2
![Page 3: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/3.jpg)
Why TensorFlow?
▪ A powerful & flexible platform for machine intelligence
▪ Reusable machine learning models ▪ C++ core / Java language binding ▪ Ease of integration with Apache Flink
3
![Page 4: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/4.jpg)
TF Scenarios
▪ Language Understanding • “syntaxnet”, Google Translate
▪ Image/Video/Audio Recognition • “Inception”
▪ Creative Arts • “Magenta”
4
![Page 5: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/5.jpg)
TF Models
▪ Portable using “Saved Model” format • Train on GPU-equipped cluster • Perform inference anywhere
▪ Well-defined interactions and data types using “signatures”
▪ Moving towards a Model Zoo5
![Page 6: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/6.jpg)
TF Graphs
6
x
W
*
b
+ softmax() y
x = tf.placeholder(tf.float32, [None, 784]) W = tf.Variable(tf.zeros([784, 10])) b = tf.Variable(tf.zeros([10])) y = tf.nn.softmax(tf.matmul(x, W) + b)
![Page 7: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/7.jpg)
TF in Flink
7
Source map() window() sink
x
W*
b+ max() y
![Page 8: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/8.jpg)
Introducing Flink-Tensorflow
8
![Page 9: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/9.jpg)
Project Status
▪ A prototype focused on inference using pre-trained TF models
▪ Scala-only (for now) ▪ A community effort
9
![Page 10: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/10.jpg)
Basic Idea
▪ Use TF functionality in a Flink program; Not a TF compatibility layer
▪ “TF graph as a Flink map function” ▪ Support inference today, online learning
in the future
10
![Page 11: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/11.jpg)
Demo
11
![Page 12: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/12.jpg)
“Johnny”
▪ A hypothetical security system based on picture passwords • Present three specific pictures within one minute:
Access Granted! • On timeout: Access Denied!
▪ TF model for image labeling (“Inception”) ▪ Flink CEP library for sequence detection
12
![Page 13: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/13.jpg)
Inception Model
▪ Pretrained with ImageNet dataset ▪ Supports “retraining” for learning new objects
not in original dataset
13
load() label()
burger.jpg ladybug.jpg
Stream (Connector)
1 0.02
..
623 0.97
…
![Page 15: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/15.jpg)
Using the Library
15
![Page 16: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/16.jpg)
Basic Usage
1. Import a TensorFlow model 2. Write code to convert your domain
objects to/from tensors 3. Use the model in a batch or streaming
function
16
![Page 17: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/17.jpg)
Importing a Model
1. Define the graph method(s) supported by the model (ref)
2. Specify how to load the model 1. ”saved model” loader, or 2. graphdef loader, or 3. ad-hoc graph builder
17
![Page 18: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/18.jpg)
Importing a Model (Con’t)
18
![Page 19: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/19.jpg)
Working with Tensors
▪ Tensors are off-heap, AutoCloseable multi-dimensional arrays
▪ You: convert input records to tensor ▪ Use Scala Automatic Resource
Management (ARM) • Supports both imperative and monadic style
19
![Page 20: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/20.jpg)
Writing a Flink Function
▪ Design goal: use TF in any transformation function • `MapFunction` • `ProcessFunction` (with event-time timers!) • `WindowFunction`
▪ Required: model lifecycle support • `ModelAwareFunction`
20
![Page 21: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/21.jpg)
Runtime
▪ TF embedded within the Flink JobManager / TaskManager
▪ One model instance per sub-task ▪ Large unmanaged memory blocks ▪ No Python needed
21
![Page 22: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/22.jpg)
Future Directions
22
![Page 23: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/23.jpg)
Stateful Models
23
▪ Integrated checkpointing
▪ Support for keyed streams
▪ Emphasize online learning
![Page 24: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/24.jpg)
Graph Builder (DSL)
▪ Construct TensorFlow graphs from scratch (ref)
▪ Code generation for TF operations ▪ Incorporate other libraries, high-level APIs
(e.g. TF Keras)
24
![Page 25: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/25.jpg)
Other
▪ Instrumentation • TF Summaries (for TensorBoard) • Flink Metrics
▪ Model versioning • Leverage Flink job versioning methods • Treat models as side-input
25
![Page 26: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/26.jpg)
Eron Wright @eronwright
TensorFlow & Apache FlinkTM
An early look at a community project
Apache®, Apache Flink™, Flink™, and the Apache feather logo are trademarks of The Apache Software Foundation. TensorFlow, the TensorFlow logo and any related marks are trademarks of Google Inc.
https://github.com/cookieai/flink-tensorflow
![Page 27: TensorFlow & Apache Flink...Use the model in a batch or streaming function 16 Importing a Model 1. Define the graph method(s) supported by the model (ref) 2. Specify how to load the](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed372b587321e22e0482561/html5/thumbnails/27.jpg)
Thanks!
27