Apache Flink - Akka for the Win!
-
Upload
fabian-hueske -
Category
Data & Analytics
-
view
340 -
download
1
Transcript of Apache Flink - Akka for the Win!
![Page 1: Apache Flink - Akka for the Win!](https://reader035.fdocuments.in/reader035/viewer/2022071814/55a69ade1a28abef7d8b4569/html5/thumbnails/1.jpg)
Apache Flink
Akka – Actors for the win
Till Rohrmann
![Page 2: Apache Flink - Akka for the Win!](https://reader035.fdocuments.in/reader035/viewer/2022071814/55a69ade1a28abef7d8b4569/html5/thumbnails/2.jpg)
Problems of the old RPC
service• Proprietary server/client architecture to
wrap RPCs
• Shortcomings:
– Blocking calls without timeouts
– Poor exception handling
– Error-prone programming abstraction
– Limited scalability
flink.apache.org 1
![Page 3: Apache Flink - Akka for the Win!](https://reader035.fdocuments.in/reader035/viewer/2022071814/55a69ade1a28abef7d8b4569/html5/thumbnails/3.jpg)
Akka – Actor-based
concurrency• Framework to write distributed and highly
reactive applications
• Actor-based concurrency inspired by Erlang’sactor model
• Open source, Apache License 2.0
2flink.apache.org
![Page 4: Apache Flink - Akka for the Win!](https://reader035.fdocuments.in/reader035/viewer/2022071814/55a69ade1a28abef7d8b4569/html5/thumbnails/4.jpg)
Actor model
flink.apache.org 3
![Page 5: Apache Flink - Akka for the Win!](https://reader035.fdocuments.in/reader035/viewer/2022071814/55a69ade1a28abef7d8b4569/html5/thumbnails/5.jpg)
How can Akka help us?
• Nice programming abstraction
• Asynchronous messages with callbacks
• Location transparency of actors
• Fault-tolerant and self-healing
• High throughput and scalability
• Smaller code base to maintain
flink.apache.org 5
![Page 6: Apache Flink - Akka for the Win!](https://reader035.fdocuments.in/reader035/viewer/2022071814/55a69ade1a28abef7d8b4569/html5/thumbnails/6.jpg)
Akka’s integration with Flink
• Job/TaskManager and JobClient are actors
• Old RPC are replaced by messages
• Easy to extend functionality by defining new messages
flink.apache.org 6
JobClientJobManag
er
TaskManag
er
TaskManag
er
Job
![Page 7: Apache Flink - Akka for the Win!](https://reader035.fdocuments.in/reader035/viewer/2022071814/55a69ade1a28abef7d8b4569/html5/thumbnails/7.jpg)
Possible improvements
• Make more components an actor to
increase parallelism
– Scheduler/InstanceManager
– ExecutionGraph/Execution
• Make the system more reactive by
asynchronous calls
• Use Akka persistence to recover state of
failed actors
flink.apache.org 7
![Page 8: Apache Flink - Akka for the Win!](https://reader035.fdocuments.in/reader035/viewer/2022071814/55a69ade1a28abef7d8b4569/html5/thumbnails/8.jpg)
flink.apache.org
github.com/apache/flink
meetup.com/Apache-Flink-
Meetup