4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language
-
Upload
antoniogarrote -
Category
Technology
-
view
1.278 -
download
0
Transcript of 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language
![Page 1: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/1.jpg)
Jobim: an Actors Library for the Clojure Programming Language
Antonio GarroteMaría N. Moreno García
![Page 2: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/2.jpg)
Clojure
• New Lisp dialect
• Hosted: JVM, CLR
• Focus on concurrency: (STM, Agents, futures, fork-join, java.util.concurrent.*)
![Page 3: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/3.jpg)
Distributed concurrent applications in Clojure?
“Given the diversity, sophistication, maturity, interoperability,robustness etc of these options, it's unlikely I'm going to fiddle around with some language-specific solution.”
Rich Hickey
![Page 4: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/4.jpg)
Distributed applications on the JVM
• JINI
• Java Spaces
• JMS
• Terracota
• RabbitMQ
• ZeroMQ
• Gearman
![Page 5: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/5.jpg)
Problems
• Different computational models
• Hard to port code from one solution to other
• Friction with Clojure semantics
![Page 6: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/6.jpg)
Extending Clojure
• Computational model?
• Suitable notation?
• Underlying implementation?
![Page 7: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/7.jpg)
![Page 8: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/8.jpg)
Computational Model
![Page 9: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/9.jpg)
Actors: components
• Named channel (PID)
• Message box
• Execution context
PID
MBox ExecutionContext
Computation based on the exchange of messages
![Page 10: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/10.jpg)
Actors: mobile processes
PID1
MBox ExecutionContext
PID2
MBox ExecutionContext
PID3
MBox ExecutionContext
PID3 PID3
Msg
![Page 11: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/11.jpg)
Notation
![Page 12: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/12.jpg)
Actors: minimal interface
(def *pid* (spawn f))
(send! *pid* msg)
(def msg (receive *pid*))
(is (= *pid* (self)))
![Page 13: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/13.jpg)
Selective Reception(let [p (promise) pid (spawn #(let [a (receive odd?) b (receive even?)] (deliver p [a b])))]
(send! pid 2) (send! pid 1)
(is (= @p [1 2])))
![Page 14: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/14.jpg)
Implementation
![Page 15: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/15.jpg)
Message Box
• Lamina - Z. Tellman (https://github.com/ztellman/lamina)
• Transforms Clojure sequences into event-driven channels
• Synchronous and asynchronous interface
![Page 16: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/16.jpg)
PIDs/Channel Names
• Plain Java strings: easy to exchange
• Globally unique identifiers for actors across all the nodes in a Jobim cluster
• Generated by Jobim’s runtime
• GUID node + process counter
![Page 17: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/17.jpg)
Execution context
• Threaded actors:
- Java thread per actor
- End of thread execution, releases resources
- Threaded actors do not scale (~2000 threads per node max.)
![Page 18: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/18.jpg)
Execution context
• Evented actors (Scala):
- Actors context = closure + callback functions
- Small number of reactor threads execute all the evented threads
![Page 19: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/19.jpg)
(spawn #(loop [] (let [[from msg] (receive)] (send! from msg) (recur))))
(spawn-evented #(react-loop [] (react [[from msg]] (send! from msg) (react-recur))))
Threaded
Evented
![Page 20: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/20.jpg)
Multiplexer
Reactor1 ReactorN...
Events
Messages
Callbacks+Contexts queues react-loop
react-recur
listen-once
publish
react-future
thread-pool handler
publish
![Page 21: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/21.jpg)
Distribution
![Page 22: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/22.jpg)
3 problems
Messaging Coordination Serialization
![Page 23: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/23.jpg)
Modular solution
Messaging Coordination Serialization
Protocols: jobim.services.*
Plugins
![Page 24: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/24.jpg)
• Name service
• Membership groups: presence
• Distributed agreement: 2PC protocol
Coordination
![Page 25: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/25.jpg)
Coordination
• Apache ZooKeeper plugin
- Light-weight
- Scalable
- Small set of primitives to build sophisticated coordination protocols
![Page 26: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/26.jpg)
Coordination: name service - group membership
(nodes)
(resolve-node-name node-name)
(register-name name *pid*)
(def *pid* (resolve-name name))
![Page 27: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/27.jpg)
Coordination: 2PC - group membership
(link *pid1* *pid2*)
Signal
![Page 28: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/28.jpg)
Messaging
• TCP plugin: Netty, Z.Tellman’s Aleph [https://github.com/ztellman/aleph]
• RabbitMQ plugin
• ZeroMQ plugin
![Page 29: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/29.jpg)
Serialization
• Java Serialization plugin
• JSON plugin
• Kryo serialization library plugin (Yahoo S4)
![Page 30: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/30.jpg)
Behaviours
![Page 31: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/31.jpg)
Reusing distributed components?
• Encapsulate distributed patterns
• Hide message passing logic
• Building blocks for larger distributed systems
• Built using Clojure protocols
• Threaded and evented versions
![Page 32: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/32.jpg)
Behaviours
• Supervisor
• Generic Server
• FSM
• Event Manager / Event Handler
• Generic TCP server
![Page 33: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/33.jpg)
;; FSM Lock type
(def-fsm Lock (init [this code] [:locked {:so-far [] :code code}]) (next-transition [this state-name state-data message] (let [[topic _] message] (condp = [state-name topic] [:locked :button] handle-button [:open :lock] handle-lock action-ignore))) (handle-info [this current-state current-data message] (do (cond-match [[?from :state] message] (send! from current-state)) (action-next-state current-state current-data))))
![Page 34: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/34.jpg)
Demo
![Page 35: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/35.jpg)
Future work
• Benchmarking + Performance
• Packaging / deployment / managing of distributed apps
• Missing functionality
![Page 36: 4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language](https://reader034.fdocuments.in/reader034/viewer/2022052619/55582357d8b42a25588b4c98/html5/thumbnails/36.jpg)
Code+deps
• https://github.com/antoniogarrote/jobim
[jobim-core "0.1.2-SNAPSHOT"]
[jobim-rabbitmq "0.1.1-SNAPSHOT"]
[jobim-zeromq "0.1.1-SNAPSHOT"] *