Finagle & Clojure - events.static.linuxfound.org · Attendify Use Case ‣ Private social networks...
35
Finagle & Clojure by Alexey Kachayev for #FinagleCon 2015
-
Upload
truongnguyet -
Category
Documents
-
view
220 -
download
0
Transcript of Finagle & Clojure - events.static.linuxfound.org · Attendify Use Case ‣ Private social networks...
Finagle & Clojure
by Alexey Kachayev for #FinagleCon 2015
About Me‣ Alexey Kachayev, @kachayev
‣ CTO at Attendify.com
• Almost-all-day-coding-kind-of-CTO
‣ Active open source contributor
Attendify Use CaseDetailed Event information.
Private event community and social network.
A fully featured event app with
essential content.
Attendify Use Case‣ Private social networks for events (thousands)
‣ A lot of microservices both in Scala & Clojure
‣ Finagle to run services written in Scala
‣ Finagle to run services written in Clojure (???)
finagle-clojure‣ https://github.com/finagle/finagle-clojure
‣ “A thin Clojure wrapper around Finagle”
‣ Scala interop
‣ Bridge to Thrift/ThriftMux and more
‣ lein template (quick start)
lein template$ lein new finagle-‐clojure schedule $ tree -‐L 2 . !"" schedule #"" README.md #"" project.clj #"" schedule-‐client #"" schedule-‐core !"" schedule-‐service
Thrift definitionnamespace java schedule.thrift
struct SessionRequest { 1: string id }
struct Session { 1: string id 2: string title 3: string speaker }
service Schedule { Session fetchSession(1: SessionRequest req) }
Service implementation(ns schedule.service (:import [schedule.thrift Schedule Session]) (:require [finagle-‐clojure.futures :as f] [finagle-‐clojure.thrift :as thrift]) (:gen-‐class))
(defn make-‐service [] (thrift/service Schedule (fetchSession [req] (let [id (.id req)] (f/value (Session. id "Clojure" "Alexey"))))))
(defn -‐main [& args] (f/await (thrift/serve ":9999" (make-‐service))))
Client implementation(ns schedule.client (:import [schedule.thrift Schedule]) (:require [finagle-‐clojure.futures :as f] [finagle-‐clojure.thrift :as thrift]))
(defn make-‐client [address] (thrift/client address Schedule))
(defn session-‐title [client id] (-‐> (.fetchSession client (SessionRequest. id)) (f/map [v] (.title v))))
Client implementation
(defn speaker-‐name [client session-‐id] (-‐> (.fetchSession client (SessionRequest. session-‐id)) (f/flatmap [v] (let [speaker-‐id (.speaker v) req (SpeakerRequest. speaker-‐id)] (-‐> (.fetchSpeaker client req) (f/map [v] (.firstName v)))))))
(defn -‐main [& args] (let [c (make-‐client "localhost:9999")] (println (f/await (session-‐title c "42"))) (println (f/await (speaker-‐name c "42")))))
finagle-clojure?
‣ Scala API is not idiomatic for Clojure
• easy to translate examples from the Internet
• hard to play with the rest of the code
‣ Should we write Scala code using Clojure syntax?
Everything is a Data‣ Builder API for servers & clients
‣ Hash-maps instead chained mutators
‣ Hash-maps instead of HTTP Request/Response builders
Not A Clojure(-‐> (builder-‐server/builder) (builder-‐server/codec http-‐codec/http) (builder-‐server/bind-‐to 3000) (builder-‐server/named “test”) (builder-‐server/build hello-‐world))
(new-‐server hello-‐world {:codec http-‐codec/http :tls nil :max-‐request-‐size 200 :max-‐response-‐size 400 :bind-‐to 3000 :name :test})
Clojure(-‐> (builder-‐server/builder) (builder-‐server/codec http-‐codec/http) (builder-‐server/bind-‐to 3000) (builder-‐server/named “test”) (builder-‐server/build hello-‐world))
(new-‐server hello-‐world {:codec http-‐codec/http :tls nil :max-‐request-‐size 200 :max-‐response-‐size 400 :bind-‐to 3000 :name :test})
Scala Futures in Clojure‣ Two branches of execution (success & errors)
‣ Exception-driven errors handling
‣ Hard to manage execution context
‣ There is a better way to work with chaining
Clojure Futures‣ A lot of “native” primitives:
• futures, promises
• agents
• pmap, parallel reducers
‣ Clojure Futures are not that great for doing I/O
Manifold‣ https://github.com/ztellman/manifold
‣ Abstraction for event-based async programming
‣ Streams & deferreds
‣ Deferreds: @, zip, chain, success, catch
‣ Pluggable concurrent executors
Finagle & Deferred(def req (SessionRequest. "42")) (def f1 (.fetchSession client req)) (def d4 (d/deferred)) (f/on-‐success f1 [v] (d/success! d4 v)) (f/on-‐failure f1 [v] (d/error! d4 v))
user> @d4 user> #object[Session[…]] user> @(d/chain d4 #(.title %)) user> "Finagle & Clojure"
Finagle & Manifold(defn future-‐>deferred [sf] (let [d1 (d/deferred)] (f/on-‐success sf [v] (d/success! d1 v)) (f/on-‐failure sf [v] (d/error! d1 v)) d1))
(defn fetch-‐session [c id] (f-‐>d (.fetchSession c (SessionRequest. id))))
(defn fetch-‐speaker [c id] (f-‐>d (.fetchSpeaker c (SpeakerRequest. id))))
Let-Flow(defn speaker-‐name [c session] (let-‐flow [session (fetch-‐session c session) sid (.speaker session) speaker (fetch-‐speaker c sid)] (str (.firstName speaker) " " (.lastName speaker))))
user> (speaker-‐name client "42") user> #object[Deferred] user> @(speaker-‐name client "42") user> "Alexey Kachayev"
Let-Flow(defn speaker-‐name [c session] (let-‐flow [session (fetch-‐session c session) sid (.speaker session) speaker (fetch-‐speaker c sid)] (str (.firstName speaker) " " (.lastName speaker))))
valuesdeferreds
core.async‣ https://github.com/clojure/core.async
‣ “Library designed to provide facilities for async programming and communication”
‣ CSP as a library
‣ go macro to turn async code into a state machine
‣ Widely adopted by Clojure community
Finagle & core.async(defn future-‐>chan [sf] (let [c1 (a/chan 1)] (f/on-‐success sf [v] (a/put! c1 v)) (f/on-‐failure sf [v] (a/put! c1 (e/left v))) c1))
(defn fetch-‐session [c id] (f-‐>c (.fetchSession c (SessionRequest. id))))
(defn fetch-‐speaker [c id] (f-‐>c (.fetchSpeaker c (SpeakerRequest. id))))
Finagle & core.async(defn speaker-‐name [c session-‐id] (go (let [session (<! (fetch-‐session c session-‐id)) speaker-‐id (.speaker session) speaker (<! (fetch-‐speaker c speaker-‐id))] (str (.firstName speaker) " " (.lastName speaker)))))
user> (speaker-‐name client "42") user> #object[ManyToManyChannel] user> (<!! (speaker-‐name client "42")) user> "Alexey Kachayev"
Finagle & core.async(defn speaker-‐name [c session-‐id] (go (let [session (<! (fetch-‐session c session-‐id)) speaker-‐id (.speaker session) speaker (<! (fetch-‐speaker c speaker-‐id))] (str (.firstName speaker) " " (.lastName speaker)))))
statemachine
What About Server(:import [com.twitter.util Promise])
(defn propagate-‐to [promise value] (if (e/left? value) (.setException promise (e/left-‐value value)) (.setValue promise (e/right-‐value value))))
(defn chan-‐>future [c] (let [promise (Promise.)] (a/take! c (partial propagate-‐to promise)) promise))
What About Server(:import [com.twitter.util Promise])
(defn propagate-‐to [promise value] (if (e/left? value) (.setException promise (e/left-‐value value)) (.setValue promise (e/right-‐value value))))
(defn chan-‐>future [c] (let [promise (Promise.)] (a/take! c (partial propagate-‐to promise)) promise))
Finagle & core.async‣ Clean & concise async code
‣ A lot of built-in primitives (timeout, pub/sub etc)
‣ Compatibility with tons of Clojure libraries based on core.async
‣ Limited usage in request-reply world though
Stitch → Muse‣ Stitch is a Scala library for composing RPC services
‣ Created in Twitter, introduces by Jake Donham
‣ As Stitch is not open sourced…
‣ github.com/kachayev/muse
‣ EuroClojure talk about it
Stitch → Muse‣ Runs independent data fetches concurrently
• Uses BFS to group fetches level-by-level
‣ Caches previously made fetches during execution
‣ Batches requests when applicable
‣ Uses the idea of building and interpreting AST
‣ Uses core.async to deal with concurrency
More Clojure Codec‣ Scala has Scodec & finagle-serial
‣ Clojure has Fressian
• Designed with EDN in mind
• Rich set of core types & extensions
• Middleware-friendly with tagged objects
‣ Working on Fressian codec right now
EDN & Fressian
{:id 42 :title "Finagle & Clojure" :tracks ["Libraries" "Practice"] :tags #{"Finagle" "Clojure"} :speaker {:name "Alexey Kachayev"} :at #inst "2015-‐08-‐13T14:30:00" :duration #duration "10s"}
EDN & Fressian
{:id 42 :title "Finagle & Clojure" :tracks ["Libraries" "Practice"] :tags #{"Finagle" "Clojure"} :speaker {:name "Alexey Kachayev"} :at #inst "2015-‐08-‐13T14:30:00" :duration #duration "10s"}
tags
Conclusion‣ Finagle is great
‣ Clojure is amazing
‣ Finagle can solve a lot of problems for Clojure ecosystem
‣ Finagle & Clojure integration still requires a lot of work
Thank You!Questions?
