Microservices in Clojure
-
Upload
lucas-cavalcanti -
Category
Software
-
view
323 -
download
0
Transcript of Microservices in Clojure
![Page 1: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/1.jpg)
Lucas Cavalcanti@lucascs
Microservices in Clojure
![Page 2: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/2.jpg)
ContextMicroservices
~80 Clojure services ~60 engineers
~10 teams 3.5 years old
![Page 3: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/3.jpg)
OOP Objects, the mainstream abstraction
Image @ http://www.eduardopires.net.br/2015/01/solid-teoria-e-pratica/
![Page 4: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/4.jpg)
What about Functional Programming?
SÃO PAULO, BRASIL
![Page 5: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/5.jpg)
TABLE OF CONTENTS
Immutability Components Pure Functions Schemas Ports and Adapters
SÃO PAULO, BRASIL
![Page 6: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/6.jpg)
Immutability
SOUTHEAST BRAZIL REGION FROM SPACE
![Page 7: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/7.jpg)
Immutability Definition
“If I’m given a value, it’s guaranteed that it won’t ever change”
![Page 8: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/8.jpg)
Technology choices Immutability
![Page 9: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/9.jpg)
Clojure Immutability
All default data structures are immutable: -Maps, Lists, Sets -Records
Mutability is explicit: atoms/refs @, dynamic vars *…*
![Page 10: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/10.jpg)
Datomic Immutability
Datomic stores the changes/transactions, not just the data -append only -db as a value -everything is data (transaction, schema, entities)
![Page 11: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/11.jpg)
Kafka Immutability
Persistent Queues/Topics -each consumer has its offset -ability to replay messages
![Page 12: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/12.jpg)
AWS + Docker Immutability
Ability to spin machines with a given image/configuration -Each build generates a docker image -Each deploy spins a new machine with the new
version -As soon as the new version is healthy, old version is
killed. (blue-green deployment)
![Page 13: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/13.jpg)
Components
SOUTHEAST BRAZIL REGION FROM SPACE
![Page 14: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/14.jpg)
Components https://github.com/stuartsierra/component
(defprotocol Database (query [this query-str]))(defrecord SomeDatabase [config-1 config-2 other-components] component/Lifecycle (start [this] (assoc this :connection (connect! config-1 config-2 other-components))) (stop [this] (release! (:connection this)) (dissoc this :connection)) Database (query [this query-str] (do-query! (:connection this) query-str)))
![Page 15: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/15.jpg)
System map Components
{:database #SomeDatabase{...} :http-client #HttpClient{...} :kafka #Kafka{...} :auth #AuthCredentials{...} ...}
-Created at startup -Entrypoints (e.g http server or kafka consumers) have access to all
components the business flows need -dependencies of a given flow are threaded from the entry point until
the end, one by one if possible -Thus no static access to system map! (e.g via a global atom) -Any resemblance to objects and classes is just coincidence ;)
![Page 16: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/16.jpg)
Pure functions
SOUTHEAST BRAZIL REGION FROM SPACE
![Page 17: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/17.jpg)
Pure functions Definition
"Given the same inputs, it will always produce the same output"
![Page 18: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/18.jpg)
Simplicity Pure functions
-easier to reason about, fewer moving pieces -easier to test, less need for mocking values -parallelizable by default, no need for locks or STMs
![Page 19: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/19.jpg)
Datomic Pure functions
-Datomic’s db as a value allows us to consider a function that queries the database as a pure function -db is a snapshot of the database at a certain point in time. -So, querying the same db instance will always produce the same result
![Page 20: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/20.jpg)
Impure functions Pure functions
-functions that produce side effects should be marked as such. We use `!` at the end. -split code which handles and transforms data from code that handles side effects -should be moved to the borders of the flow, if possible -Consider returning a future/promise like value, so side effect results can be composed (e.g with manifold or finagle)
https://github.com/ztellman/manifoldhttps://github.com/twitter/finagle
![Page 21: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/21.jpg)
Schema/Spec
SOUTHEAST BRAZIL REGION FROM SPACE
![Page 22: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/22.jpg)
Schema Legacy
Majority of our code base was written before clojure.spec existed, so I’ll be talking about the Schema library instead. Most principles apply to clojure.spec as well.
![Page 23: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/23.jpg)
Schema/Spec Documentation
-Clojure doesn’t force you to write types -parameter names are not enough -declaring types helps a lot when glancing at the function -values can be verified against a schema
![Page 24: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/24.jpg)
Function declaration Schema/spec
-All pure functions declare schemas for parameters and return value -All impure functions declare for parameters and don’t declare output type if it’s not relevant. -Validated at runtime in dev/test environments, on every function call -Validation is off on production.
![Page 25: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/25.jpg)
Wire formats Schema/Spec
-Internal schemas are your domain models -Wire schemas are how you expose data to other services/clients -If they are different, you can evolve internal schemas without breaking clients -Need an adapter layer -wire schemas are always validated on entry/exit points, specially in production -single repository for all wire schemas (for all 60+ services) -caveat: this repository has a really high churn. Beware
![Page 26: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/26.jpg)
Growing Schemas Spec-ulation
Please watch Rich Hickey’s talk at Clojure Conj 2016 Spec-ulation:
https://www.youtube.com/watch?v=oyLBGkS5ICk
![Page 27: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/27.jpg)
Ports and Adapters (a.k.a Hexagonal Architecture)
SOUTHEAST BRAZIL REGION FROM SPACE
![Page 28: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/28.jpg)
Ports and Adapters Definition
Core logic is independent to how we can call it (yellow) A port is an entry-point of the application (blue) An adapter is the bridge between a port and the core logic (red)
http://www.dossier-andreas.net/software_architecture/ports_and_adapters.htmlhttp://alistair.cockburn.us/Hexagonal+architecture
![Page 29: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/29.jpg)
Ports and Adapters (Nubank version) Extended Definition
Pure business logic (green) Controller logic wires the flow between the ports (yellow) A port is an entry-point of the application (blue) An adapter is the bridge between a port and the core logic (red)
![Page 30: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/30.jpg)
Ports (Components) Ports and Adapters
-Ports are initialised at startup -Each port has a corresponding component -Serializes data to a transport format (e.g JSON, Transit) -Usually library code shared by all services -Tested via integration tests
HTTP
Kafka
Datomic
File Storage
Metrics
![Page 31: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/31.jpg)
Adapters (Diplomat) Ports and Adapters
-Adapters are the interface to ports
-Contain HTTP and Kafka consumer handlers
-Adapt wire schema to internal schema
-Calls and is called by controller functions
-Tested with fake versions of the port components, or mocks
HTTP
Kafka
Datomic
File Storage
Metrics
![Page 32: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/32.jpg)
Controllers Ports and Adapters
-Controllers wires the flow between entry-point and the side effects
-Only deals with internal schemas
-Delegates business logic to pure functions
-Composes side effect results -Tested mostly with mocks
HTTP
Kafka
Datomic
File Storage
Metrics
![Page 33: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/33.jpg)
Business Logic Ports and Adapters
-Handles and transforms immutable data
-Pure functions -Best place to enforce invariants and type checks (e.g using clojure.spec)
-Can be tested using generative testing
-Should be the largest part of the application
HTTP
Kafka
Datomic
File Storage
Metrics
![Page 34: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/34.jpg)
Microservices Ports and Adapters
-Each service follows about the same design
-Services communicate with each other using one of the ports (e.g HTTP or Kafka)
-Services DON’T share databases
-HTTP responses contain hypermedia, so we can replace a service without having to change clients
-Tested with end to end tests, with all services deployed
![Page 35: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/35.jpg)
Clojure is simple
Keep your design simple
Keep your architecture simple
SÃO PAULO, BRASIL
![Page 36: Microservices in Clojure](https://reader031.fdocuments.in/reader031/viewer/2022013117/58a2c9a71a28ab217a8b6551/html5/thumbnails/36.jpg)
Lucas Cavalcanti@lucascs
Thank you