Lewis Marshall - Flynn at #DOXLON
-
Upload
dataloopio -
Category
Technology
-
view
1.036 -
download
1
description
Transcript of Lewis Marshall - Flynn at #DOXLON
![Page 1: Lewis Marshall - Flynn at #DOXLON](https://reader035.fdocuments.in/reader035/viewer/2022073101/5591fb781a28abea658b4710/html5/thumbnails/1.jpg)
Flynn
![Page 2: Lewis Marshall - Flynn at #DOXLON](https://reader035.fdocuments.in/reader035/viewer/2022073101/5591fb781a28abea658b4710/html5/thumbnails/2.jpg)
What is Flynn?
Flynn is a set of components for solving ops
![Page 3: Lewis Marshall - Flynn at #DOXLON](https://reader035.fdocuments.in/reader035/viewer/2022073101/5591fb781a28abea658b4710/html5/thumbnails/3.jpg)
What problem needs solving?
● Deploy a web app to multiple servers
● Upgrading dependencies
● Deploy a fault tolerant, scalable database cluster
![Page 4: Lewis Marshall - Flynn at #DOXLON](https://reader035.fdocuments.in/reader035/viewer/2022073101/5591fb781a28abea658b4710/html5/thumbnails/4.jpg)
postgres-db flynn-controller flynn-scheduler
gitreceive
blobstore postgres-api router-api
router-http router-tcpLAYER 1
APPLICATION + DATABASE AWARE SERVICES
LAYER 0flynn-hostdiscoverd
etcd
JOB SCHEDULING + SERVICE DISCOVERY API
flynn-hostdiscoverd
etcd
flynn-hostdiscoverd
etcd
Flynn is divided into two layers
![Page 5: Lewis Marshall - Flynn at #DOXLON](https://reader035.fdocuments.in/reader035/viewer/2022073101/5591fb781a28abea658b4710/html5/thumbnails/5.jpg)
Layer 0: discoverd + etcd
● provides service discovery to the cluster
● runs as a daemon on every node
● listens on port 1111
● communicates with other nodes via etcd
● determine a “leader” for any given service
![Page 6: Lewis Marshall - Flynn at #DOXLON](https://reader035.fdocuments.in/reader035/viewer/2022073101/5591fb781a28abea658b4710/html5/thumbnails/6.jpg)
Layer 0: discoverd + etcd
● register a service as online by periodically sending name, address and attributes to the daemon
● [“redis”, “192.168.1.2:6379”, “password” => “s3cr3t”]
● subscribe to services to be notified when they scale up / down
![Page 7: Lewis Marshall - Flynn at #DOXLON](https://reader035.fdocuments.in/reader035/viewer/2022073101/5591fb781a28abea658b4710/html5/thumbnails/7.jpg)
Layer 0: flynn-host
● runs as a daemon on all nodes
● responsible for starting and tracking local jobs
● pluggable job backend (docker / libvirt-lxc)
● leader holds the state of the cluster
● all job requests sent to the leader
![Page 8: Lewis Marshall - Flynn at #DOXLON](https://reader035.fdocuments.in/reader035/viewer/2022073101/5591fb781a28abea658b4710/html5/thumbnails/8.jpg)
Layer 1
● consists of ~10 components
● loosely coupled
● provides Heroku-like git push deploys
![Page 9: Lewis Marshall - Flynn at #DOXLON](https://reader035.fdocuments.in/reader035/viewer/2022073101/5591fb781a28abea658b4710/html5/thumbnails/9.jpg)
Layer 1: controller
● exposes a JSON REST API to:● create / delete applications● create releases for applications● scale releases● run custom jobs● stream job events● create / delete routes
![Page 10: Lewis Marshall - Flynn at #DOXLON](https://reader035.fdocuments.in/reader035/viewer/2022073101/5591fb781a28abea658b4710/html5/thumbnails/10.jpg)
Layer 1: postgresql
● Publishes credentials via discoverd
● Exposes a REST API for creating users and databases
● Automatic failover via leader changes
![Page 11: Lewis Marshall - Flynn at #DOXLON](https://reader035.fdocuments.in/reader035/viewer/2022073101/5591fb781a28abea658b4710/html5/thumbnails/11.jpg)
Layer 1: scheduler
● responsible for scaling an apps processes
● watches for changes in PostgreSQL
● runs on every node, with only the leader actually scheduling jobs
![Page 12: Lewis Marshall - Flynn at #DOXLON](https://reader035.fdocuments.in/reader035/viewer/2022073101/5591fb781a28abea658b4710/html5/thumbnails/12.jpg)
Layer 1: gitreceived
● An SSH server made specifically for accepting git pushes
● auth script: authchecker $USER $KEY
● receive script: receiver $PATH $COMMIT
![Page 13: Lewis Marshall - Flynn at #DOXLON](https://reader035.fdocuments.in/reader035/viewer/2022073101/5591fb781a28abea658b4710/html5/thumbnails/13.jpg)
Layer 1: flynn-receive
● uses the controller to check user & key
● uses Heroku buildpacks to compile a slug and detect process types
● creates an app release
![Page 14: Lewis Marshall - Flynn at #DOXLON](https://reader035.fdocuments.in/reader035/viewer/2022073101/5591fb781a28abea658b4710/html5/thumbnails/14.jpg)
Layer 1: blobstore
● simple, fast HTTP file service
curl -X PUT -T file http://blobstore/path/to/file
curl http://blobstore/path/to/file
curl -X DELETE http://blobstore/path/to/file
![Page 15: Lewis Marshall - Flynn at #DOXLON](https://reader035.fdocuments.in/reader035/viewer/2022073101/5591fb781a28abea658b4710/html5/thumbnails/15.jpg)
Layer 1: router
● provides HTTP & TCP routing / load balancing to services
● uses discoverd to detect backends● provides REST API for managing routes● stores routes in etcd for high availability● runs on every node
![Page 16: Lewis Marshall - Flynn at #DOXLON](https://reader035.fdocuments.in/reader035/viewer/2022073101/5591fb781a28abea658b4710/html5/thumbnails/16.jpg)
DEMO
![Page 17: Lewis Marshall - Flynn at #DOXLON](https://reader035.fdocuments.in/reader035/viewer/2022073101/5591fb781a28abea658b4710/html5/thumbnails/17.jpg)
TRY IT OUT!
https://flynn.io#flynn on Freenode
github.com/flynn/flynn