StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK
-
Upload
storageos -
Category
Technology
-
view
132 -
download
2
Transcript of StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK
![Page 1: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/1.jpg)
Storage for containers shouldn’t
be annoying
Karolis Rusenas@KRusenas
https://github.com/[email protected]
![Page 2: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/2.jpg)
Agile Storage. On-Demand, Anywhere, Made Easy.
What we care about
Developer experience Use existing tools (create and use Docker volumes without ever interacting with StorageOS directly
(UI/CLI/API). Reasonable defaults - compression, encryption, replication…
Operating experience Run StorageOS as a container, separate image or whatever way you like, it’s just a binary! :) API first - UI and CLI reuses same single HTTP API, easy to automate stuff. Pluggable configuration backends for whatever you have in your stack - Consul, Zookeeper, BoltDB,
Etcd (easy to add custom ones). Security Perfomance
![Page 3: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/3.jpg)
Challenges• Persistent storage
• Networking
• Security
• Infrastructure
• Orchestration
![Page 4: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/4.jpg)
Our targets• Persistent storage
• Networking
• Security
• Infrastructure
• Orchestration
![Page 5: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/5.jpg)
Persistent storage
![Page 6: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/6.jpg)
Jim was almost ready to implement his first app with persistent storage running in Docker
container
![Page 7: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/7.jpg)
How easy is it to get persistent storage with
StorageOS?
![Page 8: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/8.jpg)
![Page 9: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/9.jpg)
That’s the line
• sudo docker run -d --name test-redis01 -v test-dev-redis01:/data --volume-driver=storageos redis redis-server --appendonly yes
Usual stuff
Let’s give our new friend a name
Telling Docker to use StorageOS
driverMore of the usual
stuff
![Page 10: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/10.jpg)
![Page 11: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/11.jpg)
Infrastructure
![Page 12: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/12.jpg)
Infrastructure• Cloud providers such as GCE or AWS provide
EBS volumes for persistent storage.
![Page 13: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/13.jpg)
You can always build something on your own…
![Page 14: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/14.jpg)
On-premises infrastructure &
StorageOS• Benefits of containers without moving to cloud by providing EBS volume
alternative via Docker volume plugin.
• Hyper-converged mode - run your workloads on the same nodes as StorageOS controllers for maximum performance.
• Client mode - present virtual volumes to Docker containers, easy access to remote volumes.
• QoS
• Compression
• Data deduplication
• E2E encryption
![Page 15: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/15.jpg)
Agile Storage. On-Demand, Anywhere, Made Easy.
StorageOS Use Cases
Stateful Containers for Databases and Fast DB recovery
Continuous Integration/Delivery
Secure Cloud Mobility and Cost Reduction Performance Acceleration and Volume Management
API
![Page 16: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/16.jpg)
StorageOS insides
![Page 17: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/17.jpg)
Controller Node
control plane
HTTP API NatsUI
(Angular)
Docker
pluginSchedule
rDataplane drivers
Supervisor 5 secret binaries
Configfs
data plane
![Page 18: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/18.jpg)
Controller Node
control plane
HTTP API NatsUI
(Angular)
Docker
pluginSchedule
rDataplane drivers
Supervisor 5 secret binaries
Configfs
Client Node
control plane
Docker
pluginNats HA
Client
Supervisor 5 secret binaries
data plane
![Page 19: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/19.jpg)
controplane
High level overview
controplane controplane
dataplane dataplane dataplane
nats
Consul Consul Consul
http
clienthttp/nats
Docker dataplane
server
![Page 20: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/20.jpg)
Our stack• Consul/BoltDB - store configuration. BoltDB is
useful when running a single node or during development/testing.
• Nats - messaging system
• Go - control plane is written in this awesome language
• C - data plane, mostly for speed and available libraries
![Page 21: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/21.jpg)
Consul/BoltDB
KV store, easy to use, backup. Service discovery (when using BoltDB
it’s not important since we assume that you are running a single node)
Leadership election - some components of the system should be running only on one node so they are all fighting for leadership (i.e. scheduler, retry logic).
Split brain detection
![Page 22: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/22.jpg)
Leadership election.. why?
Sometimes (and quite often) you need only one node in your distributed system performing specific actions, i.e. scheduling, retrying some actions..
Most of the distributed KV stores implement locking mechanism on keys, that could be used to elect leaders and detect leader failures.
Check out https://github.com/docker/leadership - probably not enough code there to include it as a library though.
![Page 23: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/23.jpg)
Node 1 Node 2 Node 3 Node 4 Node 5
Leader’s key
All nodes try to acquire a lock on specific key
![Page 24: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/24.jpg)
Node 1(leader) Node 2 Node 3 Node 4 Node 5
Leader’s key
Only one will succeed
P.S. Don’t forget to use locks with TTL!
![Page 25: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/25.jpg)
Nats (https://nats.io)
Lightweight Server is just a goroutine in your main
process Instant messaging between
components Simple pub/sub or request/reply
syntax Mesh networking
![Page 26: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/26.jpg)
Nats (https://nats.io) auto discovery:https://github.com/nats-io/gnatsd/issues/126
![Page 27: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/27.jpg)
Safer messaging
Messaging wrapper
Dispatch event
Persist event in KV store
Publish event to
Nats
Send event:
Receive event:
Receive event
Update event to “receive”
stateDo stuff
Then we can have a process that constantly looks for forgotten events :)
![Page 28: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/28.jpg)
Statik (https://github.com/rakyll/statik)
Compile your static assets into Go binary!
![Page 29: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/29.jpg)
Suture (https://github.com/thejerf/suture)
Erlang-ish supervisor trees for Go Stable Combined with statik and go-bindata
you can drastically simplify your deployments
![Page 30: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/30.jpg)
Using Docker’s healthcheck functionality (added in 1.12)
https://docs.docker.com/engine/reference/builder/#/healthcheck
Dockerfile:
$docker ps
Useful when you have several dependencies, like KV store.
![Page 31: StorageOS, Storage for Containers Shouldn't Be Annoying at Container Camp UK](https://reader036.fdocuments.in/reader036/viewer/2022062522/58a3a2191a28abb1348b6fb3/html5/thumbnails/31.jpg)
Thanks!@Storage_OS
We are hiring!http://storageos.com/careers/