Is OpenStack Delivering on the Promise of Transforming Comcasts Business Model?

Consuming Cinder from DockerJohn GriffithSolidFire/NetAppAugust 23, 2016

About meContributing to OpenStack a while (longer than some, less than others)

Have been pretty passionate about core Cinder and things like reference driver

Worked at a little Storage Startup called SolidFire recently bought by NetApp

Have an AWESOME job where I get to work on OSS and always try new things

Can be opinionated and blunt at times sorry about that

2

2

Maybe you showed up because you heard rumors that Id talk about Docker?

#

OpenStack Cinder -Block Storage as a ServiceAbstraction layer to provide a pool of Block Resources

Use backend storage from different vendors, dont have to care of know whats actually serving up the blocks

Scale out just keep plugging in back-ends

Scheduler can figure out where to place volumes for you

#

Its like having an infinite number of disks That you can hot plug in and out of your Instances

Photo Credit: Roger Smith

#

Only really need a few thingsCreate/DeleteAttach/DetachSnapshot

#

Of course theres more for those that want itReplicationCGsBackupsMigrationImports/ExportsTransfer ownershipExtend...(This is me trying not to be opinionated and blunt moving on)

#

I was told this would be a Docker talk!!

#

Yes, Im going to talk about DockerDocker is the best geek bait EVER!!!OpenStack in ContainersContainer Orchestration in OpenStackOpenStack on Kubernetes

#

Unicorns for EVERYONEAll sorts of interesting ideas and plans on the horizon

Lets bypass some of the hype and just talk about some cool stuff you can do today

Try to remember a little bit of the past while were at it

#

History repeats itself in tech

$NEW_THING is like pets vs cattle$NEW_THING needs better networking$NEW_THING needs persistent storage$NEW_THING is a different development paradigm$NEW_THING is about small ephemeral servicesecho $NEW_THINGOpenStack

export NEW_THING=Containers

#

Just like we heard in OpenStack Containers need networking and storage optionsVolume Plugin capability for Docker introduced initially in 1.8

Continues to mature

List of Vendors racing to provide a plugin accelerating rapidly

Nobody wants to be late to the party, especially those that were late to Cinder

#

Docker Volume Plugins General things to know

Docker provides a simple Volume APIINCLUDES PROVISIONING!!!!!Driver runs as a daemonMost common right now are simple UNIX domain socketsRuns on same node as the Docker Enginejson-rpc over http POST Works with Swarm, Engine and Compose

#

So I wrote a Cinder PluginWritten in GolangFocus on JUST CinderVendor neutral/independentOpen SourceGladly welcome contributors and feedbackAnticipating/Hoping for Cinder community support14

Cant I already do this?Yep, you can

Cool stuff out there already

Adoption is the greatest compliment

#

Dont get me wrongSome of those existing plugins that wrap up Cinder are pretty cool

Some offer additional benefits

Some might fit your use case better

Some of them you may have already invested in and have relationships with the contributing vendors

Do your thing, thats AWESOME

Dont hate, were all in this together

#

Brace yourselves, its about to get terrifying

Well for a few people at least

#

These Plugins arent under an umbrellaDocker Plugins are NOT in a Docker repo

Cinder Docker Plugin isnt in an OpenStack repo

#

Some OpenStack folks just had a strokeFor now: https://github.com/j-griffith/cinder-docker-driver

Licensed under the unlicense

Potential for inclusion under OpenStack some day? Or maybe Docker?

#

So how does this workIts not much different than how we do things with OpenStack/Nova

Create a volume

Attach a volume

Its all the same stuff weve been doing for years, we just change the consumer

Cinder really doesnt care what youre doing on the other side

By the way, were talking Docker, but it doesnt have to be Docker either

#

I have to give a shout out to Docker on1.12Docker 1.12 was a HUGE step forwardSwarm advancements are my favoriteI can deploy a Swarm cluster wicked fastSwarm in OpenStack or Public Cloud is stupid easy

#

Recipe for a tasty Swarm Cluster with persistent dataStart with some Peanut ButterBasic OpenStack Cloud Compute Networking and Storage

Mix in a bit of ChocolateDocker 1.12

Top it off with some frostingCinder Docker Driver

#

Well use docker-machine with the OpenStack driver Because we can, and it works pretty well

(Our OpenStack Cloud)

#

We have our ingredients, heres the basic stepsdocker-machine to create 3 Nova Instances and setup Docker

Create a Swarm Cluster

Install , configure and start the cinder-docker-driver

Deploy a Swarm service that creates and uses a Cinder Volume

#

Create our nodes use env vars instead of argsWell use docker-machine and the built in OpenStack driver for this

Theres a LOT of arguments required to the cli, so lets start by creating an env file rather than typing everything in.export OS_FLAVOR_ID=2export OS_DOMAIN_NAME=$OS_USER_DOMAIN_NAMEexport OS_IMAGE_ID=d5c276bc-cb70-42c4-9291-96f40a03a74cexport OS_SSH_USER=ubuntuexport OS_KEYPAIR_NAME=jdgexport OS_PRIVATE_KEY_FILE=$HOME/.ssh/id_rsaexport OS_SSH_USER=ubuntuexport OS_TENANT_ID=$OS_PROJECT_ID

#

Create our nodesThis just does our nova boot . for us, creating the Instances based on env varsIt does a few additional things for us too thoughInstall DockerConfigure and Setup certs for DockerVerify Docker is up and runningCreate a node entry in the docker nodes db docker-machine create d openstack swarm-1 docker-machine create d openstack swarm-2 docker-machine create d openstack swarm-3

#

We can view our nodes using docker-machine

docker-machine lsNAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORSswarm-1 - openstack Running tcp://172.16.140.157:2376 v1.12.0swarm-2 - openstack Running tcp://172.16.140.159:2376 v1.12.0swarm-3 - openstack Running tcp://172.16.140.161:2376 v1.12.0

#

Set a node up as a Swarm Manager

eval $(docker-machine env swarm-1) SWARM-1-IP=$(docker-machine ip swarm-1) docker swarm init --advertise-addr $SWARM-1-IP listen-addr $SWARM-1-IP:2377

Swarm initialized: current node (5oi3h06yci5mvsau6czcbbxqu) is now a manager.To add a worker to this swarm, run the following command:

docker swarm join \ --token SWMTKN-1-33zfeg2ppr9043o4itdn2cznwn7yuy7na1fqg2aduoemihw93o\-3znh32dbpmb5goc8l1ia286it 172.16.140.157:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

#

Set our other two Instances as Workers

eval $(docker-machine env swarm-2) docker swarm join --token SWMTKN-1-33zfeg2ppr9043o4itdn2cznwn7yuy7na1fqg2aduoemihw93o-3znh32dbpmb5goc8l1ia286it 172.16.140.157:2377

This node joined a swarm as a worker.

eval $(docker-machine env swarm-3) docker swarm join --token SWMTKN-1-33zfeg2ppr9043o4itdn2cznwn7yuy7na1fqg2aduoemihw93o-3znh32dbpmb5goc8l1ia286it 172.16.140.157:2377

This node joined a swarm as a worker.

#

Thats it, you now have a Swarm Cluster!!!!

#

Now its time to install the Cinder Driver on each NodeInstall the driverCopy over a config file Start the daemonRestart Docker

#

You can just use docker-machine from your laptop

for each in $(docker-machine ls -q); do; docker-machine ssh $each "curl sSL \ https://raw.githubusercontent.com/j-griffith/cinder-docker-driver/master/install.sh \| sh -" ; done

for each in $(docker-machine ls -q); do; docker-machine scp cdd.config.json \ $each:~/config.json; done

for each in $(docker-machine ls -q); do; docker-machine ssh $each "sudo \cinder-docker-driver --config config.json > cdd.log 2>&1 &" ; done

for each in $(docker-machine ls -q); do; docker-machine ssh $each "sudo \Service docker restart" ; done

#

About that install.Config file is just OpenStack CredsCreating a Service file for the driver has made its way up pretty far on the TODO list

{ "Endpoint": "http://172.16.140.243:5000/v2.0", "Username": "jdg", "Password": ABC123", "TenantID": "3dce5dd10b414ac1b942aba8ce8558e7}

#

Now you can do cool things

Lets build the simple counter application

Redis container with Cinder VolumeWeb front end to take user input

Well run this as a Swarm service, so we can do things like scale it, drain-nodes and move the containers uninterrupted.

All while persisting our Redis data

#

Create a Docker network first so the Swarm nodes have a layer to communicate on

eval $(docker-machine env swarm-1) docker network create demo-netBd45fad9911005ce2ff8e311a2738681d179589d8d06989a136e8020bc5a8155

#

Launch our services, start with the Redis service

eval $(docker-machine env swarm-1) docker service create --name redis --network demo-net -mount \ type=volume,src=counter-demo,dst=/data,volume-driver=cinder -p 6379:6379 redis

This will:Pull the Redis image if its not availableGet/Create the volume on the Cinder backendAttach the Volume to the Swarm nodePartition, Format and Mount the VolumeLink the volume to the Redis Containers /data directoryStart the Redis Container

#

You can see the attached volume on the Swarm Node.

ubuntu@swarm-3:~$ ls /dev/disk/by-path/ip-10.10.9.1:3260-iscsi-iqn.2010-01.com.solidfire:ayyb.uuid-59e99b3b-c7d6-45c2-924c-virtio-pci-0000:00:04.0virtio-pci-0000:00:04.0-part1

Same as we do for Nova Compute nodes nothing really different except we add the file system

#

Now, the web service/frontend

docker service create --name web --network demonet -p 80:80 \jgriffith/jgriffith webbase

This will:Connect to our Redis container (regardless of what Swarm node its on)Expose port 80 to all of the Swarm Nodes (access from any Swarm Node IP)Count input/clicks and store them in the Redis DB

#

Cant show you a live demo Next best thing. Movies!!!

Create Swarm Cluster and install the Driver on NodesCreate our Service using a Cinder VolumeDemo the Service and fail it over to another Node

#

#

#

#