Nebulaworks Docker Overview 09-22-2015

© 2015 Nebulaworks, Inc., All rights reserved.

DOCKER: THE ANYWHERE FRAMEWORK Chris Ciborowski September, 22 2015


A BIT ABOUT @CHRISCIBOROWSKI §  Founder & Managing Partner at Nebulaworks ̶  Technology & Engineering

§  Been working with UNIX my entire career ̶  Solaris and Linux, automation ̶  Open source supporter

§  Large-scale enterprise challenges are my thing ̶  DevOps, scale, performance

2


A BIT ABOUT NEBULAWORKS §  We are a new breed of SI ̶  Delivering your code at Speed, Frequency, and Scale ̶  Application Logistics

§  Application Delivery §  Process Optimization §  Innovation Support

§  Specializing In: ̶  Containerized Application Delivery (Docker, Kubernetes, Mesos) ̶  DevOps Enablement (CI/CD, workflows)

§  Docker Authorized Consulting & Training Partner 3


AGENDA §  Today’s challenges, how does Docker help? §  Docker, the platform

§  DevOps workflows & challenges §  Containers in production

§  Extended technology ecosystem

4


TODAY’S CHALLENGES §  Time to provision (TTP) apps §  Many languages and dependencies

§  Monolithic stacks §  Managing software artifacts

§  Low levels of standardization

5


How Does Docker Help?

6


DEPLOY ANYTHING §  Web apps §  Backends

§  SQL, NoSQL §  Big data

§  Message queues

§  … and more

7


DEPLOY EVERYWHERE §  Linux servers §  VMs or bare metal

§  Any distro §  Public clouds

8


DEPLOY RELIABLY & CONSISTENTLY §  If it works locally, it will work on the server §  With exactly the same behavior

§  Regardless of versions §  Regardless of distros

§  Regardless of dependencies

9


HOW? §  Portable artifacts (images) §  Platform agnostic

§  Consistent environments §  Immutable infrastructure

§  No HCL J

10


DOCKER, THE PLATFORM §  Docker Images §  Docker Engine

§  Docker Hub/Trusted Registry §  Docker Swarm

§  Docker Machine

§  Docker Compose §  Docker Toolbox

11


HIGH LEVEL VIEW: IT IS A LIGHTWEIGHT VM §  Own process space §  Own network interface

§  Can run stuff as root §  Can have its own /sbin/init (different from the host)

Call it a “machine container”

12


LOW LEVEL VIEW: IT IS CHROOT ON STEROIDS §  Can also not have its own /sbin/init §  Container = isolated process(es)

§  Share kernel with host §  No device emulation (neither HVM nor PV)

Call it an “application container”

13


COMPARING VM TO CONTAINER

14


Different namespaces, different “views” of the Kernel

15

HOW DOES IT WORK? NAMESPACE ISOLATION

Namespace Function

Mount (mnt) Mount points

UTS Hostname

IPC Interprocess communication

PID Processes in different PID namespaces can have same PID

Network (net) Network devices, IP addresses, routing tables, iptables entries

User Root privileges for operations inside a user namespace but unprivileged outside the namespace. Not all FS are user namespace aware


Virtual groupings, limits, priority, accounting

16

§  memory §  cpu

§  blkio §  devices

HOW DOES IT WORK? CONTROL GROUPS


Let’s Talk Workflows

17


NEW WAY TO DEVELOP APPS

18


DEVOPS WORKFLOW

19


DOCKER DEVELOPER EXAMPLE

20


WORKFLOW SUMMARY 1/2 §  Work in dev environment (local machine or container) §  Package dependencies with app code

§  Other services (databases etc.) in containers (and behave just like the real thing!)

§  Whenever you want to test « for real »:

§  Build in seconds §  Run instantly

21


WORKFLOW SUMMARY 2/2 §  Satisfied with your local build? ̶  Push it to a registry (public or private) ̶  Run it (automatically!) in CI/CD ̶  Run it in production ̶  Canary and/or blue/green deployments ̶  Happiness!

§  Something goes wrong? Rollback painlessly!

22


ADDRESSING DEVOPS CHALLENGES §  Picking the right model (application centric, workflow centric) §  Escalation

§  Authoritative repos §  Configuration and secrets

23


What About Production?

24


NOT QUITE AS EASY AS DEV §  Understand current to target environment deltas §  Stack correlation: Logging and monitoring

§  Knowledge of distributed platforms and tooling §  DevOps workflow centric model FTW ̶  If not, TechOps with strong dev experience

25


EXAMPLE CONTAINER WORKFLOW

26 COMPANY: GLIDEWELL LABORATORIES

MANAGER: ROBERT EDWARDS

PROJECT: DOCKER SCALABLE INFRASTRUCTURE

SUBJECT: CONTAINER WORKFLOW

DRAWING NAME: GW-Scalableinf_containerworkflow-v1

FILENAME: GW-Scalableinf_containerworkflow-v1

AUTHOR: CHRIS CIBOROWSKI

AM: CHRIS CIBOROWSKI

DATE: 09/01/2015

Consul Consul-template

/etc/consul-template/haproxy.ctmpl /etc/haproxy/haproxy.cfg

Docker Engine CS

Docker Swarm ManagerDocker Trusted Registry

HAProxy

Registrator

$ docker push $ docker run

TLS Secured TCP:443 TLS Secured TCP:2376

Docker UNIX Socket TCP:8500

TCP:8500

Client Operations

Docker Host Operations


MANY ITEMS TO CONSIDER §  Choosing a cluster/scheduler §  How do you handle networking?

§  Providing secrets, securely §  Persistent storage

27


EXTENDED TECHNOLOGY ECOSYSTEM

28


We’re available to help with your projects

29

§  [email protected] §  949-584-7589

§  www.nebulaworks.com §  @nebulaworks

THANKS FOR LISTENING!


DOCKER ENGINE §  Open Source engine to commoditize LXC §  Uses copy-on-write for quick provisioning

§  Written in Go, runs as a daemon, comes with a CLI §  Everything exposed through a REST API

§  Allows to build images in standard, reproducible way

§  Allows to share images through registries §  Defines standard format for containers (stack of layers; 1 layer =

tarball+metadata) 31


Collection of services to make Docker more useful

32

§  Public registry (push/pull your images for free) §  Private registry (push/pull secret images for $)

§  Automated builds (link github/bitbucket repo; trigger build on commit)

DOCKER HUB / DOCKER TRUSTED REGISTRY


DOCKER SWARM §  Native clustering for Docker. ̶  It turns a pool of Docker hosts into a single, virtual host

§  Serves the standard Docker API ̶  Any tool which communicates with a Docker daemon can be used

transparently: Dokku, Compose, Krane, Flynn, Deis, DockerUI, Shipyard, Drone, Jenkins... and, of course, the Docker client itself

§  Follows "batteries included but removable" principle ̶  Ships with a simple scheduling backend, API will develop to enable

pluggable backends like Mesos

33


DOCKER SWARM $ docker-machine create –driver virtualbox dev

$ docker-machine create -d virtualbox --swarm --swarm-master --swarm-discovery token://14699b753350745b3e59fa985925d193 swarm-master

$ docker-machine create -d virtualbox --swarm --swarm-discovery token://14699b753350745b3e59fa985925d193 swarm-node-00

$ $(docker-machine env --swarm swarm-master)

Demo!

https://asciinema.org/a/17908 34


DOCKER MACHINE §  Easily create Docker hosts on your computer, on cloud providers

and inside your own data center

§  It creates servers, installs Docker on them, then configures the Docker client to talk to them

§  Once your Docker host has been created, it then has a number of commands for managing them: ̶  Starting, stopping, restarting ̶  Upgrading Docker ̶  Configuring the Docker client to talk to your host

35


DOCKER MACHINE $ docker-machine create –driver virtualbox test

$ docker-machine ls

$ $(docker-machine env test)

Demo!

https://asciinema.org/a/17907

36


DOCKER COMPOSE §  Describe your stack with one file: docker-compose.yml§  Run your stack with one command: docker-compose up

§  Example: run an app with key/value datastore ̶  Python app ̶  Redis

37


DOCKER COMPOSE $ docker-compose up

J

Demo!

https://asciinema.org/a/17909

38


With Docker I can:

39

§  Put my software in containers §  Run those containers anywhere

§  Create workflows to automatically build containers §  Easily setup Docker hosts with Machine

§  Use Compose to effortlessly start stacks of containers

§  Run containers on multiple hosts

IN SUMMARY!

Nebulaworks Docker Overview 09-22-2015

Technology

Transcript of Nebulaworks Docker Overview 09-22-2015