Container as a Service with Docker

Patrick Chanezon, Docker Inc.@chanezon

French

Polyglot

Platforms

Software Plumber

San Francisco

Developer Relations

@chanezon

1995 2015

“The future is already here — it's just not very evenly

distributed”William Gibson, Neuromancer

Docker’s mission is tobuild tools of mass

innovation

Internet (hardware layer)

Servers Desktops Phones Cars Houses Drones

Network equipmen

tPublic transit

TVsIndustri

al facilities

Scientific instrumen

tsFinancial system

Programmers

Internet (software layer)

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

Internet (hardware layer)

Servers Desktops Phones Cars Houses Drones

Network equipmen

tPublic transit

TVsIndustri

al facilities

Scientific instrumen

tsFinancial system

Programmers

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

App

a software layer to program the internet

Cloud Market

PublicHybridPrivate

IT Pros Devops DevelopersArchitects

Linux Container Ecosystem

flockerglusterfs

weavecalicomidokuracisconuage

Cloud

OS

Plugins

Orchestration

Devops

25

Mainframe

Client-Server

26

27

Web

28

Cloud - Devops

25

Mainframe

Devops• Cultural movement • Inspired by agile methods• People, Processes & Tools• Continuous delivery• Infrastructure as code• Cross silo collaboration• Small iterations• Feedback loop, measurement

Image from Patrick Deboishttp://www.slideshare.net/jedi4ever/devops-the-war-is-over-if-you-want-ithttp://www.slideshare.net/jedi4ever/devopsdays-downundervfinal

Devops: singing Kumbaya?

Low MTBIAMSH

MTBIAMSH (Mean Time Between Idea And Making Stuff Happen)

Devops + Agility == $$

Docker

Isolation using Linux kernel featuresnamespaces

pid mnt net uts ipc user

cgroups memory cpu blkio devices

Image layers

Docker Mission

Docker for developers

https://registry.hub.docker.com/_/java/

docker-compose: running multiple containers Run your stack with one command: docker-compose

up Describe your stack with one file: docker-compose.ymlweb: build: . command: python app.py ports: - "5000:5000" volumes: - .:/code links: - redis:redis

redis: image: redis

Docker for Mac and Windows private beta

https://beta.docker.com/

defaultDOCKER_HOST=tcp://xxx.xxx.xxx.xxx:2376DOCKER_MACHINE_NAME=defaultDOCKER_TLS_VERIFY=1DOCKER_CERT_PATH=$HOME/.docker/machine/machines/default

IP xxx.xxx.xxx.xxx

manages

Docker Toolbox

unset ${!DOCKER_*}

docker.local

symlink /var/tmp/docker.sock

manages

Docker for Mac

unset ${!DOCKER_*}

docker.local

symlink /var/tmp/docker.sock

manages

default

IP xxx.xxx.xxx.xxx

manages

DOCKER_HOST=tcp://xxx.xxx.xxx.xxx:2376DOCKER_MACHINE_NAME=defaultDOCKER_TLS_VERIFY=1DOCKER_CERT_PATH=$HOME/.docker/machine/machines/default

Docker for Mac and Toolbox

Linux X11 Apps on Docker for Mac

https://github.com/chanezon/docker-tips/x11

docker-machine

docker-machine create -d azure \—azure-subscription-id="c4f51be3-784c-xxx-7c50ad9e1b7c" \--azure-subscription-cert="/Users/pat/.ssh/docker-azure-cert.pem" \--azure-location="East US" \--azure-size=Small \--azure-username="pat" \pat-docker-machine-n

Kitematic

Docker Mission

Docker Hub

Docker Hub

Dev & QA ColleaguesDevelopers

QA

Build & Ship

Docker Trusted Registry

Docker Mission

SwarmScheduler

plugins

Engine

Volumes plugins

Network plugins

Service discovery plugins

Engine

Volumes plugins

Network plugins

Service discovery plugins

mesos

flockerglusterfs

weavecalico

consuletcdzookeeper

midokuraciscoazurenuagenetworks

Docker Plugins

Batteries included but removable

DockerCLI

DockerCLI

DockerCLI

Docker Engine

us-west us-east

DockerCLI

DockerCLI

Docker Swarm

Swarm

Swarm load balancing: interlock

https://github.com/ehazlett/interlock/tree/master/plugins/haproxy

Docker Datacenter

Docker Cloud

Open Standards

Plumbing

Notary“Let’s stop using curl|sh”

Trusted collections for any content

Transport-agnostic

Reliable updates, proof of origin, resistant to untrusted transport, survivable key compromise

Build on industry-leading standards and research

RunCThe universal container runtime

https://runc.io

containerdA daemon to control runC

built for performance and density

http://containerd.tools/

containerd

Docker 1.11

Docker & Microsoft

Docker & Microsoft• Build

• Docker Toolbox & Kitematic for Windows• Docker for Windows beta• Docker engine on Windows Server 2016 TP4• yo-docker to dockerize existing projects• Visual Studio Docker Tools

• Ship• VSTS extension for Docker beta

• Run• Azure Docker agent• ACS• Docker datacenter on Azure ARM template• Azure Container Service

Unikernels

Unikernel Systems + Docker

Unikernels

http://unikernel.org/

Unikernelsspecialised, single-address-space machine images constructed by using library

operating systems.• Smaller

• link only the parts of the OS lib you need• Faster

• fast boot• compiler can perform whole-system optimization

• More secure• reduced attack surface

Unikernels & Docker

http://unikernel.org/blog/2015/unikernels-meet-docker/https://github.com/Unikernel-Systems/DockerConEU2015-demo

Docker CaaS

The Docker mission

Build Ship Run

Anywhere

Distributed Applications

5

XaaS Pyramid

Platform As A Service

Infrastructure As A Service

SoftwareAs A Service

5

Goldilocks and the 3 XaaS

Just rightToo highToo lowIaaS PaaS CaaS

5

Goldilocks and the 3 XaaS

Platform As A Service

Infrastructure As A Service

SoftwareAs A Service

Too high

Too low

Just right

Container As A Service

5

Goldilocks and the 3 XaaS

Container As A Service

Infrastructure As A Service

SoftwareAs A Service

Docker Containers as a Service (CaaS)An IT managed and secure application content and infrastructure where developers can self service build and deploy applications

The Docker Journey: The Power of AND

Manage and secure at scale

Frictionless movement

Innovation at speed

+ +Agility Portability Control

Docker survey 2016 - Enabling the Software Supply Chain

Lessons learned: Avoid these pitfalls

12

3

Developers don’t adopt locked down systems

Existing “end to end” solutions break the Docker experience

Beware of lock-in and loss of portability

Let’s Play: Where’s Whaledo

in Google Container Engine?$ docker build -t gcr.io/${PROJECT_ID}/hello-node .$ gcloud docker push gcr.io/${PROJECT_ID}/hello-node$ gcloud container clusters create hello-world \ --num-nodes 1 \ --machine-type g1-small$ kubectl run hello-node --image=gcr.io/$

{PROJECT_ID}/hello-node --port=8080$ kubectl get services hello-node$ kubectl scale rc hello-node --replicas=3

https://cloud.google.com/container-engine/docs/tutorials/hello-node

in Kubernetes?

in EC2 Container Service?$ ecs-cli up --keypair id_rsa --capability-iam \—size 2 --instance-type t2.medium

create a compose file

$ ecs-cli compose --file hello-world.yml up$ ecs-cli ps$ ecs-cli compose --file hello-world.yml scale 2$ ecs-cli compose --file hello-world.yml service up

http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_CLI_tutorial.html

in Red Hat?$ ./openshift kube apply -c docker-registry-config.jsonOpenShift relies on the concept of Builds to turn your

application source into a runnable Docker image$ ./openshift kube create buildConfigs -c application-

buildconfig.json$ curl -s -A "GitHub-Hookshot/github" -H "Content-

Type:application/json" -H "X-Github-Event:push" -d @github-webhook-example.json http://localhost:8080/osapi/v1beta1/buildConfigHooks/build100/secret101/github$ ./openshift kube process -c application-template.json |

./openshift kube apply -c -

https://blog.openshift.com/openshift-v3-deep-dive-docker-kubernetes/

in Red Hat?

https://blog.openshift.com/openshift-v3-deep-dive-docker-kubernetes/

in Pivotal Cloud Foundry?$cf api --skip-ssl-validation api.bosh-lite.com$cf auth admin admin$cf create-org diego$cf target -o diego$cf create-space diego$cf target -s diego$cf push my-app --no-start$cf start my-app

https://github.com/cloudfoundry-incubator/diego-release

in Pivotal Cloud Foundry?

runC

Developers IT Operations

BUILDDeveloper Workflows

SHIPSecure Content & Collaboration

RUNDeploy, Manage, Scale

Docker CaaS Platform

Docker Containers as a Service platform

BUILDDeveloper Workflows

SHIPRegistry Services

RUNManagement

Docker Toolbox Docker Trusted Registry Docker Universal Control Plane

Docker Cloud

Docker Engine

Ecosystem Plugins and Integrations

Characteristics of a CaaS

Any Infrastructure

Any Operating System

Any Language

Any App Architecture

Any Application

Stage

Developers + IT Ops

The Power of AND

Open APIs Broadest Ecosystem Support

Docker accelerates modern app initiatives

Cloud

Microservices

80%Docker is central to

cloud strategy

State of App development Survey: Q1 - 2016

3 out 4 Top initiatives revolve around applications

44%Looking to adopt

DevOpsDevOps

Use Case: Decentralized CaaS for hybrid and multi cloud portability

Private datacenter for regulated apps

Central Portal

• Provision resources

• RBAC to VPC / datacenter

• Trusted Registry hosted application templates

Cloud for all other apps

VPC 1 VPC2

App 1 App 2 App

App 1 App 2 App

Cloud Portability

App Portability

Use Case: Centralized CaaS for transformation to DevOps and micro services

After

Authorization

App Registration

Session Management

Marketplace Integration

Logging

…more

Trusted Registry App Service App Service

App A App B

Auth

…more

App Reg

Marketplace

Logging

Auth

Session

…more

App Reg

Logging

Before

App TeamsApp BAuth

App Reg Marketplace

Logging

App Service

Universal Control Plane

App AAuth

App Reg Marketplace

Logging

App Service

App BAuth

App Reg Marketplace

Logging

App Service

App AAuth

App Reg Marketplace

Logging

App Service

Portability

Demos

• Docker Swarm, Compose and networking• docker 1.11• swarm 1.1.0• compose 1.6.0 with networking• Run/Debug with STS IDE in

a container

Spring Boot App using MongoDB

https://github.com/joshlong/spring-doge

https://github.com/chanezon/docker-tips/orchestration-networkinghttps://github.com/chanezon/spring-doge

Docker Universal Control Plane

https://github.com/chanezon/docker-tips/tree/master/azure-acs-ucp

Load balancing in UCP with Interlock

See "Reference Architecture: Service Discovery and Load Balancing with Docker Universal Control Plane (UCP)"

ucp-controller

Cloud LB

ucp-node-1 ucp-node-2

Cloud LB

etcd InterlockSwarm ucp-proxy nginx nginx

Configuration

Reconfigures

myapp:314

myapp:42

myapp:1968

myapp.comdocker run myapp

Load balancing in UCP with Interlock

https://github.com/chanezon/docker-tips/orchestration-networking

InterlockEvents

THANK YOU

Let’s Dockerize a Neo4J App

https://github.com/neo4j-examples/movies-java-spring-data-neo4j-4

• Service Discovery• https://github.com/gliderlabs/registrator• https://github.com/hashicorp/consul-template• https://github.com/ehazlett/interlock

• Persistent volumes with Swarm and Rex Ray on AWS• http://blog.emccode.com/2015/11/03/use-docker-swarm-with-

a-data-persistence-layer/• https://github.com/emccode/rexray

• Kubernetes on Swarm• https://github.com/docker/swarm-frontends

Orchestration projects

• IPVS, Andrey Sibiryov, http://www.slideshare.net/Docker/kernel-load-balancing-for-docker-containers-using-ipvs

• DNS Service Discovery for Docker Swarm, Ahmet Alp Balkan, http://www.slideshare.net/Docker/dns-service-discovery-for-docker-swarm

Load Balancing

• Spring Boot, MongoDB, compose, swarm, networking• https://github.com/joshlong/spring-doge• https://github.com/chanezon/docker-tips/orchestration-

networking• Java EE 7 / Angular App with Docker Swarm by @mgreau

Compose for build and deploy, Wildfly, Apache, Angular, Mysql, Redis, batch and API apps• https://github.com/mgreau/docker4dev-tennistour-app

• Java EE Docker & Kubernetes by @arun-gupta• https://github.com/javaee-samples/docker-java

Java Examples

• Docs https://docs.docker.com/engine/userguide/networking/dockernetworks/

• Create a Swarm cluster with networking https://github.com/chanezon/docker-tips/orchestration-networking

• Networking in compose https://github.com/docker/compose/blob/master/docs/networking.md

• Nathan Leclaire Seamless Docker Multihost Overlay Networking on DigitalOcean With Machine, Swarm, and Compose, RethinkDB http://nathanleclaire.com/blog/2015/11/17/seamless-docker-multihost-overlay-networking-on-digitalocean-with-machine-swarm-and-compose-ft.-rethinkdb/

Docker networking

• Using Ansible with Docker Machine to Bootstrap Host Nodes http://nathanleclaire.com/blog/2015/11/10/using-ansible-with-docker-machine-to-bootstrap-host-nodes/

• Seamless Docker Multihost Overlay Networking on DigitalOcean With Machine, Swarm, and Compose, RethinkDB http://nathanleclaire.com/blog/2015/11/17/seamless-docker-multihost-overlay-networking-on-digitalocean-with-machine-swarm-and-compose-ft.-rethinkdb/

Nathan’s tips