Post on 17-Jan-2017
Container as a Service with Docker
Patrick Chanezon, Docker Inc.@chanezon
French
Polyglot
Platforms
Software Plumber
San Francisco
Developer Relations
@chanezon
1995 2015
“The future is already here — it's just not very evenly
distributed”William Gibson, Neuromancer
Docker’s mission is tobuild tools of mass
innovation
Internet (hardware layer)
Servers Desktops Phones Cars Houses Drones
Network equipmen
tPublic transit
TVsIndustri
al facilities
Scientific instrumen
tsFinancial system
Programmers
Internet (software layer)
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
Internet (hardware layer)
Servers Desktops Phones Cars Houses Drones
Network equipmen
tPublic transit
TVsIndustri
al facilities
Scientific instrumen
tsFinancial system
Programmers
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
a software layer to program the internet
Cloud Market
PublicHybridPrivate
IT Pros Devops DevelopersArchitects
Linux Container Ecosystem
flockerglusterfs
weavecalicomidokuracisconuage
Cloud
OS
Plugins
Orchestration
Devops
25
Mainframe
Client-Server
26
27
Web
28
Cloud - Devops
25
Mainframe
Devops• Cultural movement • Inspired by agile methods• People, Processes & Tools• Continuous delivery• Infrastructure as code• Cross silo collaboration• Small iterations• Feedback loop, measurement
Image from Patrick Deboishttp://www.slideshare.net/jedi4ever/devops-the-war-is-over-if-you-want-ithttp://www.slideshare.net/jedi4ever/devopsdays-downundervfinal
Devops: singing Kumbaya?
Low MTBIAMSH
MTBIAMSH (Mean Time Between Idea And Making Stuff Happen)
Devops + Agility == $$
Docker
Isolation using Linux kernel featuresnamespaces
pid mnt net uts ipc user
cgroups memory cpu blkio devices
Image layers
Docker Mission
Docker for developers
https://registry.hub.docker.com/_/java/
docker-compose: running multiple containers Run your stack with one command: docker-compose
up Describe your stack with one file: docker-compose.ymlweb: build: . command: python app.py ports: - "5000:5000" volumes: - .:/code links: - redis:redis
redis: image: redis
Docker for Mac and Windows private beta
https://beta.docker.com/
defaultDOCKER_HOST=tcp://xxx.xxx.xxx.xxx:2376DOCKER_MACHINE_NAME=defaultDOCKER_TLS_VERIFY=1DOCKER_CERT_PATH=$HOME/.docker/machine/machines/default
IP xxx.xxx.xxx.xxx
manages
Docker Toolbox
unset ${!DOCKER_*}
docker.local
symlink /var/tmp/docker.sock
manages
Docker for Mac
unset ${!DOCKER_*}
docker.local
symlink /var/tmp/docker.sock
manages
default
IP xxx.xxx.xxx.xxx
manages
DOCKER_HOST=tcp://xxx.xxx.xxx.xxx:2376DOCKER_MACHINE_NAME=defaultDOCKER_TLS_VERIFY=1DOCKER_CERT_PATH=$HOME/.docker/machine/machines/default
Docker for Mac and Toolbox
Linux X11 Apps on Docker for Mac
https://github.com/chanezon/docker-tips/x11
docker-machine
docker-machine create -d azure \—azure-subscription-id="c4f51be3-784c-xxx-7c50ad9e1b7c" \--azure-subscription-cert="/Users/pat/.ssh/docker-azure-cert.pem" \--azure-location="East US" \--azure-size=Small \--azure-username="pat" \pat-docker-machine-n
Kitematic
Docker Mission
Docker Hub
Docker Hub
Dev & QA ColleaguesDevelopers
QA
Build & Ship
Docker Trusted Registry
Docker Mission
SwarmScheduler
plugins
Engine
Volumes plugins
Network plugins
Service discovery plugins
Engine
Volumes plugins
Network plugins
Service discovery plugins
mesos
flockerglusterfs
weavecalico
consuletcdzookeeper
midokuraciscoazurenuagenetworks
Docker Plugins
Batteries included but removable
DockerCLI
DockerCLI
DockerCLI
Docker Engine
us-west us-east
DockerCLI
DockerCLI
Docker Swarm
Swarm
Swarm load balancing: interlock
https://github.com/ehazlett/interlock/tree/master/plugins/haproxy
Docker Datacenter
Docker Cloud
Open Standards
Plumbing
Notary“Let’s stop using curl|sh”
Trusted collections for any content
Transport-agnostic
Reliable updates, proof of origin, resistant to untrusted transport, survivable key compromise
Build on industry-leading standards and research
containerdA daemon to control runC
built for performance and density
http://containerd.tools/
containerd
Docker 1.11
Docker & Microsoft
Docker & Microsoft• Build
• Docker Toolbox & Kitematic for Windows• Docker for Windows beta• Docker engine on Windows Server 2016 TP4• yo-docker to dockerize existing projects• Visual Studio Docker Tools
• Ship• VSTS extension for Docker beta
• Run• Azure Docker agent• ACS• Docker datacenter on Azure ARM template• Azure Container Service
Unikernels
Unikernel Systems + Docker
Unikernelsspecialised, single-address-space machine images constructed by using library
operating systems.• Smaller
• link only the parts of the OS lib you need• Faster
• fast boot• compiler can perform whole-system optimization
• More secure• reduced attack surface
Unikernels & Docker
http://unikernel.org/blog/2015/unikernels-meet-docker/https://github.com/Unikernel-Systems/DockerConEU2015-demo
Docker CaaS
The Docker mission
Build Ship Run
Anywhere
Distributed Applications
5
XaaS Pyramid
Platform As A Service
Infrastructure As A Service
SoftwareAs A Service
5
Goldilocks and the 3 XaaS
Just rightToo highToo lowIaaS PaaS CaaS
5
Goldilocks and the 3 XaaS
Platform As A Service
Infrastructure As A Service
SoftwareAs A Service
Too high
Too low
Just right
Container As A Service
5
Goldilocks and the 3 XaaS
Container As A Service
Infrastructure As A Service
SoftwareAs A Service
Docker Containers as a Service (CaaS)An IT managed and secure application content and infrastructure where developers can self service build and deploy applications
The Docker Journey: The Power of AND
Manage and secure at scale
Frictionless movement
Innovation at speed
+ +Agility Portability Control
Docker survey 2016 - Enabling the Software Supply Chain
Lessons learned: Avoid these pitfalls
12
3
Developers don’t adopt locked down systems
Existing “end to end” solutions break the Docker experience
Beware of lock-in and loss of portability
Let’s Play: Where’s Whaledo
in Google Container Engine?$ docker build -t gcr.io/${PROJECT_ID}/hello-node .$ gcloud docker push gcr.io/${PROJECT_ID}/hello-node$ gcloud container clusters create hello-world \ --num-nodes 1 \ --machine-type g1-small$ kubectl run hello-node --image=gcr.io/$
{PROJECT_ID}/hello-node --port=8080$ kubectl get services hello-node$ kubectl scale rc hello-node --replicas=3
https://cloud.google.com/container-engine/docs/tutorials/hello-node
in Kubernetes?
in EC2 Container Service?$ ecs-cli up --keypair id_rsa --capability-iam \—size 2 --instance-type t2.medium
create a compose file
$ ecs-cli compose --file hello-world.yml up$ ecs-cli ps$ ecs-cli compose --file hello-world.yml scale 2$ ecs-cli compose --file hello-world.yml service up
http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_CLI_tutorial.html
in Red Hat?$ ./openshift kube apply -c docker-registry-config.jsonOpenShift relies on the concept of Builds to turn your
application source into a runnable Docker image$ ./openshift kube create buildConfigs -c application-
buildconfig.json$ curl -s -A "GitHub-Hookshot/github" -H "Content-
Type:application/json" -H "X-Github-Event:push" -d @github-webhook-example.json http://localhost:8080/osapi/v1beta1/buildConfigHooks/build100/secret101/github$ ./openshift kube process -c application-template.json |
./openshift kube apply -c -
https://blog.openshift.com/openshift-v3-deep-dive-docker-kubernetes/
in Red Hat?
https://blog.openshift.com/openshift-v3-deep-dive-docker-kubernetes/
in Pivotal Cloud Foundry?$cf api --skip-ssl-validation api.bosh-lite.com$cf auth admin admin$cf create-org diego$cf target -o diego$cf create-space diego$cf target -s diego$cf push my-app --no-start$cf start my-app
https://github.com/cloudfoundry-incubator/diego-release
in Pivotal Cloud Foundry?
runC
Developers IT Operations
BUILDDeveloper Workflows
SHIPSecure Content & Collaboration
RUNDeploy, Manage, Scale
Docker CaaS Platform
Docker Containers as a Service platform
BUILDDeveloper Workflows
SHIPRegistry Services
RUNManagement
Docker Toolbox Docker Trusted Registry Docker Universal Control Plane
Docker Cloud
Docker Engine
Ecosystem Plugins and Integrations
Characteristics of a CaaS
Any Infrastructure
Any Operating System
Any Language
Any App Architecture
Any Application
Stage
Developers + IT Ops
The Power of AND
Open APIs Broadest Ecosystem Support
Docker accelerates modern app initiatives
Cloud
Microservices
80%Docker is central to
cloud strategy
State of App development Survey: Q1 - 2016
3 out 4 Top initiatives revolve around applications
44%Looking to adopt
DevOpsDevOps
Use Case: Decentralized CaaS for hybrid and multi cloud portability
Private datacenter for regulated apps
Central Portal
• Provision resources
• RBAC to VPC / datacenter
• Trusted Registry hosted application templates
Cloud for all other apps
VPC 1 VPC2
App 1 App 2 App
App 1 App 2 App
Cloud Portability
App Portability
Use Case: Centralized CaaS for transformation to DevOps and micro services
After
Authorization
App Registration
Session Management
Marketplace Integration
Logging
…more
Trusted Registry App Service App Service
App A App B
Auth
…more
App Reg
Marketplace
Logging
Auth
Session
…more
App Reg
Logging
Before
App TeamsApp BAuth
App Reg Marketplace
Logging
App Service
Universal Control Plane
App AAuth
App Reg Marketplace
Logging
App Service
App BAuth
App Reg Marketplace
Logging
App Service
App AAuth
App Reg Marketplace
Logging
App Service
Portability
Demos
• Docker Swarm, Compose and networking• docker 1.11• swarm 1.1.0• compose 1.6.0 with networking• Run/Debug with STS IDE in
a container
Spring Boot App using MongoDB
https://github.com/joshlong/spring-doge
https://github.com/chanezon/docker-tips/orchestration-networkinghttps://github.com/chanezon/spring-doge
Docker Universal Control Plane
https://github.com/chanezon/docker-tips/tree/master/azure-acs-ucp
Load balancing in UCP with Interlock
See "Reference Architecture: Service Discovery and Load Balancing with Docker Universal Control Plane (UCP)"
ucp-controller
Cloud LB
ucp-node-1 ucp-node-2
Cloud LB
etcd InterlockSwarm ucp-proxy nginx nginx
Configuration
Reconfigures
myapp:314
myapp:42
myapp:1968
myapp.comdocker run myapp
Load balancing in UCP with Interlock
https://github.com/chanezon/docker-tips/orchestration-networking
InterlockEvents
THANK YOU
Let’s Dockerize a Neo4J App
https://github.com/neo4j-examples/movies-java-spring-data-neo4j-4
• Service Discovery• https://github.com/gliderlabs/registrator• https://github.com/hashicorp/consul-template• https://github.com/ehazlett/interlock
• Persistent volumes with Swarm and Rex Ray on AWS• http://blog.emccode.com/2015/11/03/use-docker-swarm-with-
a-data-persistence-layer/• https://github.com/emccode/rexray
• Kubernetes on Swarm• https://github.com/docker/swarm-frontends
Orchestration projects
• IPVS, Andrey Sibiryov, http://www.slideshare.net/Docker/kernel-load-balancing-for-docker-containers-using-ipvs
• DNS Service Discovery for Docker Swarm, Ahmet Alp Balkan, http://www.slideshare.net/Docker/dns-service-discovery-for-docker-swarm
Load Balancing
• Spring Boot, MongoDB, compose, swarm, networking• https://github.com/joshlong/spring-doge• https://github.com/chanezon/docker-tips/orchestration-
networking• Java EE 7 / Angular App with Docker Swarm by @mgreau
Compose for build and deploy, Wildfly, Apache, Angular, Mysql, Redis, batch and API apps• https://github.com/mgreau/docker4dev-tennistour-app
• Java EE Docker & Kubernetes by @arun-gupta• https://github.com/javaee-samples/docker-java
Java Examples
• Docs https://docs.docker.com/engine/userguide/networking/dockernetworks/
• Create a Swarm cluster with networking https://github.com/chanezon/docker-tips/orchestration-networking
• Networking in compose https://github.com/docker/compose/blob/master/docs/networking.md
• Nathan Leclaire Seamless Docker Multihost Overlay Networking on DigitalOcean With Machine, Swarm, and Compose, RethinkDB http://nathanleclaire.com/blog/2015/11/17/seamless-docker-multihost-overlay-networking-on-digitalocean-with-machine-swarm-and-compose-ft.-rethinkdb/
Docker networking
• Using Ansible with Docker Machine to Bootstrap Host Nodes http://nathanleclaire.com/blog/2015/11/10/using-ansible-with-docker-machine-to-bootstrap-host-nodes/
• Seamless Docker Multihost Overlay Networking on DigitalOcean With Machine, Swarm, and Compose, RethinkDB http://nathanleclaire.com/blog/2015/11/17/seamless-docker-multihost-overlay-networking-on-digitalocean-with-machine-swarm-and-compose-ft.-rethinkdb/
Nathan’s tips