Compare Docker Deployment Options in Public Cloud

Presenter Name: Sreenivas Makam

Presented at: Container conference, Bangalore

Presentation Date: April 7, 2017

About me

• Senior Engineering Manager at Cisco Systems Data Center group

• Author of “Mastering CoreOS” https://www.packtpub.com/networking-and-servers/mastering-coreos/ )

• Docker Captain(https://www.docker.com/community/docker-captains )

• Blog: https://sreeninet.wordpress.com/

• Code hacks: https://github.com/smakam

• LinkedIn: https://in.linkedin.com/in/sreenivasmakam

• Twitter: @srmakam

Agenda

• Deployment options – Overview

• Methodology used to compare

• Comparison of Deployment options

• Pick the right option for your use case

Considerations for Container solution

Which Orchestration solution to choose?

Should it be deployed in VM or bare-metal?

How to run stateful applications?How to achieve Service discovery and DNS?

Should it be deployed on premise or public cloud?

Is central logging required?

How to monitor and troubleshoot container platform?

Is the solution secure?

How to inject secrets and dynamic configuration?

Docker deployment in public cloud

Docker Machine(for AWS, Azure, GCE)

Cloud provider’s CaaS(AWS ECS, Google GKE,

Microsoft ACS)

Docker’s custom cloud(Docker for AWS, Azure,

GCE)

Docker’s CaaS (Docker cloud, Docker datacenter)

Docker deployment

Docker Machine

• Tool from Docker to create and manage Docker nodes.

• Supported for majority of cloud providers including AWS, Azure and Google cloud.

• Creates Docker node on the cloud with 1 command.

• Latest Docker version is pre-installed in the node.

• Both old and new Swarm modes can be used to create clusters.

• Provides automatic secure access to Docker node.

• Minimal integration to native cloud provider features.

• Targeted as a developer tool.

Docker custom cloud

• This is Docker’s solution to get Containers integrate well with the cloud infrastructure.

• Docker for AWS and Azure is available for general use. Docker for GCE is available as beta currently.

• Swarm mode cluster gets automatically created using a cloud provider template and with few simple inputs from the user.

• Solution integrated with Cloud networking, storage, logging, security group, load balancer.

• Targeted for hybrid cloud or multi-cloud deployments.

Docker CaaS• Container service offered by Docker.

• Docker Datacenter(DDC) - Docker’s enterprise grade container platform

• Docker cloud - Hosted service from Docker to manage Containers

• DDC can be run on-premise or in any of the major public cloud. Official support is present now for AWS and Azure. GCE would be added soon.

• UCP and DTR are main components of DDC and they can be deployed in a highly available manner.

• DDC provides enterprise grade features like high availability, RBAC and LDAP integration.

• DDC and Docker cloud provides nice user interface for management and is also compatible with regular Docker API.

• Docker cloud is targeted as a simple hosted solution for relatively small deployments.

• DDC is targeted as a complete Container platform since it includes all associated services like service discovery, logging, networking, storage with a decent cloud integration.

Docker datacenter - Components

https://www.docker.com/enterprise-edition

Docker datacenter for AWS - Architecture

https://docs.docker.com/datacenter/install/aws/

Docker datacenter for Azure - Architecture

https://blog.docker.com/2016/06/docker-datacenter-aws-azure-cloud/

Cloud provider CaaS

• Container service offered by Cloud providers.• ECS – From Amazon(Docker with Amazon’s proprietary orchestrator)• GKE – From Google(Docker with Kubernetes orchestrator)• ACS – From Microsoft(Docker with either Swarm, Kubernetes, Mesos)

• Provides very tight cloud integration.

• In some cases, there seems to be a lag with latest Docker version and features available with this option.

• Targeted for folks who want to manage VMs and Containers together. This solution suits companies which have a big cloud presence and already using public cloud.

Amazon ECS - Architecture

Uses proprietary orchestrator currently. There is a plan to make orchestrator as pluggable module in the future.

http://www.allthingsdistributed.com/2015/07/under-the-hood-of-the-amazon-ec2-container-service.html http://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html

Azure Container service- Architecture

• Supports major orchestrators like Docker Swarm, Kubernetes, Mesos.• ACS only takes care of installing and deploying the cluster. Management of

containers and services is done by orchestrator itself.

https://docs.microsoft.com/en-in/azure/container-service/container-service-intro

GKE Architecture

Kubernetes Architecture GKE Architecture – Built on Docker and Kubernetes

• GKE makes Kubernetes easier to manage and integrates well with Google compute’s other cloud services.

https://www.cloudbees.com/blog/demand-jenkins-slaves-kubernetes-and-google-container-enginehttps://www.slideshare.net/wattsteve/kubernetes-48013640

Methodology used to compare

• Built multi-node cluster in all cases. Tried to use Swarm mode in most of the cases where it is supported. In some cases, used custom orchestrator, old Swarm mode or Kubernetes where Swarm mode is not supported.

• Deployed multi-container voting application using Docker containers in the multi-node cluster.

• Accessed the externally exposed services using load balancer where it is available.

• Tried scaling up/down services as well as scaling up/down the cluster.

• Checked features like logging capability, service discovery, high availability, networking and storage.

• Evaluated ease of management, upgradability and the cloud integration.

• Details are captured here(https://github.com/smakam/dockerdeploy)

Multi-Container Voting app

https://github.com/docker/example-voting-app

Comparison parameter - Orchestration

• Both old and new Swarm modes can be used.Docker

Machine

• Swarm modeDocker’s

custom cloud• ECS – Amazon’s proprietary scheduler• GKE – Kubernetes• ACS – Supports Docker Swarm, Kubernetes,

Mesos

Cloud provider’s CaaS

• DDC – Swarm mode• Docker cloud – proprietary orchestration and

Swarm mode, Swarm mode available as beta

Docker’s CaaS

Comparison parameter - Management

• Docker-machine interface for managing cluster. Container management using Docker tools. Docker Machine

• Cluster creation using cloud provider template, Cluster management using Cloud provider tools, Container management using Docker tools.

Docker’s custom cloud

• ECS – ECS GUI and CLI• GKE – Cluster management using Google cloud,

Kubernetes dashboard to manage Containers• ACS – Cluster management using Azure cloud,

Container management using Docker tools.

Cloud provider’s CaaS

• DDC – UCP and DTR has GUI to manage, Supports Docker API.

• Docker cloud – GUI to manage. Supports Docker API.Docker’s CaaS

Comparison parameter – Networking & Storage

• Libnetwork with overlay for networking. Docker volume plugin supported by cloud provider can be used.

Docker Machine

• Libnetwork with overlay for networking. Cloudstor volume plugin is supported currently.

Docker’s custom cloud

• ECS – Overlay network not supported. Docker volume is supported with limited drivers

• GKE – Kubernetes based networking. Storage through Kubernetes persistent disk and google cloud storage

• ACS – Libnetwork with overlay for networking. Docker volume driver using Azure file storage is supported.

Cloud provider’s CaaS

• Libnetwork with overlay for networking. Docker volume plugin supported by cloud provider can be usedDocker’s CaaS

Comparison parameter – Registry

•Docker hub is default. Can be used with any Docker registry.

Docker Machine

•Can be used with any Docker registry.

Docker’s custom cloud

•ECS –Docker registry and Amazon’s ECR•GKE – Docker registry and Google’s container registry•ACS – Docker registry and Microsoft’s ACR

Cloud provider’s CaaS

•DDC – DTR•Docker cloud – Docker cloud registry and any other Docker registry

Docker’s CaaS

Comparison parameter – Cloud integration

•Minimal cloud integration.Docker Machine

•Integrated with cloud networking, firewall, load balancer, logging. This can be improved further.

Docker’s custom cloud

•Provides native integration of Containers to other cloud offerings from provider.

Cloud provider’s CaaS

•DDC – Good integration with cloud services.•Docker cloud – Swarm mode has minimal integration now. Non-swarm mode has better integration.

Docker’s CaaS

Comparison parameter – Application file format

• Compose formatDocker

Machine

• Compose formatDocker’s

custom cloud

•ECS – Proprietary task definition format, limited compose support•GKE – Kubernetes task definition format•ACS – Compose can be used with Docker Swarm

Cloud provider’s CaaS

•DDC – Compose format•Docker cloud – Compose format, supported stackfile format earlier which is similar to compose

Docker’s CaaS

Comparison parameter – Docker version & upgrade

•Uses CE latest version 17.03. No easy way to use older versions. •Docker upgrade has to be done manually.

Docker Machine

•Uses CE latest version 17.03. No easy way to use older versions. •Rolling upgrade is supported.

Docker’s custom cloud

• ECS – Docker version 1.12.6. No flexibility with Docker version. Container agent upgrade will also upgrade Docker version. Upgrade is manual.

• GKE – Docker 1.11.2, Kubernetes 1.5.4. Rolling upgrade is supported.

• ACS – CE version 17.03. No option to upgrade the cluster, suggestion is to create new cluster.

Cloud provider’s CaaS

• DDC – Uses EE 17.03 version. Need to upgrade UCP, DTR and Docker engine separately. No rolling upgrade yet.

• Docker cloud – Uses CE latest version 17.03. Provides integrated upgrade option.

Docker’s CaaS

Comparison parameter – Enterprise features

•Enterprise features has to be manually setup.Docker Machine

•Enterprise features has to be manually setup

Docker’s custom cloud

•Integrated with cloud provider’s enterprise features like user management, high availability.

Cloud provider’s CaaS

•DDC – Highly available. Provides RBAC and LDAP integration.•Docker cloud – Supports organizations and teams. Docker’s CaaS

Comparison parameter - Cost

•Cost is for cloud resource usage, no separate cost for Containers

Docker Machine

•Cost is for cloud resource usage, no separate cost for Containers

Docker’s custom cloud

•AWS, ACS - Cost is for cloud resource usage, no separate cost for Containers•GKE – Chargeable after 5 Container nodes.

Cloud provider’s CaaS

•DDC – Separate cost for Container nodes.•Docker cloud – Separate charge for private repositories

Docker’s CaaS

How do I pick the right option for me?Production use case

VM and Container

usedDocker CaaS

Enterprise grade DDC

Docker CloudHybrid/Multi cloud

Cloud provider CaaS

Kubernetes GKEAzure

ECSAzure

Docker customcloud

Docker Machine

yes

yes

yesyes

yesno

no

no

no

no

Caution:This flowchart oversimplifies the problem. There are more considerations than this.

References• https://github.com/smakam/dockerdeploy

• https://sreeninet.wordpress.com/2016/09/03/docker-for-aws-deployment-options/

• https://docs.docker.com/machine/drivers/aws/

• https://docs.docker.com/machine/drivers/azure/

• https://docs.docker.com/machine/drivers/gce/

• http://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html

• https://docs.microsoft.com/en-in/azure/container-service/container-service-intro

• https://cloud.google.com/container-engine/

• https://docs.docker.com/docker-for-aws/

• https://docs.docker.com/docker-for-azure/

• https://docs.docker.com/docker-for-aws/

• https://beta.docker.com/docs/gcp/

• https://docs.docker.com/docker-cloud/

• https://www.docker.com/enterprise-edition

Thank you!

Any questions?