DockerWhy and How for Development

Foundations of a Container

Like a VM

Own Process Space

Own Network Interface

Can run stuff as Root

Can Install Packages

Can Run Services

Can manipulate routing, iptables, etc.

Not VM

Uses the Host Kernel

Can’t Boot a different OS

Can’t have its own modules

Does not need init as PID 1

Does not need, or usually have, syslogd, cron, etc.

Processes are visible on the host machine

Why

Development Environments

Runtimes and dependencies are consistent with production

Build and test applications on platforms different than the one you use for development

Build and test for multiple environments

Run backend services

Build/Test Environments

Checkout multiple branches, run tests simultaneously

Increase parallelism of worker/test nodes

Isolate different tests running on a node

Provide multiple test and build environments

Optimize utilization level of CI nodes

Make environments available for local development match those used for testing and deployment

Production

Code runs with dependencies and needed libraries bundled

Operations teams can provide base images with consistent starting points

Create clusters of services

Scale service stacks horizontally with multiple nodes running multiple containers

How

Development of NodeJS Example

Nginx Container serves connections

/src/public mounted into nginx container to serve static files

Can run locally on development machine, or spin up the NodeJS container to execute tests or run the application in “production” environment

Redis container provides Redis connectivity

Mock PuppetDB, LDAP, Sensu, and OpenTSDB services run in additional containers

Locally running NodeJS combined with Webstorm provides great development experience

Supporting services and infrastructure provided by Docker Compose, effortless testing and integration

Build and Test with CI Tools

Unit tests are run in same environment as production.

Isolated resources allow simulation of various constraints and prevent CPU/Memory intense test from impacting other tests

Build artifacts can be written to host with mounted volumes

Logging and test results shipped via Syslog or other logging drivers

Network can be built up and isolated to ensure multiple stacks can run on a host but not interact

Failure scenarios and network segmentation can be tested also

Deploying OpenTSDB Example

10x Query Nodes in Docker Containers

Host runs Docker, HBase, OpenTSDB Write Node, HAProxy

Query node resources limited, high number of isolated query slots improves average query response time, prevents long queries from impacting general population

Docker containers specified by Puppet

Configuration written to host /etc/opentsdb/opentsdb.conf

Uses volume to place it into container at runtime

HAProxy balances incoming connections across container nodes

Data Processing Pipeline Example

Service 1 is a lightweight ingest container, parses lines into structured JSON

Service 2 is a Redis cache, to prevent incoming data from overwhelming backend services or outage on backend from impacting incoming data

Service 3 reads from Redis lists and processes incoming data. Handle more ingest customers with more

Docker hosts, handle larger streams per connection with larger hosts, t2.large vs

m4.16xlarge

Thanks!

Got Questions?

● http://www.slideshare.net/jpetazzo/anatomy-of-a-container-namespaces-cgroups-some-filesystem-magic-linuxcon

● https://github.com/veggiemonk/awesome-docker● http://www.slideshare.net/SteveBargelt/automated-build-system-with-docker-jenkins-a

nd-azure-intro