The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)
-
Upload
docker-inc -
Category
Technology
-
view
7.543 -
download
1
description
Transcript of The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)
![Page 1: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/1.jpg)
doing the old thing the new wayby @rafecolton
![Page 2: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/2.jpg)
brief prologue
“rafe” (rafecolton on the internets)
software engineer, platform @ modcloth
using docker in prod since v0.7.0
*todo: explain “doing the old thing the new way”
![Page 3: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/3.jpg)
obligatory slide with a bunch of logos
![Page 4: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/4.jpg)
the stack that was
![Page 5: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/5.jpg)
the stack that was
![Page 6: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/6.jpg)
the stack that was
![Page 7: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/7.jpg)
the stack that would be
![Page 8: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/8.jpg)
motivations• simplify application architecture
• support a variety of application languages
• make provisioning and deployment more accessible
![Page 9: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/9.jpg)
motivations• simplify application architecture
• support a variety of application languages
• make provisioning and deployment more accessible
goals• push-button provisioning and deployment
• consolidated, pluggable platform
• move to linux
![Page 10: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/10.jpg)
motivations• simplify application architecture
• support a variety of application languages
• make provisioning and deployment more accessible
goals• push-button provisioning and deployment
• consolidated, pluggable platform
• move to linux
bonus points• chatops
• actual button for provisioning and deployment
![Page 11: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/11.jpg)
motivations
goals
bonus points• chatops
• actual button for provisioning and deployment
does docker
facilitate such a
solution?
• simplify application architecture
• support a variety of application languages
• make provisioning and deployment more accessible
• push-button provisioning and deployment
• consolidated, pluggable platform
• move to linux
![Page 12: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/12.jpg)
the stack that would be
![Page 13: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/13.jpg)
Clear and Simple Statement.
case study: modcloth.com/style-gallery
![Page 16: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/16.jpg)
case study: modcloth.com/style-gallery
smartos
nginx ruby
rails
webserver
![Page 17: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/17.jpg)
case study: modcloth.com/style-gallery
smartos
nginx ruby
rails
smartos
cron ruby
rails
sidekiq workerswebserver
![Page 18: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/18.jpg)
case study: modcloth.com/style-gallery
smartos
nginx ruby
rails
webserver
smartos
cron ruby
rails
sidekiq workers
how complex could
it be?
![Page 19: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/19.jpg)
case study: modcloth.com/style-gallery
nginxruby
rails
webserver
ubuntu
docker
![Page 20: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/20.jpg)
case study: modcloth.com/style-gallery
nginxruby
rails
webserver
ubuntu
docker
cron
![Page 21: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/21.jpg)
case study: modcloth.com/style-gallery
nginxruby
rails
webserver
ubuntu
docker
cron
supervisord
![Page 22: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/22.jpg)
case study: modcloth.com/style-gallery
nginxruby
rails
webserver
ubuntu
docker
cron
supervisord
rails
sidekiq
workers
![Page 23: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/23.jpg)
case study: modcloth.com/style-gallery
nginxruby
rails
webserver
ubuntu
docker
cron
supervisord
rails
sidekiq
workers
nodejs rsyslogd
nad
sshd
![Page 24: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/24.jpg)
• overall complexity
• maintainability
• image consistency
• container reliability
• log aggregation
• monitoring
challenges
![Page 25: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/25.jpg)
• overall complexity
• maintainability
• image consistency
• container reliability
• log aggregation
• monitoring
challenges
• don’t do the new thing the old way
• consider division of responsibility
lessons
![Page 26: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/26.jpg)
case study: modcloth.com/style-gallery
webserver
ruby
rails
ubuntu
docker cron
nginx
sidekiq workers
ruby
rails
nodejs rsyslogd
nad
![Page 27: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/27.jpg)
the stack
![Page 28: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/28.jpg)
observations
docker is an excellent packaging and distribution system
![Page 29: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/29.jpg)
observations
docker is an excellent packaging and distribution system
containers are the canonical building block for a continuous delivery pipeline
![Page 30: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/30.jpg)
• docker/fig
• deis/deis
• flynn/flynn
• coreos/fleet
• ansible/ansible
• opscode/chef
• progrium/dokku
• newrelic/centurion
begin github search…projects for orchestrating containers:
![Page 31: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/31.jpg)
• docker/fig
• deis/deis
• flynn/flynn
• coreos/fleet
• ansible/ansible
• opscode/chef
• progrium/dokku
• newrelic/centurion
• mesosphere/marathon
• airbnb/chronos
• GoogleCloudPlatform/kubernetes
• openshift/geard
• VoltFramework/volt
projects for orchestrating containers:
begin github search…
![Page 32: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/32.jpg)
• docker/fig
• deis/deis
• flynn/flynn
• coreos/fleet
• ansible/ansible
• opscode/chef
• progrium/dokku
• newrelic/centurion
• mesosphere/marathon
• airbnb/chronos
• GoogleCloudPlatform/kubernetes
• openshift/geard
• VoltFramework/volt
• octohost/octohost
• makeusabrew/decking
• signalfuse/maestro-ng
• shipyard/shipyard
• DevTable/gantryd
• mcuadros/dockership
• longshoreman/longshoreman
• marmelab/gaudi
• etc.
begin github search…projects for orchestrating containers:
![Page 33: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/33.jpg)
• rafecolton/docker-builder
• mitchellh/packer
• swipely/dockly
• ???
begin github search…projects for building containers:
![Page 34: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/34.jpg)
observation
everybody is building containers differently.
![Page 35: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/35.jpg)
observation
everybody is building containers differently.
how do we build production-ready containers?
![Page 36: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/36.jpg)
writing a good Dockerfilelesson 0: getting started
![Page 37: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/37.jpg)
writing a good Dockerfilelesson 0: getting started use a docker hub base
![Page 38: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/38.jpg)
writing a good Dockerfilelesson 0: getting started
set your env
![Page 39: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/39.jpg)
writing a good Dockerfilelesson 1: order matters
deps before bundling
![Page 40: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/40.jpg)
writing a good Dockerfilelesson 1: order matters
ADD only Gemfile* first
![Page 41: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/41.jpg)
writing a good Dockerfilelesson 1: order matters
`ADD .` as late as possible
![Page 42: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/42.jpg)
writing a good Dockerfilelesson 2: optimize for size, repeatability
combine RUN commands
whenever possible
![Page 43: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/43.jpg)
writing a good Dockerfilelesson 2: optimize for size, repeatability
RUN dependent
steps together
![Page 44: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/44.jpg)
writing a good Dockerfilelesson 3: use a standard entrypoint
use a *simple*
entrypoint script
![Page 45: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/45.jpg)
writing a good Dockerfilelesson 3: use a standard entrypoint
operate on docker-specific
environment variables
![Page 46: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/46.jpg)
writing a good Dockerfilelesson 3: use a standard entrypoint
wrap verbose
CMD options
![Page 47: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/47.jpg)
writing a good Dockerfilelesson 3: use a standard entrypoint
exec "$@" # give yourself a shell
![Page 48: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/48.jpg)
the image development lifecycle
build
tag
push*type things*
![Page 49: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/49.jpg)
the image development lifecycle
pushbuild
tag
*type things*
> docker build -t myapp:latest .
> export latest="$(docker images | grep myapp:latest | head -n 1 | awk '{print $3}’)"
> docker tag $latest "$(git rev-parse -q HEAD)" # sha
> docker tag $latest "$(git describe --always --dirty --tags)" # tag
> docker tag $latest "$(git rev-parse -q --abbrev-ref HEAD)" # branch
> for image in $(docker images | grep myapp | awk '{print $1 ":" $2}' | head -n 4) ; \
do docker push $image ; done
![Page 50: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/50.jpg)
the image development lifecycle: docker-builder
push
build
tag
*type things*
> docker-builder build .
![Page 51: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/51.jpg)
teh pipeline
docker build
server*
appapp
app
app
*https://github.com/rafecolton/docker-builder
![Page 52: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/52.jpg)
conclusion
building containers is like writing ruby code:
![Page 53: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/53.jpg)
conclusion
building containers is like writing ruby code:
it’s easy to do
it’s hard to do correctly
![Page 54: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/54.jpg)
so what did we learn? (or, through what did you sleep?)
stuff:
• the complexity will come naturally (so don’t force it)
• be intentional about your Dockerfile
![Page 55: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/55.jpg)
so what did we learn? (or, through what did you sleep?)
stuff:
• the complexity will come naturally (so don’t force it)
• be intentional about your Dockerfile
• docker is an excellent packaging and distribution system
• containers are the canonical building blocks
![Page 56: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/56.jpg)
so what did we learn? (or, through what did you sleep?)
stuff:
• the complexity will come naturally (so don’t force it)
• be intentional about your Dockerfile
• docker is an excellent packaging and distribution system
• containers are the canonical building blocks
• consider division of responsibility between the host and the container
• don’t do the new thing the old way (do the old thing the new way!)
![Page 57: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/57.jpg)
brief epilogue
goal: move to linux
=> all apps (less one) now employing docker/ansible/linux
![Page 58: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/58.jpg)
brief epilogue
goal: move to linux
=> all apps (less one) now employing docker/ansible/linux
goal: consolidated, pluggable platform
=> shared monitoring, log aggregation, & load balancing services
![Page 59: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/59.jpg)
brief epilogue
goal: move to linux
=> all apps (less one) now employing docker/ansible/linux
goal: consolidated, pluggable platform
=> shared monitoring, log aggregation, & load balancing services
goal: push-button provisioning and deployment
=> it works, minimal magic… and it’s well documented
![Page 60: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/60.jpg)
brief epilogue
goal: move to linux
=> all apps (less one) now employing docker/ansible/linux
goal: consolidated, pluggable platform
=> shared monitoring, log aggregation, & load balancing services
goal: push-button provisioning and deployment
=> it works, minimal magic… and it’s well documented
could easily be maintained by only two people… dun dun dun
![Page 61: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/61.jpg)
brief epilogue
I’m job hunting…
![Page 62: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/62.jpg)
brief epilogue
I’m job hunting…
…and I haven’t shot anything yet.
so if you’re hiring, come talk to me after the show.
twitter: @rafecolton
github: rafecolton
rafecolton.com
![Page 63: The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCloth)](https://reader033.fdocuments.in/reader033/viewer/2022042715/559b60d41a28ab1d5f8b469a/html5/thumbnails/63.jpg)
thank you