Safe deployments with Blue-Green and Spinnaker
-
Upload
mihnea-dobrescu-balaur -
Category
Software
-
view
176 -
download
1
Transcript of Safe deployments with Blue-Green and Spinnaker
![Page 1: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/1.jpg)
Safe deployments with Blue-Green and Spinnaker
Platform Bucharest 23.02.2017
![Page 3: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/3.jpg)
Deployment
• getting product out to clients
• agility matters - first to market
• speed vs risk
![Page 4: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/4.jpg)
Rolling deployment
• classic/easiest way of doing things
• on every server: git pull; restart
• you are probably doing some variation of this
![Page 5: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/5.jpg)
http://searchitoperations.techtarget.com/definition/rolling-deployment
![Page 6: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/6.jpg)
Rolling deployment (cont.)
• Advantages:
• simple
• no extra hardware
• can do partial deploys
![Page 7: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/7.jpg)
Rolling deployment (cont.)
• Disadvantages:
• might drop requests
• slow rollback
• finding the "last good state"
• testing in staging only
![Page 8: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/8.jpg)
Blue-Green deployment
• two identical production environments
• only one serves requests
• deploy to the new one; test; route traffic
![Page 9: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/9.jpg)
http://fbrnc.net/blog/2016/05/green-blue-deployments-with-aws-lambda-and-cloudformation
![Page 10: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/10.jpg)
Blue-Green deployment (cont.)
• Advantages:
• testing in production
• no downtime
• quick rollback
• predictable release time
![Page 11: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/11.jpg)
Blue-Green deployment (cont.)
• Disadvantages
• cost - 2x servers
• databases require extra work
• development time to implement
![Page 12: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/12.jpg)
Fixing the database problem
![Page 13: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/13.jpg)
Optimizations• connection draining
• canary
• sticky sessions
• automated tests on green
• reproducible deploys using AMIs/Docker images
![Page 14: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/14.jpg)
Real world
• using it at Hootsuite since summer 2015
• 1K deploys in prod with no issues*
• has not affected our velocity
• deploy to prod is a mundane task
* trivial rollback in case we had any
![Page 15: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/15.jpg)
Real world (cont.)• Tech stack:
• Docker
• Asgard
• Jenkins
• ELB
• Hubot
![Page 16: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/16.jpg)
Our deployment pipeline
http://code.hootsuite.com/blue-green-deployments/
![Page 17: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/17.jpg)
Our deployment pipeline (cont.)
• cluster check
• conflict check
• merge tests
• live tests
• git merge
• manual canary release
• manual blue-green switch
![Page 18: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/18.jpg)
Spinnaker
• successor of Asgard (thanks, Netflix)
• custom build and deploy pipelines
• multiple cloud providers supported
![Page 19: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/19.jpg)
Why Spinnaker?• easy trigger integration with Jenkins
• handles all the "cloud" stuff
• server groups
• load balancers
• security groups
• useful templates built-in
![Page 20: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/20.jpg)
https://cloud.google.com/solutions/spinnaker-on-compute-engine
![Page 21: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/21.jpg)
Spinnaker stack
• Jenkins
• Packer
• .deb repo
• Spinnaker core
![Page 22: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/22.jpg)
Deploying .debs
• not everybody used to this
• lifesaver: fpm
• running your stuff - upstart
![Page 23: Safe deployments with Blue-Green and Spinnaker](https://reader031.fdocuments.in/reader031/viewer/2022030306/58cfb0a21a28ab223a8b4a3b/html5/thumbnails/23.jpg)
Demo!
http://www.spinnaker.io/docs/from-source-to-prod