Cyansible

58
Cyansible Blending Blue/Green Deployments @Betterment

Transcript of Cyansible

Page 1: Cyansible

CyansibleBlending Blue/Green Deployments

@Betterment

Page 2: Cyansible
Page 3: Cyansible

Techcrunch DisruptMay 2010

90,000 customers, more every minute.

Fastest growing automated investing service

$2B+

Who are we?

Page 4: Cyansible
Page 5: Cyansible
Page 6: Cyansible

betterment :

investing::

devops : engineering

Page 7: Cyansible

it’s 2012.let’s ship some

code.

Page 8: Cyansible
Page 9: Cyansible

Betterment@2012

Page 10: Cyansible

A Better Migration: From Snowflakes to StormtroopersWednesday, July 22, 6:30p - 8:00p @ AWS Pop-up Loft | NYC

Shameless Plug:

Page 11: Cyansible

it’s 2015.

let’s ship new code…without interrupting

production

Page 12: Cyansible
Page 13: Cyansible

●Predictable●Repeatable●Minimal Human

Interaction●Zero User Interruption●Contained Failure

Dream Delivery

Page 14: Cyansible

Blue/Green Deployments

http://martinfowler.com/bliki/BlueGreenDeployment.html

Page 15: Cyansible

DNS

ELB

Page 16: Cyansible

Pre-flight Checklist

❏Stateless Servers❏Server Health Check❏Duplicate Full Stack

including RDS Replica

Page 17: Cyansible

Wait. Two Databases?“There's still the issue of dealing with missed transactions while the green environment was live, but depending on your design you may be able to...● feed transactions to both environments in such a

way as to keep the blue environment as a backup when the green is live. Or you may be able to...

● put the application in read-only mode before cut-over, run it for a while in read-only mode, and then switch it to read-write mode.”

http://martinfowler.com/bliki/BlueGreenDeployment.html

Page 18: Cyansible
Page 19: Cyansible

“Code𝓲 always works on Schema +1𝓲 ”

(A.K.A. old code works on new schema)

Page 20: Cyansible
Page 21: Cyansible

Publish, Migrate, Deploy

Page 22: Cyansible

Jenkins’ Job

1.Build2.Test3.Package4.Publish5.Run Migrations6.Invoke Ansible7.Cull Zombies

Page 23: Cyansible

Ansible’s Job

1.Check for S3 deliverables2.Spin up new EC2 Instance(s)3.Apply role(s) to instance(s)4.Find instance(s) in ELB5.Add new instance(s) to ELB &

tago status: in-use

6.Remove & tag instanceso status: zombie

Page 24: Cyansible
Page 25: Cyansible
Page 26: Cyansible

Bootstrapping Ansible

./exec/ directorycontains Jenkins entry

points

Ansible code lives in the repo it provisions. 

Publish it like an app.

Page 27: Cyansible
Page 28: Cyansible

./exec/brochure-deploy.sh

Page 29: Cyansible
Page 30: Cyansible

brochure-deploy.yml

you are here.

00:00:10.888

Page 31: Cyansible

provision-new-ec2-instances

brochure-deploy >

Page 32: Cyansible

identify-elb-AZs

brochure-deploy > provision-new-ec2-instances >

Page 33: Cyansible

ec2-deploy

brochure-deploy > provision-new-ec2-instances >

Page 34: Cyansible

AMI Hierarchy

Page 35: Cyansible

brochure-deploy.yml

you are here.

00:01:18.415

Page 36: Cyansible

configure-instancesbrochure-deploy >

Page 37: Cyansible

roles/deploy/brochure/tasks/main.yml

brochure-deploy > configure-instances

Page 38: Cyansible

brochure-deploy.yml

you are here.

00:04:49.414

Page 39: Cyansible
Page 40: Cyansible
Page 41: Cyansible

brochure-deploy.yml

you are here.

00:04:50.188

Page 42: Cyansible

find-instances-in-elb

brochure-deploy >

Page 43: Cyansible

brochure-deploy.yml

you are here.

00:04:50.911

Page 44: Cyansible

add-instances-to-elb

brochure-deploy >

Page 45: Cyansible

brochure-deploy.yml

you are here.

00:05:06.295

Page 46: Cyansible
Page 47: Cyansible

decommission-instances

brochure-deploy >

Page 48: Cyansible

brochure-deploy.yml

you are here.

00:05:21.226

Page 49: Cyansible
Page 50: Cyansible
Page 51: Cyansible
Page 52: Cyansible
Page 53: Cyansible
Page 54: Cyansible

EC2 INSTANCE

Page 55: Cyansible

●Predictable●Repeatable●Minimal Human

Interaction●Zero User Interruption●Contained Failure

Dream Delivery Achieved

Page 56: Cyansible

The Future

● Long Running Instances + Dockero Huge speed improvement

● Post Monolith, Abandon Jenkins?o Travis CI for Build/Testo Tower for Deployment Orchestration

● Ansible Galaxy?

Page 57: Cyansible

Questions?

[email protected]@nonrational

github.com/nonrational

Page 58: Cyansible

[email protected]

All code snippets & diagrams contained in this presentation are property of Betterment, but please learn from them.

All photographs / GIFs used in this presentation are someone else’s.Street Fighter, Back To The Future, Indiana Jones, Futurama, and Arrested Development are someone

else’s property too.