Continuous Deployment

Timothy FitzCTO of Canvas

“Continuous integration involves integrating early and often, so as to

avoid the pitfalls of "integration hell". The practice aims to reduce timely rework and thus

reduce cost and development time.”

“Continuous deployment involves deploying early and often, so as to avoid

the pitfalls of "deployment hell". The practice aims to reduce timely rework and thus

reduce cost and development time.”

The Vision

• On every key press– Compile– Run automated tests– Deploy

• “Live Coding”

Eliminate Waste

• Deploying code validates assumptions• Bad assumptions cause waste– Code built on top is waste– Design built on top is waste– Thought built on top is waste

The Reality

• Change has risk• Infrastructure isn’t free

At Canvas

• Small CD shop• 5 committers• Deploy process is “git push”

At Scale: IMVU

• Profitable• MMO + Virtual Economy Etsy• 50+ Technical Staff

At Scale: Etsy

The Deploy Equation

• Direct Value (DV)• Information Value (IV)• Deployment Risk• When IV + DV > Risk: Deploy!

Increase Information Value

• Small commits mean more information earlier• Implement features implementation-risk-first• Conscious information gathering

Increase Direct Value

• Feature shippable from day 0• Never blocked on deploy cycle• Higher velocity• Lean Thinking

Risk=

Exposure * Probability * Severity

Decrease Exposure

• Dark launch non-frontend changes• Controlled exposure via feature rollout code– Expose to staff/QA only– Expose to opt-in beta testers– Gradually increase exposure from 1-100%

• Feature-level rollback

Decrease Probability

• Automated tests– Regression / Functional / Integration tests– Unit tests– Browser tests / Click tests– 3rd party integration tests

• Manual QA prior to exposing features• Build code in a deploy mindset

Decrease Severity

• Decrease length of degradation– Production Alerts– Cluster Immune System– Instant production roll back

• Decrease effects of degradation– Stability through isolation– Product level fault tolerance– Lock down core infrastructure

FAQ

• What about shema changes?• Great, how do I get started?

Schema Changes: They hate your code

• Code and schema move in locked steps• Favor schemaless design– Minimize classical schema changes– Offend DBAs with your lack of normalization– Lightweight/Schemaless databases (“nosql”)

Schema Changes: They hate your uptime

• Did I mention schemaless databases yet?• Apply updates to standbys• Blue/Green cluster setup

Great, how do I get started?

• Nike method: Just do it

tl;dr

• We’ve come a long way• We have a long way to go• IV + DV > Exposure * Probability * Severity.• Rethink schema changes• Continous Deployment: Just do it• Questions?