Continuous Deployment

Tiago Henriques

Solutions Architect, Amazon Web Services

CONTINUOUS

INTEGRATION

DEVELOPER

SOURCE CODE

REPOSITORY

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

CONTINUOUS

INTEGRATION SERVER

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

CONTINUOUS

INTEGRATION SERVER

PICK

TASKS

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

CONTINUOUS

INTEGRATION SERVER

SUBMIT

CODE

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

CONTINUOUS

INTEGRATION SERVER

SCHEDULE

BUILD

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

CONTINUOUS

INTEGRATION SERVER

RECURRENT

BUILDS

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

CONTINUOUS

INTEGRATION SERVER

CODE

FETCH

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

CONTINUOUS

INTEGRATION SERVER

CODE QUALITY

TESTS

TEST

RESULTS

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

CONTINUOUS

INTEGRATION SERVER

BUILD OUTPUT

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

CONTINUOUS

INTEGRATION SERVER

DOCS

BINARIES

& PACKAGES

SOURCE CODE

REPOSITORY

DNS

CONTINUOUS

INTEGRATION SERVER

PROJECT

MANAGEMENT SERVER

BUILDS

PAIN POINTS • UNIT TESTS INCOMPLETE

• MOCK TESTS MAINTENANCE

• EXPENSIVE TEST ENVIRONMENT

• TEST ENVIRONMENT ≠ PRODUCTION

• DEPLOYMENT CYCLES

ON-DEMAND

PAY AS YOU GO

ELASTIC

=

PROGRAMMABLE

PLATFORM

IF YOU CAN PROGRAM IT

YOU CAN AUTOMATE IT

AWS

CLOUDFORMATION STACK-BASED DEPLOYMENT

SERVICE

CLOUDFORMATION

TEMPLATE

{ "Description" : "Create RDS with username and password", "Resources" : { "MyDB" : { "Type" : "AWS::RDS::DBInstance", "Properties" : { "AllocatedStorage" : "500", "DBInstanceClass" : "db.m1.small", "Engine" : "MySQL", "EngineVersion" : "5.5", "MasterUsername" : "MyName", "MasterUserPassword" : "MyPassword" } } } }

"AWS::CloudFormation::Init" : { "config" : { "packages" : { "yum" : { "mysql" : [], "mysql-server" : [], "httpd" : [], "php" : [], "php-mysql" : [] } }, "sources" : { "/var/www/html" : "https://s3.amazonaws.com/my-builds/build-v4.zip" } }

{ "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance", "Type" : "String" } }, }

CLOUDFORMATION

TEMPLATE

PROCEDURAL

DEFINITION Create it programmatically

KNOWN

CONFIGURATION Store stack configuration in

source control

PARAMETER

DRIVEN Dynamic and user-driven

templates

COLLABORATION Share templates with ease

as just files

APPLICATION

VERSIONS

+ INFRASTRUCTURE

VERSIONS

CLOUDFORMATION

TEMPLATE

HTTP://SORCERY.SMUGMUG.COM/

TEST ENVIRONMENTS

30,000 REQUESTS / SECOND

1 TB TRAFFIC / DAY

“…AWS seemed to be the best solution available to allow a small, independent company to rapidly develop and test a completely new infrastructure, and host it. We also loved the flexibility that AWS allowed us, when spinning up smaller test environments, for beta trials, QA, localization, and during development. The low initial cost was also crucial.” Alex Evans, CTO

LOAD TESTING

USING AMAZON EC2 TO SIMULATE

2.4 MILLION PLAYERS

CONTINUOUS

DEPLOYMENT SMALL, FREQUENT CHANGES

CONSTANTLY INTEGRATING INTO

PRODUCTION

KEY = ITERATION

ITERATION

= MODIFY THE SYSTEM TO BETTER

MEET THE EXPECTATIONS OF

YOUR USERS

11.6s

Mean time

between

deployments

(weekday)

1,079

Max number of

deployments in a

single hour

10,000

Mean number of

hosts

simultaneously

receiving a

deployment

30,000

Max number of

hosts

simultaneously

receiving a

deployment

DEPLOYMENTS AT

AMAZON.COM

SOFTWARE DEPLOY

≠ PRODUCT LAUNCH

1.5 BILLION PAGE VIEWS

OCTOBER 2012

$83 MILLION IN TRANSACTIONS

4.2 MILLION ITEMS SOLD

30 DEPLOYS PER DAY 1 DEPLOY EVERY 20 MINUTES

"Production is truly the only place you

can validate your code."

AWS OPSWORKS INTEGRATED APPLICATION

MANAGEMENT

14 BILLION REQUESTS/MONTH

50 000 DATABASE UPDATES / SEC

NO CACHE

A / B TESTING

DATA-DRIVEN

ARCHITECTURES

METRICS @ETSY

COST-ORIENTED

ARCHITECTURES

PHP+APACHE+VARNISH

NGINX+NODEJS

CONTINUOUS

INTEGRATION

CONTINUOUS

DEPLOYMENT

Continuous Deployment

Tiago Henriques

Solutions Architect, Amazon Web Services

CONTINUOUS

DEPLOYMENT

=

CONTINUOUS

EXPERIMENTATION

CONTINUOUS

DEPLOYMENT

=

CONTINUOUS

IMPROVEMENT

INNOVATE

« Want to increase innovation?

Lower the cost of failure »

Joi Ito

SPEED AND AGILITY

Experiment

Often

Fail quickly at

a low cost

More

Innovation

Experiment

Infrequently

Failure is

expensive

Less

Innovation

“ON-PREMISE”

TEST ENVIRONMENTS ON DEMAND

ODIGEO GROUP ODIGEO Group, creado en Julio 2011, está formado por eDreams, GO Voyages, Opodo y Travellink. ODIGEO Group actualmente opera y ofrece sus servicios en un total de 38 paises. En 2011, servimos conjuntamente 14 milliones de clientes y 3.9 Billion en Reservas. Somos más de 1400 empleados. Y más de 300 empleados en el equipo de IT. Más de 80 nuevos miebrss en el equipo de IT en el último año.

CI

• CI Server

• Plugin jenkins – Amazon EC2

• VCS: SVN y Bitbucket.

• 1 job por rama de desarrollo.

• Cada commit lanza una ejecución de los jobs.

• La CI está compuesta por:

• Métricas de calidad.

• Test unitarios.

• Test automáticos.

• Publicación de artefactos.

Entornos de test Problema

Requerimientos

• Incremento del equipo de IT

• No disponibilidad de entornos de test.

• Creación de los entornos bajo demanda.

• Gestión de los entornos por los usuarios.

• Los entornos sin actividad se deben autodestruir.

• Sin límite de entornos de test concurrentes.

• Start/Stop automáticos.

QAM Solución

• Amazon EC2

• Tenemos una ami con la configuración del entorno.

• Configuración por puppet del entorno.

• La versión de software se descarga de Nexus.

• Aplicación de gestión desarrollada en colaboración

con Celingest

• Gestión de usuarios.

• Auditaría del uso y consumo.

• Control de instancias, número y tiempo.