Sv jug - mar 2013 - sl

Next Generation Software Development:Continuous Cloud Deployment

@SachaLaboureyCEO, CloudBees, Inc.

March 20, 2013 – Silicon Valley JUGPhoto credit: @romainguy

©2013 CloudBees, Inc. All Rights Reserved

2©2012 CloudBees, Inc. All Rights Reserved

• Born in 1975 in Switzerland

• JBoss– Clustering lead – 2001– GM Europe - 2003– CTO - 2005

• Red Hat– JBoss acquisition in June 2006– co-GM of Red Hat’s middleware division– Left Red Hat in April 2009

• CloudBees– Started in April 2010– About 35 bees in 7 countries

Introduction – Sacha Labourey

X

3

• What’s the life of a developer today?• Challenges ahead…• Solution!• Cloud 101• Read my lips: SERVICE• Quick demos

Agenda



Life of a Developer Today

5©2013 CloudBees, Inc. All rights reserved

• Release v1.0

• Gather requirements for v2.0

• Work on v2.0 • Release v2.0

don’t screw it up: once it is out, it is out.

• Have your customer upgrade

– Support v1.0 for years

• Support and patch dozen of “branches” in parallel


• Work on v3.0

Typical Software Development

Mon

ths/Y

ears

Packaged Software

6

Developers vs. Sys Admins (aka Love Story)

Photo Credit: Joseph Viles/ The CW Photo Credit: http://www.sysadminday.com/people.html

http://www.sysadminday.com/people.html

7

M. Developer

Interests & Hobbies:• Give a try to the last

framework• Put in the production

application the last shiny object seen at a conference

• Participate to the Scala vs. Ceylon debate, static vs. dynamic typing, etc.

• Dream of meeting with James Gosling

M. Sys Admin/ops

Interests and Hobbies:• Not to be beeped at 2am in

the morning• Not to be beeped at 4am in

the morning• Participate to the Debian vs.

Fedora vs. Ubuntu debate• Dream of meeting with Linus

Torvald

<sarcastic mode=“on” />

8

Development? Friction!

SourceCode

Managemnt

JenkinsContinuousIntegration

UnitTest

FunctionalTest

LoadTest

IntegrationTest

Developer

“Cloud you please provision a server for me?”

“Could I get X, Y and Z installed on that server?”

“Could you please patch my server?”

“Could I please get a database dump?”

Could I get a new Git server?

Where could I run my Jenkins integration tests?

IT

9

Deployment? Friction!

Configure OS

Secure OS Setup JVM

Install AS on OS

Configure AS

Secure AS

provision nodes

Install load-balancer

(LB)Configure LB

Secure LBSetup BC for

clusterQA cluster

Configure DNS

Provision node:

Provision cluster:

Provision node

Install DBConfigure

DB

Secure DBSetup BC for

DB

Provision DB:

Provision cluster

Provision DBDeploy schema

Deploy App to each node

Deploy App:

(not a transparent update process)

Analyse sec. bulletins

Analyse QA bulletins

Analyse AS/OS logs

maintain OSmaintain

JVMmaintain AS

maintain DB

Manage environment (recurring)

maintain FWValidate/QA

stack

Measure App

performance

Acquire Hardware

Provision node

Update Cluster

Update LB

Metering/Scaling (recurring)

Update Security

10

Friction, friction, friction

provision nodes

Install load-

balancer (LB)

Configure LB

Secure LBSetup BC

for clusterQA cluster

Configure DNS

Provision cluster:

Provision cluster

Provision DB

Deploy schema

Deploy App to

each node

Deploy App:


Configure OS

Secure OS Setup JVM

Install AS on OS

Configure AS

Secure AS

provision nodes

Install load-balancer

(LB)Configure LB


clusterQA cluster

Configure DNS

Provision node:

Provision cluster:

Provision node

Install DBConfigure

DB

Secure DBSetup BC for

DB

Provision DB:

Provision cluster

Provision DBDeploy schema


Deploy App:




Analyse AS/OS logs

maintain OSmaintain

JVMmaintain AS

maintain DB


maintain FWValidate/QA

stack

Measure App

performance

Acquire Hardware

Provision node

Update Cluster

Update LB


Update Security

11

Why do we do software development again?

12

To serve the BUSINESS obviously!

Business Needs

Specification

Implementa-tion

Deployment

13

IT serves the BUSINESS?

Increase Revenue

s

Implement e-

commerce site

JVM? Ruby? PHP? Java? Scala?

Play? Groovy? Lift? Jruby? CFML? Grails? Spring? EE6?JBoss? Tomcat? Glassfish? Jetty? MySQL? MongoDB?

CouchDB? Cassandra? JMS? AMQP? Memcache?

Ehcache? Quartz? Lucene? Solr? Jquery? GWT? iPhone?

Android? JIRA? Redmine? SVN? Git? Maven? Ant/Ivy? Jenkins? Artifactory/Nexus?

etc.

AMD/Intel? HP? DELL? SPARC? UX? AWS? Linux? Windows?

Solaris? RHEL? Ubuntu? Debian? Arch? EXT2/3?

NFS/CIFS? Backup? Chef? Puppet? Continuous

Deployment? Apache HTTPD? Mod_jk? Mod_proxy? Nginx?

Haproxy? Firewall? SSL? Server provisioing? Load-

balancing? Failover? Monitoring? Management?

Alerting? Escalation procedure? etc.

15 seconds

2 weeks

18 months

1414

Didn’t we simply ask him to build an e-commerce site?


• Lots of FRICTION• Less time spent on

DELIVERING VALUE

And that’s about to get worse…

Software Development got Difficult


Challenges ahead!

17

• In parallel, demand for new needs are growing fast!– Mobile– Big Data– Social– Local– …

At the same time…

18

Entering the post-PC era

Thanks to Mary Meeker and Matt Murphy, From David Skok presentation @ MassTLC

1919

Source: Mary Meeker , 2011, www.kpcb.com/internettrends2011

2020

Source: Mary Meeker , 2011, www.kpcb.com/internettrends2011

21

Mobile Internet - 10x bigger!

Thanks to Mary Meeker and Matt Murphy, From David Skok presentation @ MassTLC


Small but big!

Source: Mary Meeker , 2012, http://www.kpcb.com/insights/2012-internet-trends


India just made the switch!

Source: Mary Meeker , 2012, http://www.kpcb.com/insights/2012-internet-trends


• Drivers are accelerating the need for NEW applications– Mobile, Mobile, Mobile, Social, Big Data, Etc.

• Time-to-market is more important than ever!– “I need it for yesterday!”– Can’t afford 18m cycles

Plus, you must be in DISCOVER MODE, since

…You don’t really know what’s needed!

Drivers & Acceleration


• You are (t)asked– To do many more new applications– To deliver them much earlier– To deliver incremental changes very quickly– To measure whether what you are doing

works– To scale quickly the few apps that will be

successful– To have them interact with an increasing

number of different devices

… Yet, nothing else should change?

Impact


Looking For a Solution?


Cloud 101

Read my lips: A-S A S-E-R-V-I-C-E

28

Easy: « Just use a XaaS! »


29

Traditional software stackWe have done this for 20 years!


Server

Hypervisor

VM

AS

JVM

LB

Application

setup

monitor

Patch

update

validate

You


Servers

OS

VM

AS

JVM

LB

Applications

setup

Monitor

Patch

Update

Validate

Cloud Computing: How to do it? Who does what?

Cloud Provider

?

31

• SalesForce.com, Zendesk, NetSuite, etc.

• Development environment– Rigid – mostly through

CONFIGURATION– When available “development” takes

place within the SaaS itself

• THINK– Standard applications

SaaS


32

Must probably be the same…

IaaS – Let’s try not to change too much…

©2012 Cloud Bees, Inc. All Rights Reserved

?

SUN/HP/IBM/XYZ

AWS/Rackspace/etc.

33

• AWS – the most popular example• Server Lego blocks – VM, storage, IP, etc.• Development environment:

– “Give me a server, an OS, a virtualization layer, an application server, a firewall, a database, I’ll deal with it! And patch it. And monitor it. And…”

– Flexible but cOmpLiCaTed

• THINK– Custom-built systems

IaaS


34

IaaS – Congratulations!


“Great” news: you have now become responsible for the data-center!


Directly managing your IaaS resources provides you with resource elasticity (CAPEX-free), but…

The amount of soft-IT typically required to do so is … higher!– All of the traditional IT activities remain

(maintain/patch/monitor OS+JVM+AS+DB++, etc.)

– + cloud-specific items: elasticity/security/automation

• I can read your mind: this is where you will start

• My advice? Do it and then move on…

IaaS – Consequences


• Cloud concepts are applied to Applications– On-demand, pay-as-you-go, elasticity, etc.– No need to handle updates, patches, scalability,

failover, etc.

• Development environment– “Give me my typical development

environment and manage everything else for me – servers, scalability, etc.”

• THINK– Custom applications– Harder to “grasp” initially

PaaS


• We focus a lot on the P/I/S on XaaS• But not enough on the … S!• We are entering a SERVICE-ERA

– IaaS - You are CONSUMING IT resources– SaaS - You are CONSUMING an online Service– PaaS - You are CONSUMING development services

This is not about software anymore!

The Cloud is refocusing IT roles


Again: Looking For a Solution?

Read my lips: A-S A S-E-R-V-I-C-E


• Release v1.0


• Work on v2.0 • Release v2.0

don’t screw it up: once it is out, it is out.

• Have your customer upgrade

– Support v1.0 for years

• Support and patch dozen of “branches” in parallel


• Work on v3.0

• Release v0.1• Implement micro-feature,

deploy, measure success, keep or kill

• Implement micro-feature, deploy, measure success, keep or kill

• Patch a bug• Implement micro-feature,

deploy, measure success, keep or kill



To Be Or Not To Be…M

on

ths/Y

ears

Days/W

eeks

Packaged Software As-A-Service


• What’s the big difference?– They’re just a long series of 1 and 0 at the end of

the day, right?

• A Services is not just “hosted software”– i.e. take your current software, host it, and keep

doing what you’ve been doing to date

Software vs. Service

42

Packaged Software vs Cloud Software Delivery

Version 1

• Shipped May 2011

Version 2


Version 3


Development Methodology VariesBonus: “We use CI”

Update CSS1st Feb

Move button1st Feb

New tab2nd Feb

Overhaul UI13th Feb

Bug fix14th Feb

Unachievable without Continuous Integration and Continuous Delivery

©2013 CloudBees, Inc. All rights reserved

42

43

Time

v1.0 Release

v2.0 Release

Time

v1.0.x ContinuousReleasesRi

sk/

Cost

Risk/

Cost

Packaged Software As-A-Service

Lower Risk and Cheap Experimentation

Same basic value prop as agile, but code is being released continuously


• In theory, fast-iteration is great! But…

FAST_ITERATION + FRICTION => HEAT!

You need a no-friction environment

Fast Iteration is good, but…


• Integrated & automated “pipeline”• Always be in a “release-ready”

state

Continuous Cloud Deployment

Code Build Test Stage Deploy

#Fail#Fail #Fail#Fail


Complex Scenarios are possible!

46

SQE Job

Database Tests

Compliance Tests

Stress Tests

Developer Job

Developer Tests

Functional Tests

X-Team Tests

Security Tests

Access Tests

Production Job

Backup Production

Deploy

Monitor

Developer Org

QA Org

Production Org

Promote

Promote


• Your Code ~= Your Service• Always be in a release-ready state

– This requires a NO-friction environment

• Enables high-frequency iterations– Code, test, deploy, measure, keep-or-

kill

Continuous Cloud Deployment

Why am I telling you all of this?

Enter into the world of CloudBees!


49

Public Edition - IaaS Providers


CloudBees Platform as a Service

Continuous

Deployment

Production

Build

Test

Provisioning

Maintenance

Jenkins

SaaS Vendor Applications

Enterprise UserApplications

JVM – Java EEJava, Python, Scala,

…

CloudBees Core Platform Services

RUN@DEV@

Repositories

Data Services

MeteringBilling

MonitoringManagement

Messaging IdentityLogging/Auditing

Back-end services

Runtime services

Development Projects

CloudBees Ecosystem

GIT Maven …SVN

APIs

CloudBees Smart

Plugins

Databases

Other SaaS

On Premise

Private Edition - On-Premise

Code

BuildTest

Deploy

Stage Deploy Scale

Iterate

50

What You Interact With When You Use CloudBees


CLI/SDK GrandCentral Web Console IDE


Development? No friction!

DeveloperCloudBees

PaaS


Deployment? No Friction!

Configure OS Secure OS Setup JVM

Install AS on OS Configure AS Secure AS

provision nodesInstall load-

balancer (LB)Configure LB


clusterQA cluster

Configure DNS

Provision node:

Provision cluster:

Provision node Install DB Configure DB

Secure DB Setup BC for DB

Provision DB:

Provision cluster

Provision DB Deploy schema


Deploy App:




Analyse AS/OS logs

maintain OS maintain JVM maintain AS

maintain DB


maintain FW Validate/QA stack

Measure App performance

Acquire Hardware

Provision node

Update Cluster Update LB


Update Security

(transparent, multi-version,HA, scalable)

• No need to care about servers, load-balancers, firewalls, backups, etc.

• The environment is constantly managed and monitored

• Scalability happens in realtime

• Integrated failover/HA

• We do « Ops », you do « Dev »

Deploy app to traditional Java platform

Deploy app to CloudBees

DEPLOY


Value à la carte – NO FRICTION

Code

BuildTest

Deploy

OURYOUR TEAM

YOUR PROJECT

54

• Lose it!– Mobile application– >> millions of users– > 30,000 transactions per minute, at peak time

• And… only 4 employees and …2 developers!– No IT, no servers, no DevOps– Complete focus on SOFTWARE DEVELOPMENT– Anything else is handled by CloudBees

• Unmatched productivity level!– This is possible TODAY– Will you wait for your competitors to shoot first?

Example: Lose it!

55

Continuous Cloud Delivery at CloudBees

Jenkins Core

Jenkins Plugins

Staging

Production

Web Console

SDK

API

Identity

Provisioning

Monitoring

Messaging

Services Platform

Generic App Container

Database

Router

Curated Stack

Git

SVN

Forge

Unit Tests StageCross Service Tests UAT Prod

As needed, multiple times

per day

56

Typical Mobile App with Java back-end

???CustomMobile

Back-end

Fast-IterationDiscovery

StableNO-discovery

First Demo• Complete Java Server Application…

• (Backbone.js front-end, REST back-end, DB)

• … with full lifecycle automation …

• (Code, build, test, deploy)

• … and do it in a single click!

ClickStart

58

cloudbees.com

What’s up with this new

framework!

Get started in a snap! “ClickStart”

58

CloudBeesGit

Repo

Database

Build/Test

db:create

App

PartnerServicePartnerService

CloudBees API

Grand Central Web

Console

app:creategit clone

Jenkins job create

Bind

Bind

Trigger

Deploy

Wow! At yourservice

©2013 CloudBees, Inc. All rights reserved

https://grandcentral.cloudbees.com/?CB_clickstart=https://raw.github.com/CloudBees-community/java-rest-backbone/master/clickstart.json

59

cloudbees.com

Continous Delivery at work

CloudBeesGit

Repo

Database

Build/Test

App

PartnerServicePartnerService

Bind

Bind

TriggerDeployI want to make

some changes!

Local GitRepo

git clonegi

t com

mit git push

Chan

ges

Wow! At yourservice

#fail #fail #fail

Mobile!

CodeBuild App

Code

BuildTest

Deploy

TestBetaDeplo

y

GADeplo

y

• Get started fast!

• Team Work• Automatically

push new code to test

#fail

• Build on several targets: iOS, Android, etc.

• If it fails, stop

• Test is costly!• Easy-first, then

increase!• Basic test• Multi-device• Gesture• Etc.

• If it fails, stop

• Deploy in the field on real users

• Gather input• Retrieve

dumps• Etc.• If it fails, stop

• D-Day! • Push apps to

real-life users!• If it fails, stop

Build Matrix• Testing mobile application requires testing

on lots of different devices, form factors, etc.

• Create one build job per form factor/generation?

• No!

• Jenkins Build Matrix!

Jenkins Build Matrix

http://wiki.cloudbees.com/bin/view/DEV/Matrixed_Jenkins_Android_Builds

So much more!

Getting started• Lots of new things to learn!

• (delegation of trust, access to legacy data, latency, elasticity, iterativity, etc.)

• Identify an application

• Start small & Non-critical

• Focus on « Learning »

• And extend from there

• (and enjoy!)


• We are entering a SERVICE-era• Packaged software is dead

– Too long cycles, no ability to measure, etc.

• How can you remain competitive?

I’m ready to deploy! Are you?

TIME TO WAKE-UP!

Thank you

@SachaLabourey

“It is easier to ask forgiveness than it is to ask permission”

-- Grace Hopper

Sv jug - mar 2013 - sl

Documents

Transcript of Sv jug - mar 2013 - sl