Continuous Delivery to the cloud - Innovate 2014

Continuous Delivery to the Cloud

Steven Boone IBM UrbanCode Technical Sales Specialist

Michael Elder IBM Senior Technical Staff Member

Sanjeev Sharma IBM WorldWide Lead – DevOps Technical Sales

© 2013 IBM Corporation

Please noteIBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.

Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.

The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

Agenda

• DevOps – a Lean approach

• DevOps and Cloud – Cloud Portability

• DevOps with IBM Cloud Platforms

• Full Application Stack support

Agenda

• DevOps – a Lean approach• DevOps and Cloud – Cloud Portability



New Modes of EngagementSystems of Record

Data & Transaction Integrity Smarter Devices & Assets

• Data & Transactions• App Infrastructure• Virtualized Resources

• Expanding Interface Modalities• Big Data and Analytics• Social Networking

Next Generation

Architectures

Focus on Speed and AgilityFocus on Operational Costs

We are at an inflection point in the industry

DevOps approach: Apply Lean principles accelerate feedback and improve time to value

Line-of-business

Customer

1

3

2

1. Get ideas into production fast2. Get people to use it3. Get feedback

Change speed must be an asset, Not an anchor

Non-Value-added waste

Value-added production work

DevOpsTransformation

7

DevOps: Using Lean Thinking to Eliminate Bottlenecks

Type of Waste Create Feature Deliver FeatureUnnecessary Overhead

Communicating ideas/knowledge Communicating between development and operations

Unnecessary Re-work

Tasks assigned back to developers from testing and usage

Tasks assigned back to developers from production rollbacks

Over-production Unnecessary functionality produced Unnecessary hardware, data center, personnel

Non-Value-added waste

Value-added production work

DevOpsTransformation

Agenda


• DevOps and Cloud – Cloud Portability • DevOps with IBM Cloud Platforms


Implementing a DevOps toolchain

SCM

Build / CI Server

Unit testingTest Automation

Test Stubbing

Delivery Pipeline

Environment Configuration

Automated Monitoring

Asset Repository

Continuous Delivery Pipeline

DevOps and Cloud adoptionAutomating for faster delivery with DevOps and cloud

Networking Networking

Storage Storage

Servers Servers

Virtualization Virtualization

O/S O/S

Middleware Middleware

Mid Config Mid Config

Data Data

Applications Applications

Traditional On-Premises

Infrastructureas a Service

Manual

Customization; higher costs; slower time to value

Standardization; lower costs; faster time to value

Man/Auto

UC Deploy UC Deploy

Man/Auto

Networking

Storage

Servers

Virtualization

O/S

Middleware

Mid Config

Data

Applications

Platformas a Service

Codename: BlueMixPureApplication

SystemSmartCloud OrchestratorSoftLayer

JazzHub(SaaS)

Progression of Cloud Adoption: Physical Servers

Physical Servers (DEV - TEST) Physical Servers (PROD)

Environments

QA ... PROD

World-Wide Banking Applicationwar ddl mq



DEV

UrbanCode Deploy

Application

Component

Tomcat MySQL JMS WebSphere Liberty DB2 WSMB

IBM UrbanCode Deploy

VM

Progression of Cloud Adoption: Cloud for DEV-TEST

Cloud Resources (DEV - TEST) Physical Servers (PROD)

Environments

QA ... PROD




DEV

UrbanCode Deploy

Cloud

Application

Component



Public Clouds

VM VM VM

Progression of Cloud Adoption: Cloud Portability with Patterns

Cloud Resources (DEV) Cloud Resources (PROD)

Environments

QA ... PROD



IBM Virtual System PatternsOpenStack HOT templates


DEV

UrbanCode Deploy

Cloud

Application

Component


SmartCloud Orchestrator

IBM PureApplication System


Private CloudsHybridPublic Clouds

VM VM VM

Agenda



• DevOps with IBM Cloud Platforms• Full Application Stack support

Application Blueprints: Resource + Application templates

What to be deployed

Where to be deployed (On-

prem or Multiple Cloud Resources)

- Patterns

How to be deployed



Storage Storage

Servers Servers


O/S O/S



Data Data




Manual



Man/Auto

UC Deploy UC Deploy

Man/Auto

Networking

Storage

Servers

Virtualization

O/S

Middleware

Mid Config

Data

Applications




JazzHub(SaaS)

Codename: BlueMixDelivering a Composable Services development environment

Run Your AppsThe developer can chose any language runtime or bring their own. Just upload your code and go.

DevOpsDevelopment, monitoring, deployment and logging tools allow the developer to run the entire application with JazzHub and Mobile Quality Assurance

APIs and ServicesA catalog of open source, IBM and third party APIs services allow a developer to stitch together an application in minutes.

Cloud IntegrationBuild hybrid environments. Connect to on-premises systems of record plus other public and private clouds. Expose your own APIs to your developers.

Extend SaaS AppsDrop in SaaS App SDKs and extend to new use cases (e.g,. Mobile, Analytics, Web)

DevOps Service (on BlueMix): Delivery pipelines as a Service

Developer

Running Application(Dev Space)

Create & edit

Running Application(test) Running Application

(Prod Space)Running Application(Test Space)

Everything is a service in the Cloud

Deploy & test

Build Publish build

Deploy

Promote

Test as a Service

Test

Monitoring as a Service

Monitor

Dev as a Service Build as a Service Deploy as a Service



Storage Storage

Servers Servers


O/S O/S



Data Data




Manual



Man/Auto

UC Deploy UC Deploy

Man/Auto

Networking

Storage

Servers

Virtualization

O/S

Middleware

Mid Config

Data

Applications




JazzHub(SaaS)

Driving consistency with IBM’s patterns of expertise

What’s required…What the business wants… What a pattern automates…

Patterns of Expertise: Proven best practices and expertise learned from decades of client and partner engagements

• Pre-defined architecture of an application or Cloud service

• Captures best practices for complex tasks• Optimized into a deployable form for private

or public cloud• Repeatable deployment with full lifecycle

management reducing operational costs

UrbanCode Deploy supports Virtual System PatternsPureSystems and SmartCloud Orchestrator

Virtual Application Patterns

• Highly automated deployments using expert patterns

• Business policy driven elasticity

• Built for the cloud environment

• Leverages elastic workload management services

Best TCOcloud applications

Virtual System Patterns

• Automated deployment of middleware topologies

• Traditional administration and management model

• Application and infrastruture driven elasticity

Improved TCOvirtualized applications

Standard TCOexisting applications

Virtual Appliances• Standard software installation

and configuration on OS

• Images created through extend/capture

• Traditional administration and management model

• Infrastructure driven elasticity

Virtual Appliance

Metadata

Softwareapplication

Operatingsystem

Virtual Appliance

Virtual Appliance

Metadata

ApplicationServer

Operatingsystem

Virtual Appliance

Metadata

ApplicationServer

Operatingsystem

Virtual Appliance

Metadata

HTTPServer

Operatingsystem

Virtual Application PatternsVirtual System PatternsVirtual Appliances

Softwareapplication


Artifact Library

Application Resource Template

Continuous Delivery with Cloud: IBM Cloud Platforms

SCM Build Automation

DEV QA PROD

Provision infrastructure Execute application deployment and manage settings across environments

Publish builds

Pull changes

IBM SmartCloud Orchestrator

IBM PureApplication System

Cloud Environments

Manage source code for apps under development

Execute provision with configuration settings

Application binaries (versioned)

Environment configurations (versioned)

Automate provisioning of environments as part of the end-to-end delivery process – Establish and automate deployment of Application Blueprint with resource templates imported from Cloud patterns.

Deploy early and often to ensure high quality and faster releases using repeatable, reliable, and managed automation - Seamless process flow for incremental, full stack provisioning and application deployment automation

Deployment Automation

Blueprint

Application Resource Template

Continuous Delivery to IBM Cloud – Virtual System Patterns

• Capture cloud pattern to be used for creating an Environment

• Incremental deployment of application builds to cloud environments

• Map the application to multiple cloud patterns

The freedom to provision a version of a full stack or incrementally deploy an application version into an already provisioned environment

Environments | Processes | Configurations

Create env from pattern

Deploy app QA PRODDEVDEVDEVDEV

Import pattern

Cloud Platform

Creating Environment + Application Blueprints• Components capture Process and Versions

– Automation processes know how to deploy the component, undeploy the component, or any repeatable tasks relevant to the Component

– Versions know about the artifacts which are produced by the build or third parties

• Component Resources are prototypes for real Resources

– Place these in the Resource Tree wherever a matching Component Version should be deployed

– Replace specific Component mappings

Component

VersionVersionVersionVersionVersionVersionVersionProcess

JKE DB v1.0.2

JKE Tomcat v6.0.2

JKE War v3.0.10

IBM UrbanCode Deploy and Softlayer

26

Customer Needs• DevOps Motivation / Business Drivers• Principles• Continuous Delivery Pipeline• DevOps and Cloud

Customer Use Case• Requirements• Solution Architecture• Use Cases

Disclaimer

• The Use Case is about a pilot project based on an RFP conducted by a major German customer from the automotive industry

• We have signed an NDA (Non Disclosure Agreement), so any reference to the customer is not allowed

• All Use Cases are industry-agnostic

Business Drivers

Guiding Principles for a new DevOps Operating Model

The Concept of a “Continuous Delivery Pipeline”

A “Continuous Delivery Pipeline” with Cloud

CLM / SCM

Cloud Provider

PROD Stage

Continuous Integration / Build Management

and Automation

ArtefactRepository

INT Stage`QA Stage Dev Stage`

Test Mgmt / Test Automation

DeplyomentAutomation

[triggers]

[get artifatcs]

[publish artifacts /trigger Deployment]

[provision environment / deploy into Cloud]

ServiceVirtualization

[store artifacts]

[setup]

RequirementsMgmt.

[plan work items / version source code]

Development

[specifyBusinessrequirements]

[get sources]

Application Deployments and Cloud Platform Provisioning based on a common Blueprint

INT QA PROD

Instance A Instance B Instance C

App + Config D App + Config E App + Config F

Development & Build

Cloud Pattern as Infrastructure as Code”

Cloud Instances are grouped into logical environment patterns

The High Availability Cloud Pattern is used in the Integration and Productive Environment

The Standard Cloud Pattern is used in the Development Staging Environment

HA Pattern

PatternPattern

Standard HA Cloud PatternWeb

ServicesApp

Services

WAS ClusterNode 1

WAS ClusterNode 2

– Data Services

DB ClusterNode 1

DB ClusterNode 2

Web 1

Web 2

Standard Cloud PatternWeb

ServicesApp

Services

WAS ClusterNode 1

Data Services

DB ClusterNode 1Web 1

Three types of environments

• Development environments• Test / INT environments• Production environments

A pattern is a logical composition of virtual server instances and the specific middleware components plus network elements and necessary communication configuration between the elements.

IBM UrbanCode Deploy and Softlayer

34

Customer Needs• DevOps Motivation / Business Drivers• Principles• Continuous Delivery Pipeline• DevOps and Cloud

Customer Use Case• Requirements• Solution Architecture• Use Cases

Key requirement is to implement a “DevOps workflow automation” simplifying and integrating the deployment and provisioning process

Request New Environment

Request ApplicationDeployment

Infrastructure Provisioning

ApplicationDeployment

Infrastructure Provisioning

ApplicationDeployment

Solution Architecture

VLAN

Web Services /

Internet Web

VLAN

App Server Zone

WAS Node(s)

VLAN

Data Zone

DB Node(s)Web Node(s)

VLAN Management and Security Zone

IBM SoftLayer Data Center Amsterdam

Load Balancer

Security / Jump Server

Script Server

IBM UCDServer

NagiosServer

Internet

SoftLayer AutomationSystem (IMS) and API

SoftLayer Portal

Customer

Use Cases (1 of 2)

Use Cases (2 of 2)

DMS and IBM confidential •39

•13/12/2013

Use Case Example – New Release

VLAN VLAN

WAS App Server

VLAN

DB ServerActive

Web Server

VLANManagement and Security Zone

IBM Softlayer DataCenter Amsterdam

Loadbalancer

Security / Jump Server

Script Server

uDeployServer

NagiosServer

Internet

Softlayer AutomationSystem (IMS) and API

Softlayer Portal

Customer

HA PaaS PatternPassive Line

DB ServerPassive

ExistingHA PaaS PatternActive Line

1

2

3

4

65

(1) Use IBM UrbanCode Deploy to provision environment based on HA PaaS Pattern / The Script Library is used

(2) The Script Library makes use of the Softlayer API library

(3/4) The environment is provisioned and connected to the HADR DB System

(5) UCD deploys the application followed by a functional test and a manual approval

(6) Systems are visible in Nagios Monitoring System

Some Technical Insight

• A reusable Script Library has been developed to interface with the SL API (see next slide)

• The Script Library can be used independant or integrated into IBM UrbanCode Deploy

• SL FlexImage technology has been used („portable“ between Virtual and BareMetal resources in SoftLayer) for Image templating

• Production environments share the same DB/2 cluster while test and development environments have their own

• The network infrastructure is static while the systems will be dynamically integrated. We used the SL Gateway appliance based on the Vyatta Network OS

• The only system with Internet access will be the load balancer which routes the incoming traffic to the pool of webservers. Only production environments are connected to internet. All other environments are only reachable through a Security proxy

Script Library • A mixture of SoftLayer CLI and the SL SOAP API has been used for the Script

Library implementation • All scripts have been written in Perl / no root required• Re-usable solution given you have your own SL account

w3t_create Create a new environment

w3t_destroy Destroy an Existing Environment

w3t_status Show the Current Status of Environments

w3t_changeto Change the Status of an Environment

w3t_crash Crash a Single Service Instance or a Complete Environment

w3t_switch Switch the Active Environment/Reconfigure Load Balancer

w3t_mrg_etc_hosts Merge Given Config File and /etc/hosts

Script Library example

w3t_create -[h?] [-{Dsv}] [-c] [-{dpt}] -e <envno> [-l <loc>] [-b] [-i] [-n] [-y] [-A] [-S]

[-V <version>]-b Create Bare Metal / dedicated Cloud instances-c Create clustered CCIs -d Create development environment -e <envno> Environment to create envno = unique number 1 to 99 -i Infinite - don't time out waiting for CCIs to be created-l <loc> Location were to create [default: ams01] -n No execution – dry-run: show what would have been done-p Create production environment [default] -t Create test environment -y Yes – answer questions with yes (batchmode)-A Create an All-in-a-Box system-S Use Standard images instead of Flex images-V <version> Version of the disk images to be used

config/template.cci.create.<service>[.<instance>]<service> ::= “app” | “db” | “web” | “all”

<instance> ::= “1” | “2”

IBM UrbanCode Deploy Integration Concept

Environment Definition(Dev, INT,

PROD)

Application

Components Components

CustomerApplication

WAR, EAR Database DDL

ResourceTemplate

collection of resources as a model

for setting up an environment (hierarchical

representation

Agent Prototype

Other Resources

Collection of Resources

Represent Agentnot yet installed

Application Blueprint[has]

[belong to]

[contains]

[assign to]

SoftLayerScript Library

UCD calls Script Library

to generate Environment

[instantiates]

Agenda





Introducing IBM UrbanCode Deploy with Patterns

Pattern designer– Design open, full stack application

environments in a diagram or textual editor

Design once, deploy anywhere– Deploy full stack environments to multiple

clouds

Environment lifecycle management– Manage infrastructure change and easily

apply changes to existing environments

Delivery process automation– Automated delivery process with integrated

full stack environments

Design and deploy full stack application environments for multiple clouds

IBM UrbanCode Deploy with Patterns

Application

Middleware Config

Middleware

OS Config

Hardware

Envi

ronm

ent

Blu

eprin

t

• The adoption of DevOps => increased velocity of application delivery

• Puts pressure on the infrastructure to respond more quickly

• Software Defined Environments enable you to capture infrastructure as a software artifact

Deploying infrastructure changes is the current bottleneck for delivery pipeline

Application Changes

InfrastructureChanges

What is a #fullstackapp ?

Application Changes

InfrastructureChanges

Consistent Incremental Change

… …

Long, complex lifecycle

Small iterative changes

Role Layer

… …Developers/Testers

Integrators

SpecialistsCompute, Network,

and Storage

Platforms

Apps

Developers/Testers

Integrators

SpecialistsCompute, Network,

and Storage

Platforms

Apps

Full Stack Blueprint (HOT)

Cloud Orchestration

Describe software defined resources (Compute, Network, Storage) alongside middleware and applications

Capture the full stack as a blueprint document,actionable by an automated process

Compute, Network, and

Storage

Platforms

Apps


Storage

Platforms

Apps


Storage

Platforms

Apps

Dev/Test Cloud Production Cloud

Full StackBlueprint

Dev QA Prod

Apply the same blueprint to multiple environments

51

• Diagram editor• Source text editor • HOT as-you-type validation• Annotated TODO & errors• Quick palette for searching• Editor functions: Cut, Copy,

Paste, Find & Replace• Zoom diagram or text• Comment out infrastructure

• Defined by the community and vendors

• Integration with UrbanCode Deploy enables rapid configuration of full application stack

• Support ongoing updates of deployed versions through UrbanCode Deploy

Example: Software

• Version the blueprint using your source control repo (native support for git)

• Apply the new versions of the template to an existing environment or provision an entire new stack

• Validate changes with a "canary" pattern to ensure correctness and detect problems earlier

• Verify entire stack before production

Full-StackBlueprint

Software Defined Delivery

54

• View current environments• Create new environments in

the cloud• Select available resources

from Nova, Neutron, Cinder• Validation feedback in place• Update environment over time• Deploy new component

versions

• Parameterize differences between cloud environments (network ids, image ids, software versions)

• Overlay resource types for different kinds of clouds in the same blueprint

• Enable portability but optimize for each specific cloud platform

Full StackBlueprint

Config/Params

Cloud ACloud B

Config/Params

Software Defined Portability

56

• Provision Network architecture to OpenStack

• Provision Network architecture to Amazon (no OpenStack required)

• Select pre-defined configuration for each cloud

57

Resources:

IBM DevOps Page: http://ibm.com/DevOps DevOps For Dummies Book: http://ibm.co/devopsfordummies Release and Deploy For Dummies Book: http://ibm.co/1bplaQV IBM DevOps YouTube Playlist: http://bit.ly/1fiDOtl

http://ibm.com/DevOps

http://ibm.co/devopsfordummies

http://ibm.co/1bplaQV

http://bit.ly/1fiDOtl

Thank You!

Your Feedback is Important!

Access the Innovate agenda tool to complete your session surveys from your smartphone, laptop or conference kiosk.

Continuous Delivery to the cloud - Innovate 2014

Technology

Transcript of Continuous Delivery to the cloud - Innovate 2014