© 2016 IBM Corporation

UrbanCode Deploy DevOps Best Practices MichaelD.Elder

IBMDistinguished EngineerChiefDeveloper, UrbanCode

mdelder@us.ibm.com@mdelder

Get this deckhttp://bit.ly/2c1G8sy

2© 2016 IBM Corporation

PleaseNote

2

• IBM’sstatementsregardingitsplans,directions,andintentaresubjecttochangeorwithdrawalwithoutnoticeatIBM’ssolediscretion.

• Informationregardingpotentialfutureproductsisintendedtooutlineourgeneralproductdirectionanditshouldnotbereliedoninmakingapurchasingdecision.

• Theinformationmentionedregardingpotentialfutureproductsisnotacommitment,promise,orlegalobligationtodeliveranymaterial,codeorfunctionality.Informationaboutpotentialfutureproductsmaynotbeincorporatedintoanycontract.

• Thedevelopment, release, andtimingofanyfuturefeaturesorfunctionalitydescribed forourproductsremainsatoursolediscretion.

• Performance isbasedonmeasurementsandprojectionsusingstandardIBMbenchmarksinacontrolledenvironment.The actualthroughputorperformance thatanyuserwillexperience willvarydependinguponmanyfactors,includingconsiderationssuchastheamountofmultiprogrammingintheuser’sjobstream,theI/Oconfiguration,thestorageconfiguration,andtheworkloadprocessed. Therefore, noassurancecanbegiventhatanindividualuserwillachieveresultssimilartothosestatedhere.

3© 2016 IBM Corporation

•UnderstandingDevOps

•DevOpsBestPractices

• Clouds&Containers

Agenda

IBMConfidential

© 2016 IBM Corporation

UnderstandingDevOps

5© 2016 IBM Corporation

Failures duetoinconsistentdev and

productionenvironments

Bottlenecks tryingtodelivermorefrequentreleases tomeetmarketdemands

Complex,manual,processesforreleaselackrepeatabilityand

speed

Poorvisibilityintodependencies across

releases, resources,andteams

Softwaredeliverychallenges

DailyBuild

MonthlyDelivery

Whodidthis lasttime?

Dave…

Dave’snothereman…

Dev

Prod

6© 2016 IBM Corporation

HighperformingteamsadoptDevOps

Reference: 2013 State of DevOps Report by PuppetLabs

7© 2016 IBM Corporation

Intuitive and Scalable Model Driven Deployment

Composite Applications

Components

Re-usable Workflows Environment ManagementSIT

PROD

The “What” The “How” The “Where”

DEPLOYMENT AUTOMATION

8© 2016 IBM Corporation

Intuitive and Scalable Model Driven Deployment

Composite Applications

Components:Container Images

Re-usable Workflows Environment ManagementSIT

PROD

The “What” The “How” The “Where”

DEPLOYMENT AUTOMATION

$ docker run .. {image}:{version}

9© 2016 IBM Corporation

Aboutphilosophy

Cultureofcontinualexperimentationand

learning

• Productionlikeenvironments• Fullyautomateddeployments• Accelerateddelivery cycles

Build– Measure- Learn

“Successisnotdeliveringafeature; successislearninghowtosolvethecustomer’sproblem.”

10© 2016 IBM Corporation 10

IBMBluemix GarageMethod

PracticesProcesses or actions you & your team perform to achieve an outcome.

TracksA selected set of practices, tools, and toolchains to create a cloud native or cloud enabled solution

ToolchainComplementary & compatible tools that when used together create a flexible solution

Experts & ExperiencesListen to our Experts to learn how we transformed our business

Combines industry bestpracticesforDesignThinking, LeanStartup,AgileDevelopment,DevOps,and Cloudtobuildanddeliverinnovative solutions.

https://www.ibm.com/devops/method

11© 2016 IBM Corporation © IBM Corporation

Newcontentaddedweekly

12© 2016 IBM Corporation

Accelerate Innovation

ivDeliverCodeThink Run

Systems of engagement

Systems of Record

Enterprise Applications

Production

MostenterprisesneedacombinationofDevOpspatterns

12

API Catalog

Operational Excellence

ivDeliverCodeThink Run Production

Shared Services

Digital Applications

• Coordinatingdifferentteamsworkingatdifferentspeeds

• Balancing between agility,stability, andrisk

• DecoupleacrossAPIboundary,butmanagerelease processwithconsistent approach

13© 2016 IBM Corporation

ExpectationsofContinuousDeliveryUser Experience Today Emerging

Primary Workload Type Software supports core business function

Software enables innovative business approaches to differentiate from competitionBig Data, Analytics, Mobile/Social Channels

Delivery Model Planned Incremental (DevOps)

Development and Operations Team Sizes 100s and costly 10s with built-in DevOps automation

Release Frequency Months to years Days to weeks, based on business opportunity

Integration Frequency Weeks Continuous

Infrastructure Deployment Days Minutes

Time to Value Planned Opportunistic

Operational Model Systems management Built into application, recovery-oriented, continuous availability

Service Sourcing Develop Consume and assemble (public and private)

Feedback Loop Periodic Continuous interaction with clients

14© 2016 IBM Corporation

Increasetheoperationalawarenessofyoursoftware earlierinthedevelopmentprocess.

WhatdoesDevOpsmeantome?

Howdoyoudothat?§ Architecture

§ AutomatedTesting

§ AutomatedDeployments

§ Production-LikeEnvironments

§ AutomatedReleasePromotionProcess

§ Versioncontrolofallsoftware,automation,andconfigurationWhydothis?

Becausefasterfeedbackloopsenablerapidevolution ofideasandthereforefasteriterationsofyoursoftware

ShiftLeft

15© 2016 IBM Corporation

Bettercommunication betweenthosewho createandthosewhooperate(samepeopleinsomecases)

Reducedfearofbreakingthebuild/deployment/environment

Failfastbeforeyou“failall”

Heavyfocusonexperimentationandlearning

Improvethespeedofyourfeedbacklooptoenablerapidevolutionofideas

WhatarethecharacteristicsofteamswhopracticeDevOps?

© 2016 IBM Corporation

DevOpsBestPractices

17© 2016 IBM Corporation

§ Architecture

§ AutomatedTesting

§ AutomatedDeployments

§ Production-LikeEnvironments

§ AutomatedReleasePromotionProcess

§ Versioncontrolofallsoftware,automation,andconfiguration

Aboutournexttopics

18© 2016 IBM Corporation

IBMUrbanCode forContinuousDelivery

Drive down costReduce amount of manual labor, resource wait-time, and rework by eliminating errors & providing self-service environments

Speed time to marketIncrease frequency of software delivery through automated, repeatable deployment processes across development, test and production

Reduce riskRobust configuration management, coordinated release processes, audits, and traceability

Enabling clients to continuously deliver applications with high quality and low risk

IBM UrbanCode Build is a continuous integration and build management server optimized for the enterprise, designed to make it easy to scale the configuration and management of build infrastructure and seamlessly plug in to development, testing and release tooling.

IBM UrbanCode Deploy enables hybrid cloud by automating the deployment of applications, databases, configurations and containers into development, test and production on prem or in the cloud, helping to drive down cost, speed time to market with reduced risk. With full-stack environment management and deployment automation, users to design, deploy and update full-stack environments for multiple clouds, including Bluemix, Softlayer, AWS, Azure, and virtualized environments.

IBM UrbanCode Release is an intelligent collaboration release management solution that replaces error-prone manual spreadsheets and streamlines release activities for application and infrastructure changes.

19© 2016 IBM Corporation

Aboutyourarchitecture

• ArchitectureshouldsupportDevOps principlessuchasstagedrollout,operationalinsights,andscriptability

• Eachresourceprovidessomeverypracticaladviceforbuildingsystemswhicharefocusedonreliabilityandfeedbackloops

Experiment!:Websiteconversionrate

optimizationwithA/Bandmultivariatetesting

ReleaseIt!:DesignandDeployProduction-Ready

Software

20© 2016 IBM Corporation

Applicationmodel

Reliablycaptureapplicationcontentandconfigurationsettingswithversioningandtraceability

21© 2016 IBM Corporation

Applicationenvironments

Definewherecomponentsaredeployedandcaptureconfigurationsettingsperdeploymentenvironmentforanapplication

22© 2016 IBM Corporation

Templatingforgoodarchitecture

Templatescanbecreatedtostandardizecomponentprocesses&config

23© 2016 IBM Corporation

Createapplicationtemplatesforacceleratedonboarding

Templatescanbecreatedtostandardizecomponentprocesses&config

24© 2016 IBM Corporation

Automatedtestingiscriticalforautomatedfeedback

• This isTHEHARDESTpartofthewholeexperience

• NOONEisgreatatthis,sodon’tgetdiscouraged

• Facebook’s processforcodereview include statsonhowmanytests areconsistently brokentodecidewhether thetestfailuresarenoteworthy

• Continuously improveyourcoverage, flexibility, anddiscipline

• Consider allaspects oftesting:

• ProgrammaticAPI(JUnit)

• RESTAPI(HTTPUnit, RESTFuse)

• WebUI(Selenium)

• Behaviordriventesting (Cucumber)

• Performancetesting (ApacheJMeter)

• Securitytesting (RationalAppScan &IBMQRadar)

25© 2016 IBM Corporation

Embeddingunittestsinyourclassesdirectly

• PromotedapproachbyNetflixtoreducefrictionandintroduces limited additionalbytes relativetothirdpartylibraries

• Makes iteasytowritecommands,theUnitTestbecomes thetestharnessandverification

• AlwaystestSuccess() andtestFailure() usecases toensureexpected behavior

importstaticorg.junit.Assert.assertEquals;importstaticorg.junit.Assert.assertNotNull;

publicclassOSDeployEnvironmentCommand extendsOSAbstractOrchestrationCommand<Response>{

publicstaticclassUnitTest {@TestpublicvoidtestSuccess() throwsInterruptedException,

ExecutionException,JSONException {…Future<Response>request=

newOSDeployEnvironmentCommand(…).queue();

Responseresponse =request.get();assertNotNull(response);assertEquals(HttpStatus.SC_OK,

response.getStatus());}

}

26© 2016 IBM Corporation

TestingYourDeployedApplication– RESTAPI

• WehaveusedRESTFusewhich isanEclipse project

• Captures RESTcallsasannotations, freeingupyourtestmethodtojustverifybehavior

• Requires alittle“hoopjumping”fortesting aseries ofcalls (e.g.POST..PUT..GET)

importstaticcom.eclipsesource.restfuse.Assert.assertCreated;import...restfuse.annotation.Authentication;import...restfuse.annotation.HttpTest;import...restfuse.MediaType;

@HttpTest (method=com.eclipsesource.restfuse.Method.POST, contentMethod="getBlueprintRequestBody",authentications={@Authentication(type=BASIC,user="{keystoneUser}",password="{keystonePassword}")},path="/rest/blueprint",type=MediaType.APPLICATION_JSON

)publicvoidtestCreateBlueprintOK()throwsException{assertCreated(response);}

27© 2016 IBM Corporation

BehaviorDrivenDevelopment&Testing

• Capturesnatural languagestatements using“Given”,“When”,“Then”clauses

• Eachline ismatched toasnippet ofcodebasedonregularexpression

• Complete codestatementsareexecuted tovalidatebehavior

• Bridgebetween businessgoalsandtechnical details

Scenario:Success– UploadclientdirectoryGivenIprovidevalidcredentialstotheapi clientAndIwanttousetheapi clienttomakerequestsWhenIuploadthedirectory”data/sample-app”withtheclientThenthefilesreturnedmatchestheuploaddirectory”data/sample-app"

28© 2016 IBM Corporation

TestingYourDeployedApplication– WebUI

• Selenium istheonly“apparent”choicethatwe’vecomeacrossthat’seffective

• Couples automated testtosomedetails ofyourwebUIandmustbekept insync

• We’vedeveloped afewbestpractices aroundlocationelements formaximumflexibility, couldpublishthisifthere’sinterest

publicclassEditorsPage extendsUCDPPage {…@FindBy(id="new-blueprint-button")privateWebElement newButton;…publicvoidcreateFile(String name,Stringid,String[]values){newButton.click();util().waitForElementPresentAndVisible(By.className("new-file-dialog"));WebElement widget=util().findElement(By.className("new-file-dialog"));

Dialog dialog=Widget.init(this, widget,Dialog.class);dialog.setValues(values, true,true);dialog.clickPrimaryButton();

…}…}

29© 2016 IBM Corporation

Automateddeployments• Visibility andautomatedcontrolofyourapplication deployment process

• Manageapplicationcomponents andversions

• Manageconfigurationsacrossallenvironments

• Offersecure‘self-service’capabilities

• Increasetransparency• Ensuregovernance andcompliancy

30© 2016 IBM Corporation

Defineautomatedprocesses foreachapplicationcomponentEachcomponentdefinesaseriesofsteps,backedby

plugintasksorcustomscripts,todeployit

31© 2016 IBM Corporation

Defineautomatedprocesses fortheapplicationOrdercomponentsorapplylogicwhichis“outside”of

anyspecificcomponent

32© 2016 IBM Corporation

IBMUrbanCodeDeploy– Plug-ins

Over 150 – in over 50 categories

33© 2016 IBM Corporation

Production-LikeEnvironments– UnifiedResourceTree

Provideslogicalhierarchyofdeployedartifactsandtheirmiddlewarecontainers

DEVResources- WebHost- WebComponent- Database- DatabaseComponent

QAResources- WebHost1- WebComponent- WebHost2- WebComponent- Database- DatabaseComponent

PRODResources- WebHost1- WebComponent- WebHost2- WebComponent- WebHost3- WebComponent- WebHost4- WebComponent- DatabasePrimary- DatabaseComponent- DatabaseSecondary- DatabaseComponent

34© 2016 IBM Corporation

Production-LikeEnvironments– LocalDevelopment

PossiblyoneofthemostimportantprinciplesofDevOpsShiftleftanyoperationalconcernsthatleadto

differencesfromDEVtoPROD

https://hub.docker.com/u/ibmcom/#

35© 2016 IBM Corporation

DeploymentofApplicationsacrossEnvironmentsApplicationsaremanagedfordeployment

Componentsfromtheapp

SpecificversionsoftheComponents’artifactsaredeployed

Compliancykeepstrackofwhethertheactualversionsmatchestheexpected

EachComponentknowsitsrelevantdeploymentprocesses

CompareversionsacrossenvironmentsanddefineSnapshotsofknownworkingsetsofComponents

Viewdeploymentrequestsagainsttheenvironment

https://developer.ibm.com/urbancode/AutomatedReleasePromotionProcess

36© 2016 IBM Corporation

PromotionLifecycles

• Shouldfocusonconsistencyfromendtoend

• Usethesamelogicalorganizationandcomponentautomation inDEV,QA,PRODtovalidatetheautomationearly inthedelivery cycle

• Applyapprovalgateswherenecessary toensureappropriatequalityvalidationhasoccurred

IBMCommerce

WebSphere ND

Static Content

Commerce Data

IBMCommerce

WebSphere ND

Static Content

Commerce Data

37© 2016 IBM Corporation

QualityAssurance– Automatedgates

• UseComponent Statuses andEnvironmentGateswherepossible

• Definestatuses forallApps

• UseRESTAPI/CLItotag/approveComponents orSnapshots basedonautomated verification(CIorothertesting)

• Onlycomponentsversions/snapshots withtagcanenterenvironment

38© 2016 IBM Corporation

QualityAssurance– ManualSignoff

• Manualapprovalsarepossible, butrecommendthatasmuchaspossible isautomated (e.g.withstatuses+gates)

• Approvalprocesscantriggeremails orsome clients use”chatops”tools like Slacktostreamline theprocess

39© 2016 IBM Corporation

Inventoryovertime

40© 2016 IBM Corporation

ConfigurationManagement- SnapshotsCreating a Snapshot

Component Versions

1

1

2

2

3

3

321Web

Mid. Code

DB

Snapshot

3

2

1

Mid. Config 1 2 3 3

Snapshot Deployment

ENVSnapshot

3

2

1

WEB HOST

MID HOST

DB HOST

2

1

1

3

2

X

3 3 1

Contents ofenvironments thatpasstests• containdeployable versionandconfiguration

Deployments arebasedondeltasTheyprovide Automation,Audit,andVisibility

41© 2016 IBM Corporation

VersionapplicationswithSnapshots

Capturetestedcomponentversionsandconfigurationsettingsasasnapshotthatispromotedtootherenvironmentsforbettervisibilityandcontrol

42© 2016 IBM Corporation

Better Release Planning and Management

• Enterprise Calendaring

• Release Process Checklist Templates

• Environment Reservation

Effective Change and Risk Management

• Rational Team Concert integration

• Impact Analysis

Continuous Delivery with Auto-Progression

• IBM UrbanCode Deploy integration

Increased Visibility and Control

• Pipeline View

• Federated Dashboard

• Segment Dependency Graph

UrbanCodeReleaseTransformchaoticreleaseplanningintostreamlinedreleaseevents

44© 2016 IBM Corporation

Codereviewshouldbepartofyourversioningprocess

§ Defineandenforcecodingstandardsearly intheprocess

§ Automatedeployment toproduction-like systems aspartofautomatic verification

§ Eachcommitshould runautomated testingandcodecoverage,andthetoolchainshouldenforcecompliancebeforemergingorpromotingchanges

45© 2016 IBM Corporation

§ Architecture

§ AutomatedTesting

§ AutomatedDeployments

§ Production-LikeEnvironments

§ AutomatedReleasePromotionProcess

§ Versioncontrolofallsoftware,automation,andconfiguration

Aboutthisdeck

mdelder@us.ibm.com http://linkedin.com/in/mdelder @mdelder

© 2016 IBM Corporation

Clouds&Containers

Workload styles vary, but not release methodologyTraditional IT Cloud-Enabled Cloud-Native

App Servers

+ +

DatabaseWeb Servers

Applications

App Servers

+ +

DatabaseWeb Servers

Applications ServicesApplicationsServices

Managed Runtimes

Traditional IT Cloud-Enabled IT Cloud-Native ITContinuous IntegrationContinuous delivery to

productionCloud-native runtimes (node.js)

Pipeline per microservice

Change Record

CMDB

Automated build / deploy of VMs & Containers

Stage Gates, Co-ordinated Releases

Manual Change & ReleaseSome automation

Audit reports

CABAssessment

&Approval Theory

Reality

IBM Control DeskBMC Remedy

As a Business Owner, I want my team to deliver as fast as possible, but with appropriate controls when needed.

But don’t slow everything down to the slowest speed when controls are added!

Change Management vs. Continuous Delivery

Accelerate Innovation

ivDeliverCodeThink Run

Systems of engagement

Systems of Record

Enterprise Applications

Production

Most enterprises need a combination of DevOps patterns

49

API Catalog

Operational Excellence

ivDeliverCodeThink Run Production

Shared Services

Digital Applications

§ Coordinating differentteamsworkingatdifferentspeeds

§ Balancing between agility,stability, andrisk

§ Decouple acrossAPIboundary,butmanagerelease processwithconsistent approach

Cloud-Enabled Workloads with UrbanCode Deploy

50

§ Yourapplications aremanagedthroughout theentirerelease processwithpredictable automation andinventoryservices

§ Yourfull-stack blueprints capturetherelationships between yourapplications andcloud infrastructure

§ OpenStackAPIprovidesdefactostandards forinteractingwithallofyourcloudresources

Heat Blueprints

UrbanCodeDeploy

Compute Network Storage

Your Applications & Middleware

Full-stack Application Pattern

51Innovate. Disrupt. Transform. Fast. @Enterprise Scale.Hybrid Cloud Deployments through a Single Point of Control

IBM Bluemix

Cloud Foundry

On-prem IaaS

On-prem Traditional IT

IBMCloudOrchestratorIBMPureApplicationSystem

Manageapplicationdeploymentacrossdev,test,andproductionspanningmultipleclouds

KeyPoints:• Enablefullstackdeployments(OS,patternsand

applications)acrosshybridcloudapplications• EstablishcommonDevOpspipelineswithplugpointsto

supportcontinuous delivery

Virtual Machines

DockerContainers

Other VendorsUrbanCodeDeploy

UrbanCode Release

https://www.youtube.com/watch?v=fFuRg6l3N44

SCREENCAP: Designer & Palette

Watson Health: 6 active teams, more on the way§ SocialCare(AbuDhabi):

– Complex topology, heavy UCD components (2 Gb)– 2 hours roundtrip / provision– 50% Heat engine CPU capacity / provision

§ RealWorldEvidence:

– 4 Offerings

– Analytics workload (DB2 blue, Cognos)§ AnalyticsCore:

– Analytics workload (Spark clusters)– UCDP Rest API driven variable sized cluster size

§ OperationalDashboard:

– Cloud control desk for solution management (provision, monitor, troubleshoot)

§ WatsonHealthCloud:

– Foundational micro-services (data lake, data pipeline, logging, monitoring etc)

53

Abu Dhabi Blueprint

54© 2016 IBM Corporation

Docker is great for developers – everything can be run on your laptop

54

55© 2016 IBM Corporation

Production-LikeEnvironments– LocalDevelopment

PossiblyoneofthemostimportantprinciplesofDevOpsShiftleftanyoperationalconcernsthatleadto

differencesfromDEVtoPROD

https://hub.docker.com/u/ibmcom/#

Continuous Integration for Containers

56

Jenkins

IBM UrbanCode Deploy

Docker TrustedRegistry

Build Image

TrackVersions

Deploy &Promote

§ Jenkins builds Docker images andpublishes them toDocker TrustedRegistry

§ UrbanCodeDeploy immediately becomesawareofnewversions, available fordeployment

§ Schedule deployments, trackreadinessviaenvironment gates,andmanageconfiguration properties securely

§ Deployments toDocker Universal ControlPlane

Docker Integration Architecture

Docker TrustedRegistry

IBM UrbanCode Deploy

swarm

DockerUniversalControlPlane

LegacyDatabase

Legacy Applications

Use UrbanCode Deploy to manage:

Inventory Management Promotion Lifecycle

Separation of Duties Import images from trusted sources

Docker TrustedRegistry

IBM UrbanCode DeployDocker

UniversalControlPlane

LegacyDatabase

Legacy Applications

UrbanCode Deploy & Compose

§ Importyourdocker-compose file

§ Automatically createresourcetreecomponents

§ Updates, Inventorymanagement, Deployments, etc.areallthesame.

59

Dev/Test with Hybrid Cloud

Off-

Prem

On-

Prem

Image Registry

PRODSTAGING

Delivery Pipeline

IBM Bluemixhttps://www.youtube.com/watch?v=Eo1rFsv4BW0

IBM UrbanCode Deploy

© 2016 IBM Corporation

Q&A

Get this deckhttp://bit.ly/2c1G8sy

62© 2016 IBM Corporation62

©Copyright IBMCorporation 2013. All rights reserved. The information contained in these materials is provided for informational purposes only, and isprovided AS ISwithout warranty ofanykind, express orimplied. IBMshall notbe responsible for anydamages arising outof theuse of, orotherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have theeffectof, creatinganywarranties or representations from IBMor its suppliers or licensors, oraltering the terms andconditions of theapplicable license agreement governing theuse of IBMsoftware. References in these materialsto IBMproducts, programs, or services donot imply that theywill beavailable inall countries inwhich IBMoperates. Product release dates and/or capabilities referenced in these materials maychange atanytime atIBM’s sole discretion based onmarketopportunities orother factors, and arenot intended tobe acommitment to future product or feature availability inanyway. IBM, the IBM logo, Rational, theRational logo, Telelogic, theTelelogic logo, andother IBMproducts andservices are trademarks of the International Business Machines Corporation, in the United States, other countries or both.Other company,product, or service names maybe trademarks or service marks ofothers.

www.ibm.com/software