UrbanCode Deploy DevOps Best Practices

© 2016 IBM Corporation

UrbanCode Deploy DevOps Best Practices MichaelD.Elder

IBMDistinguished EngineerChiefDeveloper, UrbanCode

[email protected]@mdelder

Get this deckhttp://bit.ly/2c1G8sy

2© 2016 IBM Corporation

PleaseNote

2

• IBM’sstatementsregardingitsplans,directions,andintentaresubjecttochangeorwithdrawalwithoutnoticeatIBM’ssolediscretion.

• Informationregardingpotentialfutureproductsisintendedtooutlineourgeneralproductdirectionanditshouldnotbereliedoninmakingapurchasingdecision.

• Theinformationmentionedregardingpotentialfutureproductsisnotacommitment,promise,orlegalobligationtodeliveranymaterial,codeorfunctionality.Informationaboutpotentialfutureproductsmaynotbeincorporatedintoanycontract.

• Thedevelopment, release, andtimingofanyfuturefeaturesorfunctionalitydescribed forourproductsremainsatoursolediscretion.

• Performance isbasedonmeasurementsandprojectionsusingstandardIBMbenchmarksinacontrolledenvironment.The actualthroughputorperformance thatanyuserwillexperience willvarydependinguponmanyfactors,includingconsiderationssuchastheamountofmultiprogrammingintheuser’sjobstream,theI/Oconfiguration,thestorageconfiguration,andtheworkloadprocessed. Therefore, noassurancecanbegiventhatanindividualuserwillachieveresultssimilartothosestatedhere.


•UnderstandingDevOps

•DevOpsBestPractices

• Clouds&Containers

Agenda

IBMConfidential


UnderstandingDevOps


Failures duetoinconsistentdev and

productionenvironments

Bottlenecks tryingtodelivermorefrequentreleases tomeetmarketdemands

Complex,manual,processesforreleaselackrepeatabilityand

speed

Poorvisibilityintodependencies across

releases, resources,andteams

Softwaredeliverychallenges

DailyBuild

MonthlyDelivery

Whodidthis lasttime?

Dave…

Dave’snothereman…

Dev

Prod


HighperformingteamsadoptDevOps

Reference: 2013 State of DevOps Report by PuppetLabs


Intuitive and Scalable Model Driven Deployment

Composite Applications

Components

Re-usable Workflows Environment ManagementSIT

PROD

The “What” The “How” The “Where”

DEPLOYMENT AUTOMATION


Intuitive and Scalable Model Driven Deployment

Composite Applications

Components:Container Images

Re-usable Workflows Environment ManagementSIT

PROD

The “What” The “How” The “Where”

DEPLOYMENT AUTOMATION

$ docker run .. {image}:{version}


Aboutphilosophy

Cultureofcontinualexperimentationand

learning

• Productionlikeenvironments• Fullyautomateddeployments• Accelerateddelivery cycles

Build– Measure- Learn

“Successisnotdeliveringafeature; successislearninghowtosolvethecustomer’sproblem.”

10© 2016 IBM Corporation 10

IBMBluemix GarageMethod

PracticesProcesses or actions you & your team perform to achieve an outcome.

TracksA selected set of practices, tools, and toolchains to create a cloud native or cloud enabled solution

ToolchainComplementary & compatible tools that when used together create a flexible solution

Experts & ExperiencesListen to our Experts to learn how we transformed our business

Combines industry bestpracticesforDesignThinking, LeanStartup,AgileDevelopment,DevOps,and Cloudtobuildanddeliverinnovative solutions.

https://www.ibm.com/devops/method

11© 2016 IBM Corporation © IBM Corporation

Newcontentaddedweekly


Accelerate Innovation

ivDeliverCodeThink Run

Systems of engagement

Systems of Record

Enterprise Applications

Production

MostenterprisesneedacombinationofDevOpspatterns

12

API Catalog

Operational Excellence

ivDeliverCodeThink Run Production

Shared Services

Digital Applications

• Coordinatingdifferentteamsworkingatdifferentspeeds

• Balancing between agility,stability, andrisk

• DecoupleacrossAPIboundary,butmanagerelease processwithconsistent approach


ExpectationsofContinuousDeliveryUser Experience Today Emerging

Primary Workload Type Software supports core business function

Software enables innovative business approaches to differentiate from competitionBig Data, Analytics, Mobile/Social Channels

Delivery Model Planned Incremental (DevOps)

Development and Operations Team Sizes 100s and costly 10s with built-in DevOps automation

Release Frequency Months to years Days to weeks, based on business opportunity

Integration Frequency Weeks Continuous

Infrastructure Deployment Days Minutes

Time to Value Planned Opportunistic

Operational Model Systems management Built into application, recovery-oriented, continuous availability

Service Sourcing Develop Consume and assemble (public and private)

Feedback Loop Periodic Continuous interaction with clients


Increasetheoperationalawarenessofyoursoftware earlierinthedevelopmentprocess.

WhatdoesDevOpsmeantome?

Howdoyoudothat?§ Architecture

§ AutomatedTesting

§ AutomatedDeployments

§ Production-LikeEnvironments

§ AutomatedReleasePromotionProcess

§ Versioncontrolofallsoftware,automation,andconfigurationWhydothis?

Becausefasterfeedbackloopsenablerapidevolution ofideasandthereforefasteriterationsofyoursoftware

ShiftLeft


Bettercommunication betweenthosewho createandthosewhooperate(samepeopleinsomecases)

Reducedfearofbreakingthebuild/deployment/environment

Failfastbeforeyou“failall”

Heavyfocusonexperimentationandlearning

Improvethespeedofyourfeedbacklooptoenablerapidevolutionofideas

WhatarethecharacteristicsofteamswhopracticeDevOps?


DevOpsBestPractices


§ Architecture

§ AutomatedTesting




§ Versioncontrolofallsoftware,automation,andconfiguration

Aboutournexttopics


IBMUrbanCode forContinuousDelivery

Drive down costReduce amount of manual labor, resource wait-time, and rework by eliminating errors & providing self-service environments

Speed time to marketIncrease frequency of software delivery through automated, repeatable deployment processes across development, test and production

Reduce riskRobust configuration management, coordinated release processes, audits, and traceability

Enabling clients to continuously deliver applications with high quality and low risk

IBM UrbanCode Build is a continuous integration and build management server optimized for the enterprise, designed to make it easy to scale the configuration and management of build infrastructure and seamlessly plug in to development, testing and release tooling.

IBM UrbanCode Deploy enables hybrid cloud by automating the deployment of applications, databases, configurations and containers into development, test and production on prem or in the cloud, helping to drive down cost, speed time to market with reduced risk. With full-stack environment management and deployment automation, users to design, deploy and update full-stack environments for multiple clouds, including Bluemix, Softlayer, AWS, Azure, and virtualized environments.

IBM UrbanCode Release is an intelligent collaboration release management solution that replaces error-prone manual spreadsheets and streamlines release activities for application and infrastructure changes.


Aboutyourarchitecture

• ArchitectureshouldsupportDevOps principlessuchasstagedrollout,operationalinsights,andscriptability

• Eachresourceprovidessomeverypracticaladviceforbuildingsystemswhicharefocusedonreliabilityandfeedbackloops

Experiment!:Websiteconversionrate

optimizationwithA/Bandmultivariatetesting

ReleaseIt!:DesignandDeployProduction-Ready

Software


Applicationmodel

Reliablycaptureapplicationcontentandconfigurationsettingswithversioningandtraceability


Applicationenvironments

Definewherecomponentsaredeployedandcaptureconfigurationsettingsperdeploymentenvironmentforanapplication


Templatingforgoodarchitecture

Templatescanbecreatedtostandardizecomponentprocesses&config


Createapplicationtemplatesforacceleratedonboarding

Templatescanbecreatedtostandardizecomponentprocesses&config


Automatedtestingiscriticalforautomatedfeedback

• This isTHEHARDESTpartofthewholeexperience

• NOONEisgreatatthis,sodon’tgetdiscouraged

• Facebook’s processforcodereview include statsonhowmanytests areconsistently brokentodecidewhether thetestfailuresarenoteworthy

• Continuously improveyourcoverage, flexibility, anddiscipline

• Consider allaspects oftesting:

• ProgrammaticAPI(JUnit)

• RESTAPI(HTTPUnit, RESTFuse)

• WebUI(Selenium)

• Behaviordriventesting (Cucumber)

• Performancetesting (ApacheJMeter)

• Securitytesting (RationalAppScan &IBMQRadar)


Embeddingunittestsinyourclassesdirectly

• PromotedapproachbyNetflixtoreducefrictionandintroduces limited additionalbytes relativetothirdpartylibraries

• Makes iteasytowritecommands,theUnitTestbecomes thetestharnessandverification

• AlwaystestSuccess() andtestFailure() usecases toensureexpected behavior

importstaticorg.junit.Assert.assertEquals;importstaticorg.junit.Assert.assertNotNull;

publicclassOSDeployEnvironmentCommand extendsOSAbstractOrchestrationCommand<Response>{

publicstaticclassUnitTest {@TestpublicvoidtestSuccess() throwsInterruptedException,

ExecutionException,JSONException {…Future<Response>request=

newOSDeployEnvironmentCommand(…).queue();

Responseresponse =request.get();assertNotNull(response);assertEquals(HttpStatus.SC_OK,

response.getStatus());}

}


TestingYourDeployedApplication– RESTAPI

• WehaveusedRESTFusewhich isanEclipse project

• Captures RESTcallsasannotations, freeingupyourtestmethodtojustverifybehavior

• Requires alittle“hoopjumping”fortesting aseries ofcalls (e.g.POST..PUT..GET)

importstaticcom.eclipsesource.restfuse.Assert.assertCreated;import...restfuse.annotation.Authentication;import...restfuse.annotation.HttpTest;import...restfuse.MediaType;

@HttpTest (method=com.eclipsesource.restfuse.Method.POST, contentMethod="getBlueprintRequestBody",authentications={@Authentication(type=BASIC,user="{keystoneUser}",password="{keystonePassword}")},path="/rest/blueprint",type=MediaType.APPLICATION_JSON

)publicvoidtestCreateBlueprintOK()throwsException{assertCreated(response);}


BehaviorDrivenDevelopment&Testing

• Capturesnatural languagestatements using“Given”,“When”,“Then”clauses

• Eachline ismatched toasnippet ofcodebasedonregularexpression

• Complete codestatementsareexecuted tovalidatebehavior

• Bridgebetween businessgoalsandtechnical details

Scenario:Success– UploadclientdirectoryGivenIprovidevalidcredentialstotheapi clientAndIwanttousetheapi clienttomakerequestsWhenIuploadthedirectory”data/sample-app”withtheclientThenthefilesreturnedmatchestheuploaddirectory”data/sample-app"


TestingYourDeployedApplication– WebUI

• Selenium istheonly“apparent”choicethatwe’vecomeacrossthat’seffective

• Couples automated testtosomedetails ofyourwebUIandmustbekept insync

• We’vedeveloped afewbestpractices aroundlocationelements formaximumflexibility, couldpublishthisifthere’sinterest

publicclassEditorsPage extendsUCDPPage {…@FindBy(id="new-blueprint-button")privateWebElement newButton;…publicvoidcreateFile(String name,Stringid,String[]values){newButton.click();util().waitForElementPresentAndVisible(By.className("new-file-dialog"));WebElement widget=util().findElement(By.className("new-file-dialog"));

Dialog dialog=Widget.init(this, widget,Dialog.class);dialog.setValues(values, true,true);dialog.clickPrimaryButton();

…}…}


Automateddeployments• Visibility andautomatedcontrolofyourapplication deployment process

• Manageapplicationcomponents andversions

• Manageconfigurationsacrossallenvironments

• Offersecure‘self-service’capabilities

• Increasetransparency• Ensuregovernance andcompliancy


Defineautomatedprocesses foreachapplicationcomponentEachcomponentdefinesaseriesofsteps,backedby

plugintasksorcustomscripts,todeployit


Defineautomatedprocesses fortheapplicationOrdercomponentsorapplylogicwhichis“outside”of

anyspecificcomponent


IBMUrbanCodeDeploy– Plug-ins

Over 150 – in over 50 categories


Production-LikeEnvironments– UnifiedResourceTree

Provideslogicalhierarchyofdeployedartifactsandtheirmiddlewarecontainers

DEVResources- WebHost- WebComponent- Database- DatabaseComponent

QAResources- WebHost1- WebComponent- WebHost2- WebComponent- Database- DatabaseComponent

PRODResources- WebHost1- WebComponent- WebHost2- WebComponent- WebHost3- WebComponent- WebHost4- WebComponent- DatabasePrimary- DatabaseComponent- DatabaseSecondary- DatabaseComponent


Production-LikeEnvironments– LocalDevelopment

PossiblyoneofthemostimportantprinciplesofDevOpsShiftleftanyoperationalconcernsthatleadto

differencesfromDEVtoPROD

https://hub.docker.com/u/ibmcom/#


DeploymentofApplicationsacrossEnvironmentsApplicationsaremanagedfordeployment

Componentsfromtheapp

SpecificversionsoftheComponents’artifactsaredeployed

Compliancykeepstrackofwhethertheactualversionsmatchestheexpected

EachComponentknowsitsrelevantdeploymentprocesses

CompareversionsacrossenvironmentsanddefineSnapshotsofknownworkingsetsofComponents

Viewdeploymentrequestsagainsttheenvironment

https://developer.ibm.com/urbancode/AutomatedReleasePromotionProcess


PromotionLifecycles

• Shouldfocusonconsistencyfromendtoend

• Usethesamelogicalorganizationandcomponentautomation inDEV,QA,PRODtovalidatetheautomationearly inthedelivery cycle

• Applyapprovalgateswherenecessary toensureappropriatequalityvalidationhasoccurred

IBMCommerce

WebSphere ND

Static Content

Commerce Data

IBMCommerce

WebSphere ND

Static Content

Commerce Data


QualityAssurance– Automatedgates

• UseComponent Statuses andEnvironmentGateswherepossible

• Definestatuses forallApps

• UseRESTAPI/CLItotag/approveComponents orSnapshots basedonautomated verification(CIorothertesting)

• Onlycomponentsversions/snapshots withtagcanenterenvironment


QualityAssurance– ManualSignoff

• Manualapprovalsarepossible, butrecommendthatasmuchaspossible isautomated (e.g.withstatuses+gates)

• Approvalprocesscantriggeremails orsome clients use”chatops”tools like Slacktostreamline theprocess


Inventoryovertime


ConfigurationManagement- SnapshotsCreating a Snapshot

Component Versions

1

1

2

2

3

3

321Web

Mid. Code

DB

Snapshot

3

2

1

Mid. Config 1 2 3 3

Snapshot Deployment

ENVSnapshot

3

2

1

WEB HOST

MID HOST

DB HOST

2

1

1

3

2

X

3 3 1

Contents ofenvironments thatpasstests• containdeployable versionandconfiguration

Deployments arebasedondeltasTheyprovide Automation,Audit,andVisibility


VersionapplicationswithSnapshots

Capturetestedcomponentversionsandconfigurationsettingsasasnapshotthatispromotedtootherenvironmentsforbettervisibilityandcontrol


Better Release Planning and Management

• Enterprise Calendaring

• Release Process Checklist Templates

• Environment Reservation

Effective Change and Risk Management

• Rational Team Concert integration

• Impact Analysis

Continuous Delivery with Auto-Progression

• IBM UrbanCode Deploy integration

Increased Visibility and Control

• Pipeline View

• Federated Dashboard

• Segment Dependency Graph

UrbanCodeReleaseTransformchaoticreleaseplanningintostreamlinedreleaseevents


Codereviewshouldbepartofyourversioningprocess

§ Defineandenforcecodingstandardsearly intheprocess

§ Automatedeployment toproduction-like systems aspartofautomatic verification

§ Eachcommitshould runautomated testingandcodecoverage,andthetoolchainshouldenforcecompliancebeforemergingorpromotingchanges


§ Architecture

§ AutomatedTesting




§ Versioncontrolofallsoftware,automation,andconfiguration

Aboutthisdeck

[email protected] http://linkedin.com/in/mdelder @mdelder


Clouds&Containers

Workload styles vary, but not release methodologyTraditional IT Cloud-Enabled Cloud-Native

App Servers

+ +

DatabaseWeb Servers

Applications

App Servers

+ +

DatabaseWeb Servers

Applications ServicesApplicationsServices

Managed Runtimes

Traditional IT Cloud-Enabled IT Cloud-Native ITContinuous IntegrationContinuous delivery to

productionCloud-native runtimes (node.js)

Pipeline per microservice

Change Record

CMDB

Automated build / deploy of VMs & Containers

Stage Gates, Co-ordinated Releases

Manual Change & ReleaseSome automation

Audit reports

CABAssessment

&Approval Theory

Reality

IBM Control DeskBMC Remedy

As a Business Owner, I want my team to deliver as fast as possible, but with appropriate controls when needed.

But don’t slow everything down to the slowest speed when controls are added!

Change Management vs. Continuous Delivery

Accelerate Innovation

ivDeliverCodeThink Run

Systems of engagement

Systems of Record

Enterprise Applications

Production

Most enterprises need a combination of DevOps patterns

49

API Catalog

Operational Excellence

ivDeliverCodeThink Run Production

Shared Services

Digital Applications

§ Coordinating differentteamsworkingatdifferentspeeds

§ Balancing between agility,stability, andrisk

§ Decouple acrossAPIboundary,butmanagerelease processwithconsistent approach

Cloud-Enabled Workloads with UrbanCode Deploy

50

§ Yourapplications aremanagedthroughout theentirerelease processwithpredictable automation andinventoryservices

§ Yourfull-stack blueprints capturetherelationships between yourapplications andcloud infrastructure

§ OpenStackAPIprovidesdefactostandards forinteractingwithallofyourcloudresources

Heat Blueprints

UrbanCodeDeploy

Compute Network Storage

Your Applications & Middleware

Full-stack Application Pattern

51Innovate. Disrupt. Transform. Fast. @Enterprise Scale.Hybrid Cloud Deployments through a Single Point of Control

IBM Bluemix

Cloud Foundry

On-prem IaaS

On-prem Traditional IT

IBMCloudOrchestratorIBMPureApplicationSystem

Manageapplicationdeploymentacrossdev,test,andproductionspanningmultipleclouds

KeyPoints:• Enablefullstackdeployments(OS,patternsand

applications)acrosshybridcloudapplications• EstablishcommonDevOpspipelineswithplugpointsto

supportcontinuous delivery

Virtual Machines

DockerContainers

Other VendorsUrbanCodeDeploy

UrbanCode Release

https://www.youtube.com/watch?v=fFuRg6l3N44

SCREENCAP: Designer & Palette

Watson Health: 6 active teams, more on the way§ SocialCare(AbuDhabi):

– Complex topology, heavy UCD components (2 Gb)– 2 hours roundtrip / provision– 50% Heat engine CPU capacity / provision

§ RealWorldEvidence:

– 4 Offerings

– Analytics workload (DB2 blue, Cognos)§ AnalyticsCore:

– Analytics workload (Spark clusters)– UCDP Rest API driven variable sized cluster size

§ OperationalDashboard:

– Cloud control desk for solution management (provision, monitor, troubleshoot)

§ WatsonHealthCloud:

– Foundational micro-services (data lake, data pipeline, logging, monitoring etc)

53

Abu Dhabi Blueprint


Docker is great for developers – everything can be run on your laptop

54


Production-LikeEnvironments– LocalDevelopment

PossiblyoneofthemostimportantprinciplesofDevOpsShiftleftanyoperationalconcernsthatleadto

differencesfromDEVtoPROD

https://hub.docker.com/u/ibmcom/#

Continuous Integration for Containers

56

Jenkins

IBM UrbanCode Deploy

Docker TrustedRegistry

Build Image

TrackVersions

Deploy &Promote

§ Jenkins builds Docker images andpublishes them toDocker TrustedRegistry

§ UrbanCodeDeploy immediately becomesawareofnewversions, available fordeployment

§ Schedule deployments, trackreadinessviaenvironment gates,andmanageconfiguration properties securely

§ Deployments toDocker Universal ControlPlane

Docker Integration Architecture



swarm

DockerUniversalControlPlane

LegacyDatabase

Legacy Applications

Use UrbanCode Deploy to manage:

Inventory Management Promotion Lifecycle

Separation of Duties Import images from trusted sources


IBM UrbanCode DeployDocker

UniversalControlPlane

LegacyDatabase

Legacy Applications

UrbanCode Deploy & Compose

§ Importyourdocker-compose file

§ Automatically createresourcetreecomponents

§ Updates, Inventorymanagement, Deployments, etc.areallthesame.

59

Dev/Test with Hybrid Cloud

Off-

Prem

On-

Prem

Image Registry

PRODSTAGING

Delivery Pipeline

IBM Bluemixhttps://www.youtube.com/watch?v=Eo1rFsv4BW0



Q&A

Get this deckhttp://bit.ly/2c1G8sy

62© 2016 IBM Corporation62

©Copyright IBMCorporation 2013. All rights reserved. The information contained in these materials is provided for informational purposes only, and isprovided AS ISwithout warranty ofanykind, express orimplied. IBMshall notbe responsible for anydamages arising outof theuse of, orotherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have theeffectof, creatinganywarranties or representations from IBMor its suppliers or licensors, oraltering the terms andconditions of theapplicable license agreement governing theuse of IBMsoftware. References in these materialsto IBMproducts, programs, or services donot imply that theywill beavailable inall countries inwhich IBMoperates. Product release dates and/or capabilities referenced in these materials maychange atanytime atIBM’s sole discretion based onmarketopportunities orother factors, and arenot intended tobe acommitment to future product or feature availability inanyway. IBM, the IBM logo, Rational, theRational logo, Telelogic, theTelelogic logo, andother IBMproducts andservices are trademarks of the International Business Machines Corporation, in the United States, other countries or both.Other company,product, or service names maybe trademarks or service marks ofothers.

www.ibm.com/software

UrbanCode Deploy DevOps Best Practices

Software

Transcript of UrbanCode Deploy DevOps Best Practices