DevOps and Continuous Testing€¦ · · 2018-03-21time it takes to release new features or...

DevOps andContinuous Testing

Mark Kachik, Micro Focus Lead Solution Consultant

Shannon Lake, B12, VP, Solutions and Strategy

Robert Linton, B12, VP, Applications Delivery

March 8th, 2018

Modern Software Lifecycle Disruptions

Overview End-to-End DevOps

Approaches to DevOps

Tools Discussion

Example Customer User Cases

Questions

2

Agenda

Modern software lifecycle disruptions

DevOps AdoptionChanges everything

Software ComplexityGrowing to the point of chaos

Cloud TransformationShift to cloud / utility models

Waterfall Iterative Agile DevOps

+

Client-ServerMonolithic N-Tier Microservices + APIs

Open SystemsMainframe Virtualization/Cloud Cloud Native

MET

HO

DO

LOG

YA

PP

LIC

ATI

ON

SIN

FRA

STR

UC

TUR

E

ORGANIZATIONShift from IT centric to LOB projects with self-service

REQUIREMENTSShift from elicitation and sign-off to dynamic backlog

QUALITYShift from quality COEs to continuous quality in every part of the lifecycle

DEPLOYMENTShift from release trains to DevOps deployment automation

TECHNOLOGYShift from enterprise architecture to developer dominance

CONTROL POINT SHIFTS

Trends do not guarantee business outcomes

0%

29%

52%

5%

12%

4%

29%

40%

15% 14%

0%

10%

20%

30%

40%

50%

60%

Very satisfied Somewhatsatisfied

Somewhatdissatisfied

Verydissatisfied

Don't know /no

measurement

How satisfied do you think your business is with the time it takes to release new features or changes to customer-facing business services or applications?

2014 2016

https://www.forrester.com/report/Six+Trends+That+Will+Shape+DevOps+Adoption+In+2016+And+Beyond/-/E-RES120481

87% are moving toward a DevOps process

Source: Dimensional Research – It Is Flying Blind In Hybrid Clouds – April 2016

86% are considering containers

85% will have a continuous delivery process

Customer

value?

Business

demandsPlan Develop Test Release

Is your entire value chain agile?

Deploy Operate

Manual testing & # number of defects

Rapidly increasing WIP

Manual build and integration processes

Insufficient test data for decision

making

Outages from undetected defects

Outages due to change related

incidents

Lack of timely customer insight

7 days 2 days 15 days 10 days1 day

Cycle Time = 70 daysLong wait time for build/test environments Manual and error prone app deploymentsIT and business release cycles misaligned

7 days12 days

Overview End-to-End Enterprise DevOps Solution

Enterprise DevOps

7

Continuousplanning

Plan

Define

Continuousintegrationand testing

Dev & Build

Test

Continuousdeployment and

release

Release

Deploy

Continuousoperations

Operate

Monitor

Continuous Security

Continuous assessment

DemandCustomer Experienc

e

Continuous Collaboration

Micro Focus Enterprise DevOps Operating Model: your transformation blueprint

Leadership

• Lean-Agile leaders as a baseline• Lead the change• Enable collaboration & motivate• Decentralize decision making

Strategy & Goals

• Implement strategy aligned to customer centric value delivery in tune with business strategy

• Align vision, roadmap, portfolio, budgets & metricsto value streams

Culture

•

• Accepting DevOps into culture & mindset• Management of organizational change for

individuals, teams and the organization

Learning & Information

• Enable for agile, lean & DevOps thinking

• Manage & coach adoption • Foster knowledge with communities

ofpractice, pairing roles & piloting

Processes

• Align to support agile, waterfall & outsourcing

•

• Standardize using best practices and align

to IT4IT value streams

Organization & Structure

• Focused on delivering value• De-silo: Foster collaboration &

strong relationships• Optimize with Scaled Agile

Framework

Mission & Vision

LeadershipLearning & Information

Strategy & Goals

PerformanceMeasures &

Rewards

Process

Systems

OrganizationalStructure

CultureSystems

•

industry standard software and open source• Automate & integrate process & people

bottlenecks• Span across the DevOps pipeline

Performance, Measures & Rewards

• KPIs & measures aligned to customer value• Measure DevOps processes across quality,

velocity, productivity & security• Reward teams and the organization

The art of the possibleEnabling transformation through end-state visualisation

Fully functional, reference implementation environment, with pre-built integrations across

Micro Focus portfolio and open source tools, showcasing key, common, real-life use cases

based on IT4IT™ and SAFe® industry standardsMicro FocusModel Office

Micro Focus DevOps Model Office Blueprint

Continuous Operations

Scale up/down Auto Failover Service Mgmt App Perf.

Monitoring Detect2Correct Secure Access Secure Data Secure App

Software Tools: Docker & K8S UCMDB Service Manager SiteScope AppPulse BSM Voltage & SDM NetIQ Fortify

AppDefender

Continuous Security

Security Requirement/Design ALM Octane & ALM.Net

Security Static Code Analysis Fortify SCA/SSC & Jenkins & IDE & Octane

Security Dynamic Testing Fortify WebInspect/SSC & ALM & Octane & SDM

Security Monitoring/Protection Fortify AppDefender, ArcSight, Voltage

Continuous Testing

Shift-Left LeanFT Functional Test Performance Test Security Test Service Virtualization Network Virtualization Mobile Testing Test Data Management Business Process Test

Software Tools: UFT/LeanFT Performance Center Service Virtualization Network Virtualization Mobile Center SDM & Secure Data Fortify SCA &WebInspect Fortify SSC & ALM & OctaneTesters

Continuous Planning

Demand Management Proposal Management Portfolio Management Program Management Time Management Resource Management Cost Management Req/Defect Management Agile Release Management Agile Time Management Architecture Design Security Design

Project Managers

Business Analysts

Architects

ProgramManagers

Software Tools: PPM ALM Octane ALM.Net 3rd party Agile Tools

Continuous Integration

Developers

Dev Testers

System Team

Web/Mobile/Backend Dev Req/Code Linkage Defect/Code Linkage Code Coverage Shift-Left Testing Container Image Build Build & Packaging Security Static Code Scan Gated Check-in Version Control on Src Version Control on Exec. Status Synchronization

Software Tools: IntelliJ, Eclipse, VS, Borland ALM Octane & Plugins LeanFT, SV, Sprinter Fortify & SSC Maven, Gradle, GITLab Nexus, Jenkins, verigreen Docker, Docker Registry

SIT QAS STG PRDTIS

Continuous Deployment & Release

Application Release Automation Hybrid Cloud based Deployment Docker-based Deployment Mobile App Deployment Configuration Integration IT Service Management

Software Tools: Jenkins & Serena & Chef/OO HCM – Premium K8S / Docker EE / Mesos Mobile Center UCMDB & Service Manager


PPMBot

ALM OctaneBot

FortifyBot

JenkinsBot

GITLabBot

SitescopeBot

OOBot

SMBot

Software Tools MatterMost, Microsoft Teams, Slack

Continuous Assessment

Software Tools: Vertica, postgresql, IDOL, ETL, PowerBI

Velocity-focusedKPIs

Productivity-focusedKPIs

Security-focusedKPIs

Quality-focusedKPIs

Micro Focus DevOps Model Office Blueprint

Continuous Operations

Scale up/down Auto Failover Service Mgmt App Perf.

Monitoring Detect2Correct Secure Access Secure Data Secure App

Software Tools: Docker & K8S UCMDB Service Manager SiteScope AppPulse BSM Voltage & SDM NetIQ Fortify

AppDefender

Continuous Security

Security Requirement/Design ALM Octane & ALM.Net

Security Static Code Analysis Fortify SCA/SSC & Jenkins & IDE & Octane

Security Dynamic Testing Fortify WebInspect/SSC & ALM & Octane & SDM

Security Monitoring/Protection Fortify AppDefender, ArcSight, Voltage

Continuous Testing


Software Tools: UFT/LeanFT Performance Center Service Virtualization Network Virtualization Mobile Center SDM & Secure Data Fortify SCA &WebInspect Fortify SSC & ALM & OctaneTesters

Continuous Planning

Demand Management Proposal Management Portfolio Management Program Management Time Management Resource Management Cost Management Req/Defect Management Agile Release Management Agile Time Management Architecture Design Security Design

Project Managers

Business Analysts

Architects

ProgramManagers

Software Tools: PPM ALM Octane ALM.Net 3rd party Agile Tools


Developers

Dev Testers

System Team


Software Tools: IntelliJ, Eclipse, VS, Borland ALM Octane & Plugins LeanFT, SV, Sprinter Fortify & SSC Maven, Gradle, GITLab Nexus, Jenkins, verigreen Docker, Docker Registry

SIT QA STG ProdTIS

Continuous Deployment & Release

Application Release Automation Hybrid Cloud based Deployment Docker-based Deployment Mobile App Deployment Configuration Integration IT Service Management

Software Tools: Jenkins & Serena & Chef/OO HCM – Premium K8S / Docker EE / Mesos Mobile Center UCMDB & Service Manager


PPMBot

ALM OctaneBot

FortifyBot

JenkinsBot

GITLabBot

SitescopeBot

OOBot

SMBot

Software Tools MatterMost, Microsoft Teams, Slack

Continuous Assessment

Software Tools: Vertica, postgresql, IDOL, ETL, PowerBI

Velocity-focusedKPIs

Productivity-focusedKPIs

Security-focusedKPIs

Quality-focusedKPIs

Continuous Integration and Testing



Software Tools: IntelliJ, Eclipse, VS ALM Octane LeanFT, SV, Sprinter Fortify & SSC Maven, Gradle, GITLab Nexus, Jenkins, verigreen Docker, Docker Registry

Continuous Testing


Software Tools: UFT/LeanFT Performance Center Service Virtualization Network Virtualization Mobile Center IM&G SDM Fortify SCA &WebInspect Fortify SSC & ALM

GatherResults &Feedback

Code

StaticAnalysisUnit Test

PrivateBuild

Check-In

IntegratedBuild

RunIntegrated

Test

Deployto Test

AutomatedTest

Continuousintegration

Continuoustesting

VirtualizeServices

VirtualizeData

Run System &End to End Tests

RunNon-Functional

Tests

ManageDefects

DefineTests

ContinuousPlanning &

DesignDefineStories

ElaborateStories

DefineFeatures

ContinuousDeployment

& Release

Source CodeRepository Build & deploy

code

Run smoketests

12

Features of continuous integration & testingContinuous integration

Use a single integratedcode repository

Automated, fast, self-testingbuild & deploy

Static analysis of codeincluding security scan

1st

Define tests first beforecode is written

Stories, reqs, code, tests, builds & defects are all linked

Developers pair up withother developers or testers

Automated tests occur inproduction-like environments

Planning, design, coding, testing& deployment are integrated

Continuous testing

Shift all types of testing left

Manage all test scriptin a common repository

Automate testing early,as much as possible

Virtualize service & networks

User targeted change impactregression testing

Automate, simulate &create test data

Apply dynamic app security &performance testing early

Implement mobile & IoTtest automation

Continuous testing flow

13

Dev UnitTest

Deploy toTIS

Team Integration Test

Continuous Testing in Continuous Delivery Flow

Issue/DefectManagement

Deploy toSIT

System Integration Test

Deploy toQA

FunctionalTest

Deploy toSTG

PerformanceTest

Deploy toPRD

Junit/LeanFTTest

HCM deploy K8S Deploy Mesos deploy Mobile Deploy

2

Junit/LeanFT/SCATest

Run Unit Test LeanFT tests SCA Code Scan LeanFT test

cases in GITLab

3

SSC/ALM/OctaneIssues/defects

Defects & test results


4

Functional test using UFT

Optionally Security Dynamic Testing

UFT test cases in GITLab

5

UFT/WebInspectTest



6

Regression UFT Functional test

Security Dynamic Testing

Test Data Mgmt

7

UFT/WebInspect/TDMTest



8

ALM/PC/WebInspectTest

Performance Testing

Optionally Security Dynamic Testing

9Defects & test results

Test results Issues/Defects Traceability

from story, code, build, test, result, defect, release

Test & quality metrics

10

Defects & Results

11

Local Unit tests LeanFT test

1

Developer

12

Security SME

QA

LeanFT/UFT Test Cases

GITLab

Speed of DeploymentComparison (VM v Docker)Feature

sentiment

FeatureUsage

MTTRs workaround

Security

Velocity

Quality

Productivity

Measuring continuous integration & continuous testing

Metrics

Mean time to restore service with fix

RequirementsCoverage Ratio

DeploymentSuccess Rate

Speed/Frequency ofBuild Verification

Security TestPass Rate

Speed ofDeployment

Frequency ofDeployment

Lead, Waite &Cycle Times

Why Micro FocusFor DevOps, continuous integration & continuous testing

The Micro Focus software services difference

Model Office

Your reference

Operating Model

Your transformation

Transformation Workshop

Roadmap to success

Education and MoC

Driving your adoption

Containerization

DockerEE, K8S, micro-services

ChatOps

Proven customer success

KPIs and analytics

Your management approach

Mobility and Security

Industry leading solutions

IT4ITThe IT4IT standard provides a vendor-neutral, technology-

agnostic, and industry-agnostic reference architecture for

managing the business of IT, enabling insight for continuous

improvement.

SAFe – Scaled Agile FrameworkScaled Agile Framework (or SAFe) is an agile software

development framework consisting of a knowledge-base of

integrated patterns intended for enterprise-scale Lean-Agile

development.

BEST PRACTICESInsustry best practices that discuss the full lifecycle

STRATEGY TO PORTFOLIODrive IT Portfolio to business innovation

S2P REQUIREMENT TO DEPLOYBuild what the business needs, when it needs it

R2D

REQUEST TO FULFILLCatalog, fulfill, & manage service usage

R2F DETECT TO CORRECTAnticipate & resolve production issues

D2C

IT4IT – YOUR MOSTLY COMPLETE VIEW OF

INFORMATION AND TECHNOLOGY

Provides the strategy to balance and

broker your portfolio1

Improves data quality for decision-

making3

Provides a unified viewpoint across

PMO, enterprise architecture, and

service portfolio

2

Provides KPIs and roadmaps to

improve business communication4

STRATEGY TO PORTFOLIO (S2P)

Provides a framework for creating,

modifying, or sourcing a service1

Enables visibility of the quality,

utility, schedule, and cost of the

services you deliver

3

Supports agile and traditional

development methodologies2

Defines continuous integration and

deployment control points4

REQUIREMENT TO DEPLOY (R2D)

Helps your IT organization transition

to a service broker model1

Efficiently manages subscriptions

and total cost of service3

Presents a single catalog with items

from multiple supplier catalogs2

Manages and measures fulfillments

across multiple suppliers4

REQUEST TO FULFILL (R2F)

Essential SAFeEssential SAFe is most basic configuration of the

framework and it provides the minimal elements

necessary to be successful with SAFe

Large Soliution SAFeLarge Solution SAFe is for enterprises that are

building large and complex solutions, which do not

require the constructs of the portfolio level.

Portfolio SAFePortfolio SAFe provides portfolio strategy and

investment funding, Agile portfolio operations, and

Lean governance

Full SAFeFull SAFe represents the most comprehensive

configuration. It supports building large, integrated

solutions.

SAFe– YOUR MOSTLY COMPLETE VIEW OF

THE AGILE ENTERPRISE

LARGE SOLUTION SAFeLarge Solution SAFe is for enterprises that are building large and complex solutions, which do not require the constructs of

the portfolio level.

FULL SAFeFull SAFe represents the most comprehensive configuration. It supports building large, integrated solutions.

Backlog Elements in SAFe

25

ENTERPRISE DEVSECOPS

PLANDEVELOP

TESTRELEASE

DEPLOY OPERATE

Enterprise DevOps starts with Business Demand and ends with Customer Value

while security is a major part of the automated process

Enterprise DevSecOps

SECURITY

Enterprise DevOps

27

Continuousplanning

PPM, ALM Octane,

ALM.Net

Plan

Define

Continuousintegrationand testing

intelliJ,Eclipse,VSALM Octane

LeanFT, SV, SprinterFortify (1, 2))

Dev & Build

Test

Continuousdeployment and

releaseJenkins& ALM &

Chef, HCM (CODAR)Docker EE (3)

Release

Deploy

Continuousoperations

Docker Mgmt, CMS, SMA-XSiteScope,

AppPulse(4)

Operate

Monitor

Continuous Security

Continuous assessment

DemandCustomer Experienc

e


(1) SSC,Maven, Cradle, GITLab, Nexus, Jenkins, verigreen, Docker, Docker Registry (2) UFT/LeanFT, Performance Center, Service Virtualization, Network Virtualizatio, Mobile

Center, Fortify, Weblnspect, ALM (3) K8S, Mesos, Mobile Center, UCMDB, SMA(4) BSM, Fortify AppDefender

Managed in a Controlled yet Open Ecosystem(Open Source Solutions are pervasive in Public Sector)

PLAN

OPERATE

BUILD

TEST

RELEASE/DEPLOY

Value driven portfolio

optimizationSteering committee

demands

Programs Projects Requests Epics FeaturesPortfolio

Sub Portfolio

Sub Portfolio

Program

Program

Agile Project

Project

Program

Agile Project

Epics

Program

Program

Agile Project

Project

ALM OctaneAgile Project

JIRAAgile Project

ALM OctaneAgile Project

JIRAAgile Project

CA RallyAgile Project

VersionOneAgile Project

Budget

Strategy

Resources

application development vs business goals

governance & industry regulations

Public Sector is still Hybridmultiple toolchains

(open source, commercial), methodologies, frameworks

Application Lifecycle Management at Enterprise Scale

We help customers manage hybrid application lifecycles (Waterfall, Agile, DevOps) and increase velocity and quality of testing.

32

Key Points

• Plan – think it thru • Process – understand it• Procedures – document them• Build It – put it together, piece by piece• Test – make sure it is working, verify • Review• Evaluate• Improve

• Don’t try to accomplish too much, too fast• Move at a speed your organization is comfortable with

33

Civilian Agency

• Issue was –• Dropped code • Manual testing • Experienced too many issues in production• Systems where not reliable

• Desired –• Better Quality • Higher reliability in production

• Solution – created test management,• Created test scripts• Executed automated functional test• Ran regular regression test • Then performance test • Monitored systems in production reusing functional test

• Instituted running SiteScope to monitor

34

Homeland Agency

• Issue was –• Waterfall methodology • Manual testing• Poor Quality

• Desired –• Embrace Agile• Test Earlier – shift left • Improve Quality• Deliver Faster

• Solution –• Leverage MF ALM as test repository • Built UFT scripts for some functionality• Leveraged Selenium tests from Agile sprints • Integrated CI into SDLC• Integrated JIRA defects with ALM • Tested more thoroughly and Increased Quality• Reduced Defects and Risk

35

Defense Organization

• Issue was –• Manual testing• 600+ test plans • 50 + SMEs • 6 month effort

• Desired –• Shorter cycle• More testing via Automation• Find Issues earlier in the SDLC

• Solution –• Leverage MF ALM for repository• Automate test scripts with UFT• Regression test in Dev and Pre Prod • Improve the CD effort • Cloud Hosted solution

36

Civilian Agency • Issue was –

• Poor Requirements management • Manual testing • Weak Defect management

• Desired –• FedRAMP approved cloud hosted solution• Test Management solution• Better Requirements management

• Full lifecycle traceability • Improved Defect resolution

• Solution –• AWS GovCloud Hosted Test Management • Leveraged MF ALM

• Requirements management• Test cases• Test scripts• Defect management

PROVEN LEADERS LEADING w/LEAN AGILITY

Successful introductions of Lean &

Agile methods in Human

Resources, Marketing, Corporate

Strategy, Risk & Compliance,

Legal and FinanceHOLISTIC APPROACH

Successful adoptions at 5 levels of

scale: Team, Train, Value Stream,

Portfolio & Enterprise

INNOVATION w/LEAN

AGILITY

Developed advanced techniques in:

• Agile Metrics

• Balanced Scorecard

• Lean-Agile Portfolio & ProdMgmt

• Team & ART Benchmarking

• Digital Strategy

GOV-WIDE REACH,

PERSONAL TOUCH

Operated in most agencies over

past 21 years , Commercial Div

PL

LE

HA

IN

GR

B12 STRATEGY CONSULTINGB12 is a Full-Service Enterprise Solutions Provider

(2012)

(1998) XP (1998)

On GovCloud

with

B12 Micro and Managed Service Solutions

Small Business Program

Small-Business

Woman-Owned

Minority-Owned

Org TransformationVideo Analytics Platform

with UniqueContent

Micro Focus Center of Excellence

DOD/Intel Cleared

GSA Schedule 70 PrimeAlliant II, EAGLE II, TIPSS, STARS, GITSS, COMMITS

subcontract positions

ADM

IM&GSecurity

ITOM

and legacy

Gov aaMS Offerings:

AM, DevOps, Gov Compliance

ITOM, Docker, TBMGovLean, & PPM

Thank You. Questions?

Shannon Lake – [email protected] - 650-417-1000 (cell)

DevOps and Continuous Testing€¦ · · 2018-03-21time it takes to release new features or...

Documents

Transcript of DevOps and Continuous Testing€¦ · · 2018-03-21time it takes to release new features or...