DevOps maturity models Knowit and DASA

27 January 2018

Kari Kakkonen

Knowit Oy (Finland)

[email protected]

Twitter @kkakkonen

Linkedin karikakkonen

DevOps maturity models by Knowit and DASA views

Knowit 1

Knowit 2

ROLES

• Knowit Oy, Director/Quality and Competences, Lead Consultant,

Trainer and Coach

• Treasurer of ISTQB Executive Committee

• Treasurer, ex-chairman of Finnish Software Testing Board (FiSTB)

• Auditor of Robot Framework Foundation.

ACHIEVEMENTS

• Influencing testing since 1996

• Ranked in 100 most influential IT persons in Finland (Tietoviikko magazine)

• Great number of presentations in Finnish and international conferences

• TestausOSY/FAST founding member.

• Co-author of Agile Testing Foundations book

• Regular blogger in Tivi-magazine.

EDUCATION

• ISTQB Expert Level Test Management Full, Advanced Full,

Agile Tester & Test Automation Engineer certified

• DASA DevOps Fundamentals, Scrum Master and SAFe certified

• SPICE provisionary assessor certified

• M.Sc.(Eng), Helsinki University of Technology (present Aalto University), Otaniemi, Espoo

• Marketing studies, University of Wisconsin-Madison, the USA.

BUSINESS DOMAINS

Wide spread of business domain knowledge

• Embedded, industry, public

• Training, telecom, commerce

• Insurance, banking, pension.

SERVICES

• ISTQB Advanced, Foundation and

Agile Testing, Knowit Quality Professional

• DASA DevOps Fundamentals

• Quality & Test process and organization development,

metrics

• Agile testing, Scrum, Kanban, Lean

• Leadership

• Test automation, mobile, cloud, DevOps

• Quality, cost, benefits.

3

Twitter: @kkakkonen

LinkedIn:

fi.linkedin.com/in/karikakkonen/

CV Kari Kakkonen - CONFIDENTIAL

Knowit is the Nordic company

that through a combination of

creativity, strong strategic

knowledge and a passion for

technology creates digital

opportunities.

Digital opportunities

Knowit 5

Knowit clients

Knowit 6

Testing and test management

• Fundamentals of testing

• Test planning and management

• Improving testing process

• Agile and Exploratory Testing

• Test case design

• Usability testing

• Security testing

Testing certifications

• ISTQB Foundation in Software Testing

• ISTQB Agile Tester

• ISTQB Advanced Level (Test Manager, Test

Analyst, Technical Test Analysit)

Robot Process Automation (RPA) & AI

• RPA for business representatives

• RPA for technical team

• RPA tools

• Robot Framework

• UI Path

• Artificial Intelligence briefings

Test automation

• Test tool selection and implementation

• Automation tools, i.e.

− Robot Framework Basics & Advanced

− HP UFT

− Jmeter & performance testing

− Management tools, i.e

− ALM, SilkCentral, Meliora

− Jira, VersionOne

Web and analytics

• Google Analytics basics & business

• Google Tag Manager

• SEO for communication professionals

Agile & DevOps

• DevOps briefing (CI ,Continuous Delivery,

DevOps Culture)

• DASA DevOps Fundamentals

• Scrum Master / Product Owner / Team

• Lean mindset

• User Story creation

• Scaled Agile Framework SAFe

727.1.2018

Examples of Knowit training offering

Software and web development

• Embedded C-programming

• Python Basics

• Security in Programming

• Cloud, Docker & Ansible

Knowit

• DevOps introduction

• Business benefits

• Terminology, e.g. Continuous Delivery

• Faster feedback

• Automate everything you can

• DevOps Culture

• DASA DevOps principles

• Organization

• Lean & Agile

• Autonomous teams

• Microservices and Cloud

• DevOps Maturity

• Step by Step to DevOps

• Maturity Model

8

Devops introduction

Knowit 9

Taking new technology into use gets faster exponentially

Knowit

How quickly a product/an application reached 50M users?

Television

Facebook

Internet

Instagram

Angry Birds

Pokemon Go

13 Years

4 Years

3,5 Years

35 days

9 days

6 months

10

Opportunities provided by DevOps (2016)

Knowithttps://puppet.com/resources/white-paper/2016-state-of-devops-report

200xmore frequent

deployments

2550x shorter lead

times

50%less time

spent remediating

security issues

3x lower change

failure rate

22% less time spent on

unplanned work and

rework

24xfaster recovery

from failures

11

https://puppet.com/resources/white-paper/2016-state-of-devops-report

Opportunities provided by DevOps (2017)

Knowithttps://puppet.com/resources/whitepaper/state-of-devops-report

46xmore frequent

deployments

440x shorter lead

times

5x lower change

failure rate

96xfaster recovery

from failures

12

https://puppet.com/resources/whitepaper/state-of-devops-report

How devops copes with software development challenges

Challenges in SW development

Long testing periods (weeks)

Long release cycle (months)

Quality problems

Unstable environments

Unused features

Lot of complicated surprises

High maintenance cost

Slow reaction to customer needs

Knowit

DevOps possibilites

Fast feedback via automation (in hours)

Faster time to market (continuous delivery)

Continuous Quality

More efficient use of environments

More profitability

Better visibility and predictability

Less maintenance effort

Higher customer satisfaction / more customers

13

DevOps and other terminology

Knowit

Continuous Integration

Continuous Delivery

Continuous Deployment

DevOps

Prepare Implement Build QA Deliver Deploy Operate

Continuous feedback

Agile development

14

Devops is about culture of high-performance IT: Business is not separate from software & product

Agile & Lean Way of

Working

Modern Software

Architecture

Continuous Delivery

Intelligent Infrastructure

Talent Management

Knowit 15

months

DevOps: from slow lead time to fast lead time?

Speed demands quality


Knowit 16

months

DevOps: analyze and improve incrementally


Measure

AnalyseOptimize


Knowit 17

months

weeks

hours

DevOps:faster cycles, visibility, better feedback and more profits

Speed demands quality



Knowit 18

Automate Everything you Can: Version Control

Knowit

Version-

controlcode

commit

Development services

19

Automate Everything you Can: Continuous integration

Knowit

Build and installation

CI Install CI Install

UAT Performance

Continuous Integration (CI)

Version-

controlcode

commit


CI Build

Integration

20

Automate Everything you Can: Continuous integration and Test Automation

Knowit

Automated tests, build and installation


UAT Performance


Version-

controlcode

commit


CI Build

Integration

21

Automate Everything you Can: Continuous integration, Test automation,Test data and Interfaces

Knowit



UAT Performance

Test data management

Virtualized Backend services


Version-

controlcode

commit


CI Build

Integration

22

Automate Everything you Can: Continuous integration and Continuous Delivery

Knowit


CI Install CI Install Deliver

UAT Performance

Continuous Delivery (CD)




Version-

controlcode

commit


CI Build

Integration

23

Automate Everything you Can: DevOps and Continuous Deployment

Knowit



Version-

controlcode

commit

CI Build CI Install Deliver

Integration

UAT

Performance

End user

monitoring

DeployProduction

Quality metrics


Robot framework testsDevelopment services

Virtualisation

24

But wait, can you automate everything?

• Some things are feasible still to execute manually

• Architecture

• Actual coding, including refactoring & optimization

• Test planning

• Risk evalution

• Test optimisation (until Artificial Intelligence replaces this...)

• Usability tests

• Exploratory tests (sometimes using scripts and automation as helper tools)

Knowit 25

Devops culture and DASA

Knowit 26

About the DevOps Agile Skills Association (DASA)

DASA is an independent and open

association supporting the development of

DevOps training and certification to the

global market.

Advocating the development of high-

performance IT professionals and

organizations through agile DevOps

initiatives.

DASA offers thought leadership as well as

practical guidance for competence

development for professionals and

organizations.https://www.devopsagileskills.org/

The DASA DevOps Qualification Program

28

1Novice

DevOps: Fundamentals

DevOps Specialization:

Create and Deliver


Specify and Verify


Enable and Scale

DevOps: Practitioner

5Master

2Competent

3Proficient

4Expert

3 days

2 days

2 days 3 days2 days

DASA Six Principles

29

1. Customer-Centric Action

• Short feedback loops

• With real customers

• Act as lean startup

Knowit 30

2. Create with the End in Mind

• See the complete picture

• Build for real customers

• Act as product companies

• Engineering mindset to find solutions

Knowit 31

3. End-To-End Responsibility

• Accountable from concept to grave

• Organized vertically

• Responsibility leads to quality

• Culture

Knowit 32

4. Cross-Functional Autonomous Teams

• Independent teams

• T-shaped all-around skill profiles

• Functional teams or platform teams

Knowit 33

5. Continuous Improvement

• Adapt continuously

• Minimize waste

• Continuously minimize product and process

• Learning from failures

Knowit 34

6. Automate Everything You Can

• Automate testing

• Automate software delivery process

• Automate infrastructure

• Container-based cloud platforms

Knowit 35

Devops organization

Knowit 36

Lean: Eliminate waste

• Defects

• Overproduction

• Waiting

• Not utilizing talent

• Transportation

• Inventory excess

• Motion waste

• Excess processing

Knowit 37

Lean: Value stream mapping

• Process from user need to fulfilling user need

• Aiming for flow efficiency: fast throughput

Knowit

Identify

needDesign Coding Testing Accept Deploy

20 days 2 days 5 days 10 days 1 days

1 days 5 days 10 days 10 days 3 days 2 days

Wait:

38 days

Add Value:

31 days

Flow efficiency = 31 / 69 = 45%

Total time:

69 days

Can we do something in parallel?

Can we remove waiting time (waste)?

38

Kanban - Visualize and Optimize the Flow Work within a value-added chain

Knowit

• Work-in-Progress Limit: The amount of parallel

active tasks is strictly limited

• Whenever a station has free capacity, the

worker pulls a ticket from the previous station.

• Lead Time: Optimize the continuous flow of

tasks by minimizing the (average) lead time for

the complete value stream.

• Kanban Board: Each column

shows a station, which is a set

of related activities. The items to

be produced or tasks to be

processed are symbolized by

tickets moving from left to right

through the stations.

39

Continuous improvement

• Deming cycle PDCA (Plan-Do-Check-Act)

• DMAIC (Define-Measure-Analyze-Improve-Control)

• Kaizen

• Etc.

Knowit

Signals

Problems

HypothesisExperiment

Analyze

40

Autonomous Teams: in charge of everything

• How they develop code

• How they test

• How they connect to other systems or teams

• When they connect

• When they deploy to production

Knowit 41

Two types of teams

• If any team needs another team’s product or service, they can always get the latest ready to deploy

version automatically

Knowit

Functional team Functional team Functional team

Platform team Platform team

Self-service

interface

42

Microservices and cloud

• Move your architecture towards Microservices

• Enables more frequent delivery

• Consider cloud principles to your platform teams

• Helps you utilize environments as a service

• Less dependencies

• More frequent delivery

Knowit 43

Devops maturity

Knowit 44

Steps to DevOps

Knowit

Responsiveness

Agile methods in use

in development

scrum, backlog etc

Automation

deployed in larger

scale. Daily build

with automated

regression tests

Ability to deliver daily

SW quality metrics

support daily

activities

Changes deployed

automatically to

production

environment as soon

as available from

production pipe

Continuous

Integration

Continuous

delivery

Continuous

deployment

DevOps

Operational metrics

transparent, feature

throughput in days,

Processes fast and

lean, continuous

learning and

improvement

Automation in

product level,

automated

regression tests

Test Automation

Qualit

y

Agile development

45

DevOps development example blocks

Knowit

Requirement Mgmt

Planning

Customer Data Mgmt

Release &

Configuration Mgmt

Architecture

Implementation Testing Delivery Deployment Production

Usage metrics

Fault MgmtAgile methods, Scrum,

backlog

Unit tests, static

analysis

Access Rights +

Version Control

Test Case Mgmt Customer Data

Mgmt

46


Knowit

Requirement Mgmt

Planning

Customer Data Mgmt

Release &

Configuration Mgmt

Architecture

Implementation

Agile methods, Scrum,

backlog

Technical debt

visualisation e.G.

SonarQube


/ Jenkins etc

Unit tests, static

analysis

Access Rights +

Version Control

Testing

Test Case Mgmt

Regression Test

Automation

Delivery Deployment

Customer Data

Mgmt

Production

Usage metrics

Fault Mgmt

47


Knowit

Backend service

virtualization

virtualisointi

Planning Implementation

Technical Debt

visualisation e.g.

SonarQube


/ Jenkins etc.

Virtualization/ cloud

environments

Testing

Test Case Mgmt

Regression Test

Automation

Test Data Mgmt

Installation Automation

Delivery

Automated transfer

KäyttöDeployment

Automated

update/upgrade

Production


backlog

Unit tests, static

analysis

Access Rights +

Version Control

Requirement Mgmt

Customer Data Mgmt

Release &

Configuration Mgmt

Architecture

Customer Data

Mgmt

Fault Mgmt

Usage metrics

48


Knowit

Käyttö

Käyttömittaristo

Planning Implementation


environments

Testing

Test Data Mgmt



environments incl.

backend services

Full test automation

Optimized transfer

Delivery

Automated transfer

Deployment

Cloud environment

(Docker, Ansible

etc.)

Automated

update/upgrade

Production

Usage metricsRequirement Mgmt

Customer Data Mgmt

Release &

Configuration Mgmt

Architecture

Technical Debt

visualisation e.g.

SonarQube


/ Jenkins etc.


backlog

Unit tests, static

analysis

Access Rights +

Version Control

Test Case Mgmt

Regression Test

Automation

Customer Data

Mgmt

Fault Mgmt

49

Optimized transfer


Knowit

DeploymentImplementationPlanning Testing Delivery

Automated transfer

Microservices & Cloud

Optimized update/

Microservices

Production

Fault Mgmt

Usage metrics

Cloud environment

(Docker, Ansible

etc.)


environments


environments incl

backend services

Optimized

cloud

environments

Automated

update/upgrade

Transparency /optimization / KPI’s

Organization / feature teams

Full test automation

Requirement Mgmt

Customer Data Mgmt

Release &

Configuration Mgmt

Architecture

Technical Debt

visualisation e.g.

SonarQube


/ Jenkins etc.


backlog

Unit tests, static

analysis

Access Rights +

Version Control

Test Data Mgmt


Test Case Mgmt

Regression Test

Automation

Customer Data

Mgmt

50

Devops/automation level visualisation

SW Development- SW development

-Unit test

-Module test

-Static analysis & security analysis

-etc

SW Building-Adding SW

modules/versions to a

build

-Static analysis

-etc

Install /Commissioning-Reservation

-SW download

-SWBT /sanity test

-etc

QA / testing-Regression set

-Entity testing

-Functional

-performance

-security

-Network Verification

Time Time Time Time

Deliver- internal / external

- Delivery logic (pull vs push)

- optimized methods

Time

Deploy-Installation / phased

-Update/upgrade

-Fallback

Time

Project SW Development SW Building Install /

Commissioning

QA / Testing Deliver Deploy

Org1 Team1

Org1 Team2

Org2 Team1

Org2 Team3

Full automation Partly automated Under planning Not reasonable to

automate

Manual

Knowit 51

1. Test automation

Step by step forward, example

Sprint 1

Snapshot

Sprint 2

Roadmap

Sprint 3

Implementation

PO and team roles defined, agile process desription

Test automation PoC, test automation plan

CI pipeline modules implementation

Agile development understanding

Test automation implementation, phase 1 Test automation implementation, phase 2

Area to be developed

CI Process descriptions and gap

analysis

Trainings, workshops

2. Continuous Integration

3. DevOps/ agile

development steps

4. Competence

development

Competence development plans, chosen courses

Knowit 52

Knowit DevOps Maturity Model

Knowit

Devops maturity 1 2 3 4

Organization and culture Planning, Development

and quality assurance in

own silos

Agile methods in use in

planning&development, QA in

separate organization

Feature team is responsible all the way to

production

Autonomous teams in both

feature and platform level

Continuous Delivery readiness (Dev &

Build & Test & Deliver)

No CI or only some

phases covered

CI in use, typical feedback loop over

a week.

CI pipeline fully automatic, includes all the

tests, cycle time max couple of days,

green build available for delivery

CI pipeline cycle less than 12

hours, test coverage and other

quality criteria meets the delivery

quality criteria

Fast deployment to production Deployment cycle

typically months

Requirement analysis and impact to

SW components takes weeks.

Development does the build ready

for acceptance tests.

Requirement analysis and impact to whole

product can be analysed in few hours.

Acceptance tests are automated,

Releasing/deployment to production done

manually.

Deployment automatic, when

critical failure in update,

automatic fallback happens and

feedback to development

delivered.

Visibility and reporting No visibility, missing or

manual reporting

Reporting of each phase exists, but

visibility only to that certain phase.

Test coverage and static code analysis

results visible for whole development

Product operative metrics visible

for developers, in case of

problems automatic alert to

development

Architecture Architecture unclear, lot of

surprises when something

changed

Product is one entity, installed as a

whole.

Product consists of subentities, and

interfaces are known and defined.

Microservices in use and

independently updateable

Product releasing and sales (pull vs

push)

No controlled feature or

configuration

management

Release content defined in advance

80%. Customer pays when release

acceptance is given and content is

inproduction in customer system.

Release content half fixed, half adapting

to order base. Product/update can be

ordered via internet.

Features done on demand, First

order and payment, then

development and delivery.

BDD/ATDD from customer

needs. Ordering and payment via

internet.

Kari [email protected]@kkakkonen

Thank you!

Knowit 54

DevOps maturity models Knowit and DASA

Technology

Transcript of DevOps maturity models Knowit and DASA