Quality Challenges of the Digitalized World – How Does Quality Assurance Change?Kari Kakkonen, Director, Quality and Competences, Knowit Oy, Finland

WCSQ, Lima, Peru 21.3.2017

Kari Kakkonen

ROLES

• Knowit Oy, Director/Quality and Competences, Lead Consultant, Trainer and Coach

• Treasurer of ISTQB Executive Committee

• Chairman of Finnish Software Testing Board (FiSTB)

• Chairman of research project STX, Lappeenranta University of Technology.

ACHIEVEMENTS

• Influencing testing since 1996

• Ranked in 100 most influential IT persons in Finland (Tietoviikko magazine)

• Number of presentations in local and global seminars

• TestausOSY/FAST founding member.

EDUCATION

• ISTQB Expert Level Test Management Full & Advanced Full & Agile Tester certified

• DASA DevOps Fundamentals, Scrum Master and SAFe certified

• SPICE provisionary assessor certified

• M.Sc.(Eng), Helsinki University of Technology (present Aalto University), Otaniemi, Espoo

• Marketing studies, University of Wisconsin-Madison, the USA.

PAGE 2 [CONFIDENTIAL] QUALITY ASSURANCE PROPOSAL

BUSINESS DOMAINS

• Wide spread of business domain knowledge

• Embedded, Industry, Public,

• Training, Telecom, Commerce,

• Insurance, Banking, Pension

SERVICES

• ISTQB Advanced, Foundation and Agile Testing +

Knowit Quality Professional

• DASA DevOps Fundamentals

• Quality & Test process and organization

development, Metrics

• Agile testing, Scrum, Kanban, Lean

• Leadership

• Test automation, mobile, cloud, DevOps

• Quality, Cost, Benefits.

Twitter: @kkakkonen

LinkedIn: fi.linkedin.com/in/karikakkonen/

Revolution of User Experience

Mobile is the new reality – research tells us

520.3.2017 © Copyright Knowit Oy 2015 | Confidential | Version 1.0

www.shopify.com/blog

“Mobile Now Accounts for 50.3% of All Ecommerce Traffic”

Change of Business

1. Services are digitalized

2. Services processes are

digitalized

3. Internet of Things

7

Central

Quality Challenges

Security

Usability

Performance

Speed of software development

8

Security TestingEverything in focus

Security Testing

920.3.2017 © Copyright Knowit Oy 2015 | Confidential | Version 1.0

PreventiveSecurity

• Expert in the team

Application levelsecurity

• Every level must besecure

Interfaces

• Internet

• Machine-2-Machine

Testing

• Vulnerability scans

• Audits

Up-to-date

• OWASP etc.

10

Usability testingThink about the UX

20.3.2017 © Copyright Knowit Oy 2013 | Confidential | Version 1.0 11

Surface

Skeleton

Structure

Scope

Strategy

Applied from source:

The Elements of

User Experience

By Jesse James Garrett

• Visual design

• Micro-interactions

• Service structure

• Page structure

• Content structure

• Content structure

• Information architecture

• Functionality

• Content needs

• Business needs

• User needs

Bo

tto

m -

Up

Abstract

Concret

e

Usability testing can be…

Usability assessments

Comparing service to User

Experience Levels

Assessing usability problems criticality

Reporting usability problems

Suggesting solutions

1220.3.2017 © Copyright Knowit Oy 2015 | Confidential | Version 1.0

User device coverage –check usability with test automation

• Different devices, same functionality, same user experience

• Responsiveness

• One test library, many test objects

• Little to maintain

• Increasing number of objects

• Start early

• Repeat daily / by build

• Tests differ just a little

• Reporting

• Also other interfaces than UI

1320.3.2017 © Copyright Knowit Oy 2015 | Confidential | Version 1.0

Apps to be tested (Web)

Test automation from cloud or devices

Performance testingTest and Measure, don’t get surprised

Performance testing

Testing tools

• Commercial

• Open source

Continuousmonitoring

• Common goal with performance testing

Identifybottlenecks

• Fix

• Tune

Test coverage

• Devices

• Cloud

• Scenarios

Repeatability

• Include in regression tests

1620.3.2017 © Copyright Knowit Oy 2015 | Confidential | Version 1.0

Speed of software developmentBalance between speed and quality

Fast software development process

DevOps

• As a culture

• Technically

Agility

• In organization

• In projects

Test automation

• Supporting the Team

• Speeding the Process

• Expand the Coverage

• Help from AI

Exploratory Testing

• Defect findingeffectiveness

• Efficiency

Competences

• Versatile: testing, development, operations, business

1820.3.2017 © Copyright Knowit Oy 2015 | Confidential | Version 1.0

DevOps principles

• DevOps is a method which emphasizes

communication, cooperation and integration

between software development teams (Dev) and IT-

support / operations teams (Ops)

• Method requires agile software development

process

• It connects operations organization to software

development by using virtualized environments

• Production environments are part of software

development and as agile as the software

development

1920.3.2017 © Copyright Knowit Oy 2015 | Confidential | Version 1.0

Software

Development

Quality

Assurance

Operations /

Production

DevOps

Development services

Reporting

End user

monitoring

Virtualised backend services

Test data management

Versioncontrol

Continuous

Integration (CI)

Business

Code Commit

TestingDevelopment

Automated tests, build & install

CI build

CI install

Deliver

Integration UAT

Performance

Access controlQuality metrics Virtualisation

Deploy

203/20/2017 KNOWIT CONFIDENTIAL

Production

Deliver

Deploy

DevOps and Continuous Deployment

Agile goes forward, what fits together?

2120.3.2017 © Copyright Knowit Oy 2015 | Confidential | Version 1.0

Scrum

Kanban

Waterfall, V-

Model

Scaled Agile

Framework SAFeLean

Lean Six Sigma

DevOps

Continuous

Deployment

Hybrid

XP

Nexus

LESS

Continuous

Delivery

Knowit Research 2014:Agile test planning on company level

Strategic level

No plans Plan in high level Plan in detail

• Almost a quarter doesn’t plan agile

testing at all (is that agile anymore…)

• Mainly testing is planned in high level

(this is maybe the definition of agile

testing…)

• 85 % of company level testing is

documented (to some extent)

Company Level

• Not just

• Scrum

• Kanban

• …

• Organization level

• UVF

• M3.0

• SAFe

• Nexus

• Spotify Way

• Capo

• Agility Path

• Lean

• …2320.3.2017 © Copyright Knowit Oy 2013 | Confidential | Version 1.0

SAFe 3.0

24

Cloud (MS, Amazon, IBM...)

Example of test automation and continuous integration

20-03-2017 © Copyright Knowit Oy 2015 | Confidential

Robot Framework

Jira +Quality Center

Jmeter

Python libraries

Selenium

Jenkins

GIT

Quick Test Pro

SoapUI

Sharepoint

Confluence

Agile Testing Quadrants

26© Copyright Knowit Oy 2015 | Confidential

Source: Lisa

Crispin and

Janet Gregory

Use Exploratory testing

Source: Sinkkonen, Elisabeth Hendrickson, Cem Kaner, James Bach, Lisa Crispin

“In exploratory testing you learn from previous tests, and put that

learning into your next tests”

Plan

• Test charter

Test session

• Notes

• Bugs

Debriefing

• Dashboard

Testing

Opinion-forming

Reporting

Designing

actions

Observations

Learn versatilely about business, development, testing, operations

Courses, certificates Coaching

Pair work Workshops

Increasecompetences

2820.3.2017 © Copyright Knowit Oy 2015 | Confidential | Version 1.0

Prototype and test

Quality assurance in Digitalized World

Build Quality, then also Test

• Security

• Usability

• Performance

Get faster feedback

• Team velocity

• Support of testing

• Speed to Operations

Questions?

kari.kakkonen@knowit.fi

@kkakkonen

3/20/2017 Knowit 32

Creating digital opportunities