Parasoft Proprietary and Confidential 1

2016-03-11

End-to-End Testing for IoT integrity

Parasoft

Mark Lambert (mlambert@parasoft.com)

Mirosław Zielinski (mirek@parasoft.com)

Parasoft Proprietary and Confidential 2Parasoft Proprietary and Confidential 2

Agenda

IoT specifics

Challenges in quality assurance

Effectiveness of end-to-end testing

Doing it better

Demonstration

Parasoft Proprietary and Confidential 3Parasoft Proprietary and Confidential 3

IoT specifics

Multi-layered systems

Disparate technologies

Functionality

spanning across many

layers

Parasoft Proprietary and Confidential 4Parasoft Proprietary and Confidential 4

IoT specifics

Even simple functionality may require different teams to develop

Debugging requires solid understanding of many technolgies

Complex functionality may be hidden behind simple interface

Parasoft Proprietary and Confidential 5Parasoft Proprietary and Confidential 5

Why it is difficult to test IoT

Disparate development technologies hamper introducing consistent testing strategies

“Not enough code to justify testing” syndrom

Limited user interface make the solution to look deceptively simple

Inverted testing pyramid problem

Strong “temptation” to rely entirely on system testing

Parasoft Proprietary and Confidential 6Parasoft Proprietary and Confidential 6

Effectiveness of end-to-end testing

Labor intensive

Difficult to automate

Difficult to start early

Requires inter-teams

synchronization

Difficult to test error handlers

Failed tests do not provide precise information about failure points

Parasoft Proprietary and Confidential 7Parasoft Proprietary and Confidential 7

Effectiveness of end-to-end testing

End-to-End testing for diabetes health system example

Difficult to apply stimulation

Difficult to validate test result

Difficult to test error handlers

Parasoft Proprietary and Confidential 8Parasoft Proprietary and Confidential 8

Why is it difficult to test IoT?

Well known problems …

What makes it special?

So far unseen combination of technologies

Applies to high/low budget projects

Parasoft Proprietary and Confidential 9Parasoft Proprietary and Confidential 9

How to do it better?

Limit manual end-to-end testing

Invest more into automated tests at component

Follow testing pyramid rules

Measure: test results

test effectiveness

Parasoft Proprietary and Confidential 10Parasoft Proprietary and Confidential 10

How to do it better?

Give absolute priority to automated tests

Invest time into designing interfaces (API)

Use API (service) testing tools to cover interfaces

Measure the quality of the test

Parasoft Proprietary and Confidential 11Parasoft Proprietary and Confidential 11

How to do it better?

Use stubs and mocks to improve:

Facilitate test automation

Increase code penetration

Parasoft Proprietary and Confidential 12Parasoft Proprietary and Confidential 12

How to do it better?

Use service virtualization to improve testingautomation

Isolate at the message layer

Simulate functional scenarios and performance conditions

Parasoft Proprietary and Confidential 13Parasoft Proprietary and Confidential 13

How to do it better?

Report test results against requirements

Try to get holistic view of your testing process

Aggregate and review test data from different layers

Assess health of given feature across all layers

Parasoft Proprietary and Confidential 14Parasoft Proprietary and Confidential 14

Test scenario example

Parasoft Proprietary and Confidential 15Parasoft Proprietary and Confidential 15

Test Scenario: REQUEST_NEW_BOLUS_UPDATE_PUMP

Tested System

Tested System

Parasoft Proprietary and Confidential 16Parasoft Proprietary and Confidential 16

Summary: End-to-End Testing for IoT integrity

IoT combines previously isolated technologies Multiple Tiers = Complexity = Risk End-to-End Testing = Chain is only as strong as it’s

weakest link

Reliable quality process allows organizations to assess modules individually and in combination Continuously automate testing Tracability across tiers, technologies and testing

techniques

Parasoft Proprietary and Confidential 17Parasoft Proprietary and Confidential 17

Q & A