COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1

This document and the information contained herein is the property of Saab AB andmust not be used, disclosed or altered without Saab AB prior written consent.

Testing Gripen E –Test Automation andExploratory Testing

Torvald MårtenssonTest Manager Gripen E/F M1, Technical Fellow

Linköping University, October 2 2020

1

COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1

Who am I?

• Torvald Mårtensson PhD MSctorvald.martensson@saabgroup.com

• Test Manager in the Gripen E systemsdevelopment project at Saab

2

COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1

Takeaways from this talk

• What is Gripen E?

• Testing a large-scale and complex softwaresystem – how is it done?

• Three examples of practices implemented inthe Gripen E project

3

COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1

Gripen A/B and C/D

• Gripen A (single seater) and B (dual seater)was developed for the Swedish Air Force

• Gripen C and Gripen D are currentlyoperational in SE, CZ, HU, TH and SA

4

PAGE 25Torvald Mårtensson

COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1

Gripen E – a new aircraft

5

First flight for the first Gripen E test aircraft (June 2017)

First flight for the first Brazilian Gripen E aircraft (August 2019)

COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1

Developing Gripen E

6

Training Systems

Support Systems

COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1

Developing Gripen E

7

Training Systems

Support SystemsLarge-scale Complexity

Testing a large-scale and complex software system

COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1

Testing on different levels

8

Testing asoftware

component

Testing asubsystem

Testing thecomplete

system

Manual testing(scripted and exploratory)

Automated testing

COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1

Testing in different types of test environments

9Simulation of targets and threats

Simulation or real hardware?

COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1

Testing in different types of test environments

10

Hardware in the loopsub-system rigs

Test aircraft

Software in the loopsimulators

Hardware in the loopsystem rigs

Example 1 Example 2

Real hardware

Simulation

Real hardwareSimulation

Real hardware

Simulation

Real hardware

Simulation

COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1

Practices implemented in the Gripen E project

11

• Testing in the continuous integrationand delivery pipeline

• A simulator on every developer’s desk

• Exploratory testing of the completesystem

COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1

The continuous integration and delivery pipeline• Creating a pipeline of test activities which evaluate

software changes made by the developers• Example:

• Design the pipeline with…• Unit/component tests and system tests• Automated testing and manual testing• Testing in simulators and on real hardware

• There is no optimal pipeline for every product andevery organization

• Designing the pipeline with a systematic approachrather than making changes and hoping for the best

12

Duvall: “Stage builds”

Larman and Vodde: “multi-stage CI system”

Humble and Farley: “deploymentpipeline” or “integration pipeline”

COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1

A simulator on every developer’s desk

• The MySim simulator is available for alldevelopers from the developer’s own desk

• The developers are more confident that theirsoftware is working

• MySim is also used for automated regressiontesting on different levels of the product

• Faster feedback to the developers13

COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1

Exploratory testing of the complete system

• Exploratory testing with experienced engineersrepresenting different subsystems

• The test teams identify complex integration problemswhich are difficult to find with automated testing

• The developers test the complete system together –not just a software module or a subsystem

14

“We could test according to the ideas we had. We wantedto understand the system that we were building and to findthe weaknesses in the system”.

“The quality of the problem reports improvesif there are people from different subsystemsparticipating at the test”.

COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1

Summary and takeaways

• What is Gripen E?• A completely new aircraft• A large-scale and complex software system

• Testing a large-scale and complex softwaresystem – how is it done?

• Testing on different levels• Testing in different types of test environments

• Three examples of practices implemented inthe Gripen E project• Testing in the continuous integration and delivery pipeline• A simulator on every developer’s desk• Exploratory testing of the complete system

15

COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1

16