Towards Continuous Software Development for …...Continuous Integration Continuous Delivery … as...

V0.3 | 2020-06-08

EMCC 2020 Virtual Edition, 30th of July 2020

Towards Continuous Software Development for AUTOSAR Systems

2

Introduction

Applying Cx to AUTOSAR Development

AUTOSAR Continuous Integration

AUTOSAR Continuous Delivery

Summary & Outlook

Agenda

3

All Tesla vehicles with our FSD computer have been updated with new software that can better detect new details in their environments […]. We are currently validating this functionality before releasing to customers, and we look forward to its gradual deployment. We also introduced in-app purchases, where our customers can buy various software updates, such as basic Autopilot, Full Self Driving […]. Software will continue to play a growing role in our business model.

[Source: Tesla:“Q4 and FY2019 Update”]

Example of a Modern Software Product

Introduction

Tesla Financial Report Q4/2019:

All Tesla vehicles with our FSD computer have been updated with new software that can better detect new details in their environments […]. We are currently validating this functionality before releasing to customers, and we look forward to its gradual deployment. We also introduced in-app purchases, where our customers can buy various software updates, such as basic Autopilot, Full Self Driving […]. Software will continue to play a growing role in our business model.

[Source: Tesla:“Q4 and FY2019 Update”]

4

Maturity Levels of Continuous Software Development

Introduction

Plan Code

Agile

SCRUM, Feature Driven Development, etc.

Feature backlog with ranked tasks

Sprint based feature planning

5


Introduction

… as before, + Server based system with:

Plan Code Build Test

Feedback

Agile

Continuous Integration

Automatic software

configuration

Software generation (ASW, RTE,

BSW)

Software compilation

Software linking

Unit-testing (Integration

Tests)

Short feedback channel for developers

+

6


Introduction

Plan Code Build TestValidate & Pack

Test

Feedback

Agile


Continuous Delivery

… as before, + Server based deployment system with:

Automatic functional

tests

Rollout to staging

area

Automatic non-

functional tests

Automatic regression

tests

Automatic acceptance

tests

Product packing (executable,

docu, release-notes, …)

+ +

7


Introduction

… as before, +


Test Release Test

Feedback

Agile


Continuous Delivery

Continuous Deployment

automatic software deployment to fleet

8


Test Release Test Operate


Introduction

Feedback

… as before, +

Agile


Continuous Delivery

Continuous Deployment

Continuous Development & Operation (DevOps)

Feedback

+

automatic collection of feedback from operation

Including feedback in product planning

9

Continuous Software Development (Cx)

Good approach for dynamic times with many changes (e.g. technological) & flexible customers

What is Continuous Software Development in Automotive Industry about?

Introduction

Traditional Software Development

Good approach for stable times & highly reliable products

Product Update

Planning

Feature Structure

Stop of Planning

Testing

Release

New product with major features Small feature increment updates for released product

Long term (3-6 y) + detailed break-down, all stakeholders approve

Abstract vision + short term & flexible ranking (1-4 sprints), product owner considers stakeholders

Top-down decomposition of functionality Cross-structure features

Early feature freeze Dynamic changes in function scope

Long testing interval, manual tests (SIL, HIL, car) Fast testing, nearly 100% automation

New major software in 1-3 years Tiny software increment with weekly/daily updates

11

Introduction




Summary & Outlook

Agenda

12

… where it brings the most benefit

When activities are done with a high frequency, e.g. per commit

When activities are time-critical, e.g. before release/testing

When activities are repetitive and error prone

(Duration/Effort is relative – depends on time criticality)

Where to start with Cx


traditional

objective

13


Test Release Test Operate Test

Feedback

Test

Where to focus at AUTOSAR Projects to advance in Cx


Our focus for this talkState of the art For future

Checking timing requirements

e.g. meeting reaction constraints

Validating capacity boundaries

Staying inside predefined boundaries for software parts / ECU resources (CPU, memory, stack)

(functional testing is not considered here)

Cross-Team/Organizational Development

Integration of 3rd Party Software

ASW to BSW integration

Manual job by integrator (often bottleneck for executable software)

Providing fast testing possibilities

I – AUTOSAR Continuous Integration II – AUTOSAR Continuous Delivery

14

Introduction




Summary & Outlook

Agenda

15

For Developers

Getting testable software in hours instead of weeks

Reducing number of manual & monotonic tasks

For Integrators

Reducing repetitive tasks and focus on more complex system integrations (conflicts)

For Project Manager

Having early & continuous transparency on progress of implemented functionality

Objectives for Continuous Integration


Plan Code Build Test

Feedback


Efficiency

Speed

Change of Mindset

16

Eliminating exclusive operations on central artefacts

Preventing blocking of other engineers

Applying Decision Frontloading for integration

For automatizing integration activities

Use virtual testing

Executing Smoke Test for minimal executability check

Major Concepts for AUTOSAR Continuous Integration


17

Current Approach

System Architect

Updates Communication-Matrix or Diagnostic Description

SW Developer

Modifies SWC description according to changed software

Integrator

Applies all changes by integrating application software to BSW> Runnable to Task mapping

> Runnable Positioning inside Task

> Port Mapping

Integrator is the bottleneck for SW Developers



System Description

ECU-C

OEM / System Architect

changes

Com-Matrix & Diagnostic Description

update SWC Description

SW Developer

SW Developer

SW Developer

Integrator

applies changes

18

New Approach



ECU-C (Root)

SWC Description

(cutout)

Inte-gration Instr.

Auto-Integration

Diagnostic Description

Communication

Description

System Description

SWC Description

ECU-C

(Base)

SWC Description

(cutout)

Inte-gration Instr.

SWC Description

(cutout)

Inte-gration Instr.

ECU-C Root’

Auto-Integration

modified modified modified

build

build

ECU SW Architect/Integrator

BSW/Platform Expert

SW Developer

Modify & implement

Auto-Integration

build

Modify & implement

Auto-Integration

build

Modify & implement

organize

Executable with local changes

Executable with all changes

clone

19

Workflow


Root Configuration

DPA project configured for the used ECU, without apps and RTE

Integration Result

RTE configuration based on integration Instructions

App-dependent BSW configuration

App Package

SWC (atomic or composition)

Integration Instructions

Implementation (Source-Code)

App SWC

App SWC

App SWC

Platform BSW

App SWC

App SWC

App SWC

RTE

Platform BSW

Option CI

20

Major activities in case of changed Application Software

1. Runnable to Task Mapping & Positioning

2. Port Mapping

3. Data Mapping

Integration Instructions for CI


21

SWC A SWC B

SWC C

1. Via Direct Integration Instructions

Explicit Runnable to Task Mapping

2. Via Abstract Integration Instructions

Describing only in abstract way> Activation Period

> Safety Level

Mapping via matching properties

3. Heuristic Based

Using AUTOSAR model information> Runnable Trigger

> Safety Level

> Data Dependencies

> Resource Needs

Matching to Task definition via solving algorithm

ASW Integration Instruction for CI – Runnable-to-Task Mapping


R1

R2

R4

R5

R3

Trigger: 10ms

Safety:QM

Trigger: init

Safety: ASIL AResource: Stacksize/Runtime

Execution Order

Task 1 Task 2 Task 3

OS-Application:QM OS-Application:ASIL A

Task 4

22

Task 1

1. Via Direct Integration Instructions

Defining Execution Sequence of all Runnables

2. Via formal Constraints

Using Execution Order Constraints or Data Age Constraints for describing time critical dependencies between Runnables

3. Via Heuristics (Auto-Map)

Analyzing dependencies between Runnables and minimizing backward references

ASW Integration Instruction for CI – Runnable Positioning


R-A

R-B

R-C

R-D

R-E

R-(to be mapped) ?

?

23

ASW Integration Instruction for CI – Port Mapping


SWC A SWC B

SWC C

1. Direct mapping via Integration Instructions

Mapping SWC port to > Ports of other SWC

> Service ports (e.g. Trigger Ports, …)

Non-existing SWCs and Ports in integrated App Packages can be stubbed

2. Via Heuristics (Auto-Map)

Matching of Port Names

24

Using DaVinci Configurator Pro.CI in Agile Development Process

Project Application & IT Integration

Preparation of Root Config

Application Software Development

Updates of ECU Extract

Build System

RC-v1.0 RC-v1.1 RC-v1.2

Sprint A Sprint B

ECU Extract Updates get integrated in Root Configuration at dedicated dates (e.g. till sprint start)

App Developers work during sprint on prepared Root Config (Baseline)

In case of urgent ECU Extract Updates, a manual Baseline Update is required

Commit of App Package to Build Server produces Executable, based on specified Baseline

CI

ExecutablesE E E E E E E E E E

ECU Ext

ECU Ext

ECU Ext

ECU Ext

ECU Ext

ECU Ext

ECU Ext

25

Result 3 Types of Builds (offered by Option .CI)

1. Virtualization on VFB Level (vVIRTUALtarget Pro)

For testing ASW functions standalone (e.g. for Unit-Tests)

2. Virtualization on BSW Level (vVIRTUALtarget Basic)

For testing ASW functions in combination with BSW (e.g. for Integration-Test with BSW)

3. Hardware

For functional HIL tests

For trace measurements (see later)

Pipeline & Testing


Initialize Integrate Generate Build Check

26

Introduction




Summary & Outlook

Agenda

27

For Developers & Testers

Validate all functional and most non-functional properties automatically (feedback on malfunction)

Increasing confidence in committed software

For Project Architects

Be sure all system and function specific timing requirements have been met

Getting early notice of potential future resource exceeds

For Project Manager

Be sure that all resource restrictions have been met

Making software directly deliverable

Objectives for Continuous Delivery


Quality

Confidence

Fast Function Prototyping

28

Formalize Performance & Timing Requirements

For executing non-functional tests automatically

Perform Model-based Timing & Performance Analysis

For having fast feedback and being not limited to physical resources during tests

Major Concepts for AUTOSAR Continuous Delivery (CD)


29

Composition 1

SWC A

SWC B

Application Software Requirements

e.g. utilization boundaries for Compositions, Software Components or Runnables

Hardware Requirements

e.g. maximum core utilization

CPU Resource Requirements


R1

R2

R1

CPU: 30%

CPU: 20%

CPU: 10%

CPU: 10%

CPU: 5%

CPU: 15%

Max CPU Core Load

Core 1 Core 2 Core 3

80% 80%50%

30

Application Software Requirements

e.g. maximum execution time for Runnables

e.g. data ages for communication between Runnables

Operating System Service Requirements

e.g. Task response times

e.g. Start-to-Start durations

System Requirements

e.g. Event-Chains on critical dataflow / End-to-End times

Timing Requirements


31

In contrast to normal (non CD) testing …

Testing should cover many different Scenarios

e.g. Burst-modes with many service-requests

e.g. increased load scenario, when function amount increases

e.g. different execution modes

Fast test execution is required

Tests should run in hours (max. a day) instead of weeks

Simulation allows fast exploration

Objectives for Testing


32

1. Model-based Simulation

+ Not limited to hardware, therefore theoretical unlimited parallelization

+ For Stress-tests, system stimulation scenarios can be arbitrary extended in order to cover many situations

- Requires execution times for Runnables

2. Hardware Trace Measurements

+ Allows detailed analysis of error scenarios on unlimited level of granularity

+ Execution times can be measured

- Limited through physical hardware

Types of Timing & Performance Validation


33

Model-based SimulationModel-based

SimulationModel-based Simulation

Procedure


Hardware Measurement

Model-based SimulationModel-based

SimulationModel-based Simulation

Build

System Description

ECU-C

Test Report

Runnable Execution Times

Timing Report

(alternative) Supplement Runnable Execution Times to SysDescr

Parametrized random parameter generator

34

Hardware Measurement Overview


1. Configure MICROSAR Tracing Hooks (OS and RTE)

2. Configure iSYSTEM

3. Trace the ECU and export a BTF

4. Evaluate the Traces against Requirements

TA Tool Suite

TA.Inspection

Extraction of aScheduling Trace

BTF Scheduling Trace Import

Timing Report Generation

MICROSAR

winIDEA

iSYSTEM

Requirements InputiSYSTEM Profiler XMLfor OS Awareness

1.

2.

3. 4.

35

Example of Timing Report


Showing measurement, tool and input file information

Validating system requirements (meet/failed)

Showing further timing metrics for later evaluation

Inputs such as Task Stack Consumptioncan also be stored

36

Junit Reports for automated testing

Publish direct on the CI Server

Combination with other testing results in one overview

JUnit Reports and Continuous Timing Evaluation


Showing CPU Load average, as well as minimum and maximum value (in a x ms timeframe)

Extrapolation shows CPU Limit will potentially be reached within two releases

Performance improvements should be applied now, in order to prevent a blocking of delivery

37

Introduction




Summary & Outlook

Agenda

38

Summary & Outlook

Summary & Outlook

Conclusion

CI brings already a lot of benefits for the agile software development, especially for developers and integrators

Decision Frontloading enables fast integration

Independent local integration solves blocking

CD makes sense, when software is often released (e.g. sprint based, ~quarter based)

Automatic non-functional testing, e.g. timing or performance is essential

What’s Next?

Standardization of Timing Hardware Measurement (AUTOSAR ARTI)

Over-the-Air Updates (OTA)

39 © 2020. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. V0.3 | 2020-06-08

Author:Deubzer, MichaelVector Germany

For more information about Vectorand our products please visit

www.vector.com

http://www.vector.com/

Towards Continuous Software Development for …...Continuous Integration Continuous Delivery … as...

Documents

Transcript of Towards Continuous Software Development for …...Continuous Integration Continuous Delivery … as...