© HORIBA MIRA Ltd. 2017

© HORIBA MIRA Ltd. 2017

6th April 2017

The Challenge of Verifying Highly Automated Automotive Systems

Helen Monkhouse Commercial Manager Functional Safety

© HORIBA MIRA Ltd. 2017

Agenda

■ Automotive safety

■ Functional safety paradigm

■ How greater automation changes things

■ Safety of the intended functionality

■ Verification & validation

■ The challenges

2

Backdrop

Autonomy

V&V

Summary

April 6, 2017

© HORIBA MIRA Ltd. 2017

Backdrop

April 6, 2017 3

© HORIBA MIRA Ltd. 2017

Automotive “safety” evolution

April 6, 2017 4

Serious

Injury Accident

Critical

Situation

Safe

Situation

Passive

Safety Active

Safety

Tactical

Safety

© HORIBA MIRA Ltd. 2017

Automotive functional safety timeline

5

1980’s 1990’s 2000’s 2010’s 2020’s

Engine management

Restraints Stability control

Driving support

Increasing autonomy

Early IEC drafts

MISRA Guidelines

(ISO/TR 15497)

IEC 61508

(Edition 1)

ISO 26262

(start of work)

ISO 26262

(Edition 1)

ISO 26262

(Edition 2)

IEC 61508

(Edition 2)

ISO 26262

(Edition 3)?

April 6, 2017

© HORIBA MIRA Ltd. 2017

Functional safety paradigm

April 6, 2017 6

Functional Safety

Paradigm

Single Feature

Inputs Control Actuation

Item

© HORIBA MIRA Ltd. 2017

Functional safety paradigm

April 6, 2017 7

MISRA State Machine Model of

Automotive Risk

Functional Safety

Paradigm

Single Feature

Failures cause

Hazards

© HORIBA MIRA Ltd. 2017

Functional safety paradigm

April 6, 2017 8

Functional Safety

Paradigm

Single Feature

Failures cause

Hazards

Driver in the

Control Loop

MISRA Driver in the Loop

Vehicle Control Model

© HORIBA MIRA Ltd. 2017

Functional safety paradigm

April 6, 2017 9

Functional Safety

Paradigm

Single Feature

Failures cause

Hazards

Driver in the

Control Loop

Fail Silent

© HORIBA MIRA Ltd. 2017

Highly Automated Driving

April 6, 2017 10

© HORIBA MIRA Ltd. 2017

Achieving functional safety today

April 6, 2017 11

Demand Control Actuation Reasoning Perception

Driver Vehicle

Item boundary

© HORIBA MIRA Ltd. 2017

Achieving functional safety today

April 6, 2017 12

Demand Actuation

Control

Monitoring

Torque Clamp

Control

Hazard cause: Control error results in incorrect engine torque request

Hazard: Undemanded acceleration

Hazard Risk: ASIL B

Safety Goal: Avoid undemanded acceleration

Safe State: Apply torque clamp

Accelerator Pedal Position Engine Torque Request

Temperature

Road Gradient

Driving Mode

Etc.

Item boundary

© HORIBA MIRA Ltd. 2017

Achieving functional safety with autonomy

April 6, 2017 13

Demand Control Actuation Reasoning Perception

Vehicle

Item boundary

© HORIBA MIRA Ltd. 2017

Achieving functional safety with autonomy

April 6, 2017 14

Demand Reasoning Perception

Camera Data

Lidar Data

Navigation Data

Etc.

Vehicles

Pedestrians

Road Layout

Etc.

Vehicle Motion

Demand

Longitudinal

Acceleration

Demand

Hazard cause: May not result from malfunction

Hazard: ‘Unsafe’ acceleration

Hazard Risk: ASIL D (no driver in the loop)

Safety Goal: Avoid ‘unsafe’ acceleration

Safe State: ?

Item boundary

Hazard caused by:

• Direct consequence of the intended function

• Incorrect situational comprehension

• Situational misinterpretation

• Incorrect processing

• Over-simplistic algorithm specification

• Inadequate robustness to noise factors

• Insufficient function performance

© HORIBA MIRA Ltd. 2017

Achieving functional safety with autonomy Safety of the intended function (PAS 21448 current draft)

April 6, 2017 15

© HORIBA MIRA Ltd. 2017

Verification & Validation

April 6, 2017 16

© HORIBA MIRA Ltd. 2017

Verification & Validation Safety assurance evidence

April 6, 2017 17

Environment • Why do we have confidence in the environment in which the safety activities were

undertaken?

• What evidence demonstrates that the organisation has a good safety culture?

Means • Why do we have confidence that an adequate process has been used to develop

the work products?

• Which evidence demonstrates that the right people have used the correct

methods?

Satisfaction • Why do we have confidence that the requirements have been implemented

correctly?

• Which evidence demonstrates that the correct implementation has been verified?

Rationale • Why do we have confidence about requirement correctness?

• Which evidence indicates that the requirements are complete and correct?

A Layered Model for Structuring Automotive Safety Arguments

I Habli, J Birch, R Rivett, H Monkhouse, et al, EDCC, 2014

© HORIBA MIRA Ltd. 2017

Verification & Validation Classic safety assurance

■ Safety requirements describe deterministic

safety mechanisms

■ Safety validation testing (e.g. fault injection

testing) provides evidence that the

functionality is correct

■ Verification testing throughout the

development provides evidence that

implementation satisfies requirements

April 6, 2017 18

Control

Monitoring

Control

© HORIBA MIRA Ltd. 2017

Verification & Validation Safety assurance with autonomy

■ AREA 1 – Evaluate by Analysis

- Confidence that the function is correctly

defined and interactions with its

environment fully understood.

- Confidence in verification targets –

e.g. false negatives / positives

- Confidence in validation targets – e.g.

accident statistics, scenario simulation.

April 6, 2017 19

Demand Reasoning Perception

AREA 1

© HORIBA MIRA Ltd. 2017

Verification & Validation Safety assurance with autonomy

■ AREA 2 – Evaluate Known Use Cases

- Verifying correct sensor and actuator

functionality given potential

environmental factors (e.g. weather,

reflections)

- Verifying decision algorithm’s reasoning

and ability to avoid unwanted actions

- Verifying system controllability and

robustness assumptions

April 6, 2017 20

Demand Reasoning Perception

AREA 2

© HORIBA MIRA Ltd. 2017

Verification & Validation Safety assurance with autonomy

■ AREA 3 – Evaluate Unknown Use Cases

- Validating that perception sensors and

algorithms correctly model the

environment

- Validating that decision algorithms

correctly recognise and reason about

known and unknown situations

- Confidence regarding system

robustness

April 6, 2017 21

Demand Reasoning Perception

AREA 3

© HORIBA MIRA Ltd. 2017

Challenges

April 6, 2017 22

© HORIBA MIRA Ltd. 2017

Challenges

■ Highly automated systems break the current

functional safety paradigm, however some

principles of ISO 26262 can still be applied

■ Defining definitive verification targets may no

longer be realistic, with the definition of

statistically relevant verification targets being

required

■ Simulation and data analysis tools will be needed

to support verification and validation activities,

thus building confidence of safe system operation

in the environment

April 6, 2017 23

© HORIBA MIRA Ltd. 2017

Thank you

24 April 6, 2017

© HORIBA MIRA Ltd. 2017

Contact Details

25

HORIBA MIRA Ltd.

Watling Street,

Nuneaton, Warwickshire,

CV10 0TU, UK

T: +44 (0)24 7635 5000

F: +44 (0)24 7635 8000

www.horiba-mira.com

Helen Monkhouse BEng (Hons) CEng MIET MWES

Commercial Manager – Functional Safety

Direct T: +44 (0)24 7635 58110

E: helen.monkhouse@horiba-mira.com

April 6, 2017