The Challenge of Verifying Highly Automated Automotive Systems€¦ · Helen Monkhouse BEng (Hons)...
Transcript of The Challenge of Verifying Highly Automated Automotive Systems€¦ · Helen Monkhouse BEng (Hons)...
© HORIBA MIRA Ltd. 2017
© HORIBA MIRA Ltd. 2017
6th April 2017
The Challenge of Verifying Highly Automated Automotive Systems
Helen Monkhouse Commercial Manager Functional Safety
© HORIBA MIRA Ltd. 2017
Agenda
■ Automotive safety
■ Functional safety paradigm
■ How greater automation changes things
■ Safety of the intended functionality
■ Verification & validation
■ The challenges
2
Backdrop
Autonomy
V&V
Summary
April 6, 2017
© HORIBA MIRA Ltd. 2017
Backdrop
April 6, 2017 3
© HORIBA MIRA Ltd. 2017
Automotive “safety” evolution
April 6, 2017 4
Serious
Injury Accident
Critical
Situation
Safe
Situation
Passive
Safety Active
Safety
Tactical
Safety
© HORIBA MIRA Ltd. 2017
Automotive functional safety timeline
5
1980’s 1990’s 2000’s 2010’s 2020’s
Engine management
Restraints Stability control
Driving support
Increasing autonomy
Early IEC drafts
MISRA Guidelines
(ISO/TR 15497)
IEC 61508
(Edition 1)
ISO 26262
(start of work)
ISO 26262
(Edition 1)
ISO 26262
(Edition 2)
IEC 61508
(Edition 2)
ISO 26262
(Edition 3)?
April 6, 2017
© HORIBA MIRA Ltd. 2017
Functional safety paradigm
April 6, 2017 6
Functional Safety
Paradigm
Single Feature
Inputs Control Actuation
Item
© HORIBA MIRA Ltd. 2017
Functional safety paradigm
April 6, 2017 7
MISRA State Machine Model of
Automotive Risk
Functional Safety
Paradigm
Single Feature
Failures cause
Hazards
© HORIBA MIRA Ltd. 2017
Functional safety paradigm
April 6, 2017 8
Functional Safety
Paradigm
Single Feature
Failures cause
Hazards
Driver in the
Control Loop
MISRA Driver in the Loop
Vehicle Control Model
© HORIBA MIRA Ltd. 2017
Functional safety paradigm
April 6, 2017 9
Functional Safety
Paradigm
Single Feature
Failures cause
Hazards
Driver in the
Control Loop
Fail Silent
© HORIBA MIRA Ltd. 2017
Highly Automated Driving
April 6, 2017 10
© HORIBA MIRA Ltd. 2017
Achieving functional safety today
April 6, 2017 11
Demand Control Actuation Reasoning Perception
Driver Vehicle
Item boundary
© HORIBA MIRA Ltd. 2017
Achieving functional safety today
April 6, 2017 12
Demand Actuation
Control
Monitoring
Torque Clamp
Control
Hazard cause: Control error results in incorrect engine torque request
Hazard: Undemanded acceleration
Hazard Risk: ASIL B
Safety Goal: Avoid undemanded acceleration
Safe State: Apply torque clamp
Accelerator Pedal Position Engine Torque Request
Temperature
Road Gradient
Driving Mode
Etc.
Item boundary
© HORIBA MIRA Ltd. 2017
Achieving functional safety with autonomy
April 6, 2017 13
Demand Control Actuation Reasoning Perception
Vehicle
Item boundary
© HORIBA MIRA Ltd. 2017
Achieving functional safety with autonomy
April 6, 2017 14
Demand Reasoning Perception
Camera Data
Lidar Data
Navigation Data
Etc.
Vehicles
Pedestrians
Road Layout
Etc.
Vehicle Motion
Demand
Longitudinal
Acceleration
Demand
Hazard cause: May not result from malfunction
Hazard: ‘Unsafe’ acceleration
Hazard Risk: ASIL D (no driver in the loop)
Safety Goal: Avoid ‘unsafe’ acceleration
Safe State: ?
Item boundary
Hazard caused by:
• Direct consequence of the intended function
• Incorrect situational comprehension
• Situational misinterpretation
• Incorrect processing
• Over-simplistic algorithm specification
• Inadequate robustness to noise factors
• Insufficient function performance
© HORIBA MIRA Ltd. 2017
Achieving functional safety with autonomy Safety of the intended function (PAS 21448 current draft)
April 6, 2017 15
© HORIBA MIRA Ltd. 2017
Verification & Validation
April 6, 2017 16
© HORIBA MIRA Ltd. 2017
Verification & Validation Safety assurance evidence
April 6, 2017 17
Environment • Why do we have confidence in the environment in which the safety activities were
undertaken?
• What evidence demonstrates that the organisation has a good safety culture?
Means • Why do we have confidence that an adequate process has been used to develop
the work products?
• Which evidence demonstrates that the right people have used the correct
methods?
Satisfaction • Why do we have confidence that the requirements have been implemented
correctly?
• Which evidence demonstrates that the correct implementation has been verified?
Rationale • Why do we have confidence about requirement correctness?
• Which evidence indicates that the requirements are complete and correct?
A Layered Model for Structuring Automotive Safety Arguments
I Habli, J Birch, R Rivett, H Monkhouse, et al, EDCC, 2014
© HORIBA MIRA Ltd. 2017
Verification & Validation Classic safety assurance
■ Safety requirements describe deterministic
safety mechanisms
■ Safety validation testing (e.g. fault injection
testing) provides evidence that the
functionality is correct
■ Verification testing throughout the
development provides evidence that
implementation satisfies requirements
April 6, 2017 18
Control
Monitoring
Control
© HORIBA MIRA Ltd. 2017
Verification & Validation Safety assurance with autonomy
■ AREA 1 – Evaluate by Analysis
- Confidence that the function is correctly
defined and interactions with its
environment fully understood.
- Confidence in verification targets –
e.g. false negatives / positives
- Confidence in validation targets – e.g.
accident statistics, scenario simulation.
April 6, 2017 19
Demand Reasoning Perception
AREA 1
© HORIBA MIRA Ltd. 2017
Verification & Validation Safety assurance with autonomy
■ AREA 2 – Evaluate Known Use Cases
- Verifying correct sensor and actuator
functionality given potential
environmental factors (e.g. weather,
reflections)
- Verifying decision algorithm’s reasoning
and ability to avoid unwanted actions
- Verifying system controllability and
robustness assumptions
April 6, 2017 20
Demand Reasoning Perception
AREA 2
© HORIBA MIRA Ltd. 2017
Verification & Validation Safety assurance with autonomy
■ AREA 3 – Evaluate Unknown Use Cases
- Validating that perception sensors and
algorithms correctly model the
environment
- Validating that decision algorithms
correctly recognise and reason about
known and unknown situations
- Confidence regarding system
robustness
April 6, 2017 21
Demand Reasoning Perception
AREA 3
© HORIBA MIRA Ltd. 2017
Challenges
April 6, 2017 22
© HORIBA MIRA Ltd. 2017
Challenges
■ Highly automated systems break the current
functional safety paradigm, however some
principles of ISO 26262 can still be applied
■ Defining definitive verification targets may no
longer be realistic, with the definition of
statistically relevant verification targets being
required
■ Simulation and data analysis tools will be needed
to support verification and validation activities,
thus building confidence of safe system operation
in the environment
April 6, 2017 23
© HORIBA MIRA Ltd. 2017
Thank you
24 April 6, 2017
© HORIBA MIRA Ltd. 2017
Contact Details
25
HORIBA MIRA Ltd.
Watling Street,
Nuneaton, Warwickshire,
CV10 0TU, UK
T: +44 (0)24 7635 5000
F: +44 (0)24 7635 8000
www.horiba-mira.com
Helen Monkhouse BEng (Hons) CEng MIET MWES
Commercial Manager – Functional Safety
Direct T: +44 (0)24 7635 58110
April 6, 2017