ADAS/AD - NVIDIA · AVL DRIVE for ADAS KPI SC Vehicle Under Test Proving Ground very close to the...
Transcript of ADAS/AD - NVIDIA · AVL DRIVE for ADAS KPI SC Vehicle Under Test Proving Ground very close to the...
AVL List GmbH Tobias Düser | | 11 Oktober 2018 | 1Public
Technical• Functional safety & Reliability in all
conditions (Traffic, Weather & Road)• Infinite high number of test cases• Complex and new technology• Time & Cost pressure during development
Public relations• Public & politic reactions in
case of accidents• Customer acceptance• Cyber security• Ethic
Legislative• Homologation criteria• Responsibility in case of accident
The rocky road to
ADAS/ADDevelopment, Validation and Homologation
AVL List GmbH Tobias Düser | | 11 Oktober 2018 | 2Public
Level 5 Future
Now
Responsibility
Dri
ver’s
Lic
ense
Regula
tions
?
?
AVL List GmbH Tobias Düser | | 11 Oktober 2018 | 3Public
“The critical path to introduce autonomous driving vehicles will not be the technology but the development of a metric which empowers for an approval”
Prof. Dr. rer. nat. H. Winner, November 2013
A lot of progress concerning the
TECHNOLOGYis visible, but:
… and the next question will be how to execute scenarios, test cases etc. of this metric in a limited time
AVL List GmbH Tobias Düser | | 11 Oktober 2018 | 4Public
“The biggest hurdle is validation to confirm that the system does not cause failures. One has to execute 250 million test kilometers”
Bosch Executive Director.
Same approaches as 36 years ago?
Mercedes-Benz 190 E (1982) - Demonstration and advantages of ABS
New and Innovative Approaches in
Testing and Validation
are necessary!
AVL List GmbH Tobias Düser | | 11 Oktober 2018 | 5Public
Facing the Challenges of Development and Homologation of Autonomous Driving Using Virtual
Approaches
In cooperation with research partners
Driving Software and Digital Driver
Legislation, Regulations and
Conformity
Vehicle Validation and Self Driving
Vehicle
§
AVL List GmbH Tobias Düser | | 11 Oktober 2018 | 6Public
Background and Homologation
Dr. Houssem Abdellatif
Global Head Autonomous Driving and ADAS
Driving Test: Human vs. Machine
718-10-11TÜV SÜD Auto Service GmbH | GTC Europe 2018
Human Machine
Level 3:
Highly
automated
Level 4:
Fully
automated
Level 5:
Driverless
Level 0:
Driver Only L1: Assisted
L2: Partially
automated
!
Today
Responsibility
Tomorrow
http://www.onlinewahn.de/generator/f-maker.htm http://www.onlinewahn.de/generator/f-maker.htm
Homologation and Type Approval
818-10-11TÜV SÜD Auto Service GmbH | GTC Europe 2018
Self certification vs. type approval 3rd party principle
DefinitionHomologation refers to the certification process of a product (vehicle) granting that it complies with all local standards and legal regulations such as safety and environmental regulation.
No homologation No CoC No sales
Type
Approvalconcept
phase
vehicle development cycle
X
• European Union: Directive 2007/46/EC Type approval, tests are based on
United Nations Economic Commission for Europe (UN/ECE) procedures;
• North America: Federal Motor Vehicle Safety Standards (FMVSS)
regulations released by the NHTSA;
• Australian Design Rules (ADR) regulations;
• Japan follows UN/ECE regulations and their own Test Requirements and
Instructions for Automobile Standards (TRIAS) regulations;
• Other countries that accept or base their own regulation on those mentioned
above, following the latest release or previous versions of the regulations.
Type Approval in vehicle development• Last step of
development• Accomplishment of the
v-cycle• legal and technical
approval of the concept
UN/ECE-Regulation 140: Homologation of ESC Systems
918-10-11TÜV SÜD Auto Service GmbH | GTC Europe 2018
Sine with Dwell (SwD) maneuver
© Pearson Education, Ltd., Upper Saddle River, NJ 07458
Approval testing & collecting data
real TestsVehicle
response
Vehicle
responsesimulation
Inputs
Simulation
results
correlationmodel
adjustment
Model validationApproval & assessment of variants by
using validated simulation model
Inputs
N Simulation resultsN vehicle/model variants
„Where a vehicle has been physically tested in accordance with […],
the compliance of versions or variants of that same vehicle type may be demonstrated by a computer simulation […].”
Generate the vehicle simulation
and correlate the obtained data
02Define vehicle representative of
the type to be homologated, and
test it under dynamic maneuvers
01
Simulate critical variants
03
ESC vs. Automated Driving
1018-10-11TÜV SÜD Auto Service GmbH | GTC Europe 2018
Sense and control of internal vehicle
Sense and control in external environment
✓ Similar homologation / validation architecture:
▪ Environment
▪ Sensor – Function / active system – Vehicle (dynamics)
▪ (Driver)
✓ Methodology can be reused and extended to automated driving
Simulation aided / supported homologation of automated driving functions
vehicle dynamicsdriver
active system
environment
information
physics
perception
interpretation
reasoning
acting
executing
driver
vehicleInternal sensors
for vehicle states
vehicle dynamicsdriver
active system
environment
information
physics
information
perception
interpretation
reasoning
acting
executing
driver
vehicle
external sensors
paradigm shift
Combining Tools (Tool Chain) for Concise Approval
1118-10-11TÜV SÜD Auto Service GmbH | GTC Europe 2018
Scenario data base• description• parameters• fail/pass criteria (KPI)
subset for approval
subset for physical testing
subset for simulation testing
real vehicle
virtual vehicle
results: m vehicle variants X nscenario variants X …
test track
simulation
test bench
subset for testbench testing
Virtual-based Assessment for L3+AVs
1218-10-11TÜV SÜD Auto Service GmbH | GTC Europe 2018
Requirements
LegislationRegulations
Norms
AV System Specification
Assessment Database:
Scenario, KPIsand Criteria
Trustworthiness of Simulation via Validation
Tool ChainValidation
Vehicle Model Validation
World Model Validation
Sensor & PerceptionValidation
Evaluation
Methods
KPIs
CriteriaTesting
SimulationFramework
Proving GroundFieldTests
Assessment Selection
SafetyKnow-How
CriticalityAnalysis
FunctionalDecomposition
18-09-28
AV Assessment
Component System Vehicle
AVL List GmbH Tobias Düser | | 11 Oktober 2018 | 13Public
Virtual Testing Methods
Prof. Bernhard Schick
Professor for Vehicle Dynamics and Advanced Driver Assistance Systems
11.10.2018
14
GTC 2018
11.10.2018
GTC 2018
14
Positive driving experience remain important!
11.10.2018
15
GTC 2018
11.10.2018
GTC 2018
15
Object Hopping
Object Lost
Sudden System Drop-Out
1
2
3
11.10.2018
16
GTC 2018
Challenges
Subjects feel more stressed and unsaved with LKAS instead of without LKAS
Statistical Analysis >50 Measurements1
2
3
4
5
120 kph 160 kph
Subjectively perceived stress "with" versus "without" lane-keeping assistant
with LKAS early with LKAS late without LKAS
goo
db
ad
65 %
105 %
11.10.2018
17
GTC 2018
11.10.2018
GTC 2018
17
"The most important thing about a ADAS/AD
is that it is common used.
And it's only used when it's experienced as
safe, usefulness and ease of use.“
People want a perceived safety of 100%.
11.10.2018
18
GTC 2018
Paradigm change
Fun to be Driven Fun to DriveTestingSimulation
Team Play
Development Efficiency Driving Experience
How to transfer todays “fun-to-drive” into future “fun-to-be-driven” within a attribute driven development?
11.10.2018
19
GTC 2018
11.10.2018
GTC 2018
19
We need a very high level of environment knowledge and anticipatory understanding of the situation!
11.10.2018
20
GTC 2018
Rule based vs. artificial intelligence
How complex, reproducible and understandable are the methods?
Steering Angle
Folding LevelSensor Input
(Image)
Pooling Level Fully Connected Level Output Vector
ADAS/AD
FunctionSteering Torque
Maneuver InterpretationSensor Input
(Fused Sensors)Decision Tree Function Output Vector
11.10.2018
21
GTC 2018
11.10.2018
GTC 2018
21
Lateral control in complex environments!
• markings heavily weathered
• partly no center marking
• dirty roads
• no side lines in sections
• uneven road with inclination changes
• heavy vehicle response
11.10.2018
22
GTC 2018
11.10.2018
GTC 2018
22
Drift Weaving
Sudden System Drop-Out
1
2
11.10.2018
23
GTC 2018
Learning challenges of artificial intelligence
Real Ground Truth
Virtual Ground Truth
Detailing of labeling and the process to build up the simulation environment will have a effect to the results.
11.10.2018
24
GTC 2018
The Toolchain
Dr. Tobias Düser
Department ManagerAdvanced Solution Lab
AVL List GmbH Tobias Düser | | 11 Oktober 2018 | 25Public
A seamless but open toolchain with new approach is necessary
Simulation
Same Results
Virtual Testing Proving Ground Real World
Maneuver, Scenario, Test Case Abstraction incl. KPIs
Environment Abstraction
Vehicle Abstraction
The most efficient validation will be done by those who will use the best combination!
AVL List GmbH Tobias Düser | | 11 Oktober 2018 | 26Public
Reference: Proving Ground
NVIDIA DRIVE
AD Function
Vehicle Interface
Real Vehicle on Real Proving Ground
Real Vehicle on Real Proving Ground
Methodology, Test Catalogue and KPIsfor AD Function
AVL DRIVE for ADAS
KPI
SC
Vehicle Under Test
Proving Ground
very close to the real operation
expensive, high effort, less repeatability
AVL List GmbH Tobias Düser | | 11 Oktober 2018 | 27Public
Architecture Virtual Testing@ the DrivingCube
NVIDIA DRIVE
AD Function
Vehicle Interface
CAN Interface
TB Interface
AVL V
ECO
N
Real Vehicle on AVL DRIVINGCUBE™
Virtual Vehicle on Virtual ProvingGroundT
estb
ed.C
ON
NECT™
with M
odel.CO
NN
ECT™
NVIDIA DRIVE SIM
DRIVE SIM
Wrapper
Methodology, Test Catalogue and KPIsfor AD Function
AVL DRIVE for ADAS
DRIVE Interface
KPI
SC
Vehicle Under Test
Virtual Testing
close to real operation, chassis dynos are already established for homologation (emissions)
limited in terms of lateral dynamics
AVL List GmbH Tobias Düser | | 11 Oktober 2018 | 28Public
AVL List GmbH Tobias Düser | | 11 Oktober 2018 | 29Public
Architecture Simulation
NVIDIA DRIVE
AD Function
CAN Interface
VSM Wrapper
AVL V
SM
Virtual Vehicle in Vehicle Simulation
Virtual Vehicle on Virtual ProvingGroundM
odel.CO
NN
ECT™
-
NVIDIA DRIVE SIM
DRIVE SIM
Wrapper
Methodology, Test Catalogue and KPIsfor AD Function
AVL DRIVE for ADAS
DRIVE Interface
KPI
SC
Vehicle Under Test
Simulation
fast, flexible and cheap in operation, there are already ESC homologation
processes in simulation is only as good as the model(s)
AVL List GmbH Tobias Düser | | 11 Oktober 2018 | 30Public
First Results / First Comparison
0 5 10 15 20 25 30 35 40 45 50 55 60
Time / s
Velo
city T
SV /
(m
/s)
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Proving GroundVHiL Repetition 1
MiL
0 5 10 15 20 25 30 35 40 45 50 55 60
Time / s
Desir
ed A
ccele
ration /
(m
/s^
2)
-3,0
-2,5
-2,0
-1,5
-1,0
-0,5
0,0
0,5
1,0
1,5
2,0
2,5
3,0
Proving GroundVHiL Repetition 1
MiL
0 5 10 15 20 25 30 35 40 45 50 55 60
Time / s
Delta D
ista
nce /
m
0
5
10
15
20
25
30
35
40
45
50
Proving GroundVHiL Repetition 1
MiL
0 5 10 15 20 25 30 35 40 45 50 55 60
Time / s
Velo
city /
(m
/s)
-2
0
2
4
6
8
10
12
14
16
Proving GroundVHiL Repetition 1MiL
upper & lower limit
Riedmaier, S.; Nesensohn, J.; Gutenkunst, C.; Düser, T.; Schick, B.; Abdellatif, H.:Validation of X-in-the-Loop Approaches for Virtual Homologation of Automated Driving Functions, GSVF Symposium 2018, Graz
11.10.2018
31
GTC 2018
Validation and Homologation of Autonomous Driving will be a
challenge
Partnering is extremely important to join the forces
The most efficient validation and homologation will be done by those
who will use the best combination of different test environments
NVIDIA, TUEV SUED AND AVL will push this topic together
11.10.2018
32
GTC 2018
Source: https://de.upost.info/31383037343032393936
… we can skip plan B