ART OF AUTONOMOUS DRIVING VALIDATION...A game engine would give a near real-word environments, which...
Transcript of ART OF AUTONOMOUS DRIVING VALIDATION...A game engine would give a near real-word environments, which...
Art of Autonomous Driving Validation
[email protected] © Tata Elxsi 2019 1
ART OF AUTONOMOUS DRIVING VALIDATION
Chanjal Prakash, Tata Elxsi
Art of Autonomous Driving Validation
[email protected] © Tata Elxsi 2019 2
TABLE OF CONTENTS
ABSTRACT ...................................................................................................................................................... 3
INTRODUCTION ............................................................................................................................................. 4
CHALLENGES IN IMPLEMENTING AUTONOMOUS DRIVING SOLUTIONS ..................................................... 7
THE NEED FOR PRECISE VISION..................................................................................................................... 9
CONCLUSION ............................................................................................................................................... 10
ABOUT TATA ELXSI ...................................................................................................................................... 11
REFERENCES ................................................................................................................................................ 12
Art of Autonomous Driving Validation
[email protected] © Tata Elxsi 2019 3
ABSTRACT Autonomous driving planning and control algorithms have to ensure high levels of safety on all scenarios for which they are exposed to. Autonomous driving systems rely upon sensors and embedded software for localization, perception, motion planning and execution. Hence the authenticity is put on test by a large number of scenarios such as the non-deterministic behavior of traffic participants and by inaccurate sensors and actuators. Billions of miles of road testing would be required to validate the safety of the autonomous driving systems & its algorithms. This paper explains how these challenges are addressed by creating simulated environment using gaming technology. This method helps in generating photo realistic 3D simulation which matches the real camera images/video. Using this simulation platform, sensor simulation can be integrated into a closed-loop simulation environment that interacts with traffic simulators, enabling thousands of driving scenarios to be executed virtually. This not only helps in reducing the total effort and time required for realizing autonomous features but also ensures end-to-end safety in deep-learning-based autonomous driving systems.
Art of Autonomous Driving Validation
[email protected] © Tata Elxsi 2019 4
INTRODUCTION
A study prepared by Strategy Analytics predicts that autonomous vehicles will generate a $7 trillion revenue stream by 2050 while saving an estimated 585,000 lives per decade and freeing more than 250 million hours of commuting time per year. The greatest problem that needs to be tackled before the introduction of Level 4 or Level 5 autonomous. There could be an endless number of scenarios that may be a hindrance to the vehicle such as any pedestrian, animal, vehicle or any other objects that could appear abruptly on the road, change of weather, lighting, haze, etc. which could obstruct the field of view of sensors.
Figure 1: Autonomous Driving Market Outlook Source: Frost & Sullivan
Art of Autonomous Driving Validation
[email protected] © Tata Elxsi 2019 5
Currently, autonomous driving systems address these challenges with machine learning/deep learning software that can be trained to recognize patterns without having to be specifically programmed for every possible situation that could arise. Since, automotive manufacturers are developing advanced driver assisted system, because of the safety and reliability offered by autonomous driving technology. This would ultimately leads to fully autonomous driving.
Figure 2: Levels of autonomous driving : Source Birmingham
Taking the driver out of the control loop creates major challenges in validating the capabilities of the sensors that are now responsible for the perception of the driving environment. As the complexity of driver assistance systems increases, so does the required effort for testing and evaluation. Exhaustive physical vehicle tests for validation would be economically infeasible.
Advanced Driver Assisted System (ADAS) algorithms, currently are validated using the software V-cycle
defined by ISO 26262. The autonomous driving needs closed-loop simulation where the virtual vehicle is
driven in a realistic virtual world by real autonomous vehicle algorithms. Creating simulation environment
of autonomous vehicle in a very detailed manner impose enormous challenge that requires ultimate
precision over various elements:
The characteristics, state, behaviors and mechanical properties of the vehicle
Obstacles and other vehicles
Road and its various conditions
The environmental conditions, visibility, properties of the material, climatic conditions and road
signage
Art of Autonomous Driving Validation
[email protected] © Tata Elxsi 2019 6
EASE OF VALIDATIONON A SIMULATED ENVIRONMENT
Figure 3: Challenges of testing in simulation or the physical world. Testing in simulation requires determining
In-short, validating algorithms in a simulated environment could limit the cost of autonomous driving
systems. Major challenges that may arise for test planning and control systems are:
Need to identify all the relevant scenarios to be tested
Several millions of test kilometers need to be driven
Issues need to be reproduced to solve them
Fixed issues need repetitive test, which increases the expense
Art of Autonomous Driving Validation
[email protected] © Tata Elxsi 2019 7
CHALLENGES IN IMPLEMENTING AUTONOMOUS
DRIVING SOLUTIONS In real-world validation and testing of autonomous driving solutions, the test vehicle with the AD
algorithm is driven through various scenarios and conditions in the real world, for long distances and
durations, in order to collect data for training and testing the AD algorithm as well as to analyze its
performance. This approach has drawbacks to it, which are described below.
Artificial Intelligence-based autonomous driving algorithms in these vehicles, need to drive
through millions of kilometers, even then the volume of scenarios to the vehicle needs to be tested
is not practical to create.
These scenarios must include traffic conditions of various intensity, different road structure and
surface types, dynamic weather phenomenon, variations in temperature and lighting and the day-
night cycle. The creation of such scenarios in a real vehicle in the real world would be expensive
and time-consuming.
The test vehicle should have a dynamic interaction with various other actors and events, like road
accidents, road conditions, construction works, pedestrians and animals etc. Creating these
scenarios in real-world is enormously challenging.
Since the autonomous vehicle is exposed to real-world without actual driver, there is a huge
amount of health and safety risk which needs to be considered. Figure 3 shows the frequency of
disengagements in AD test vehicles and hence the risk they pose during real-world testing.
Art of Autonomous Driving Validation
[email protected] © Tata Elxsi 2019 8
Figure 4 Number of AD disengagements in vehicles tested in California from Dec 2017 to Nov 2018
The creation of edge conditions where the AD vehicle is about to collide with a pedestrian or with
another closed-loop would have obvious risks. If Ad technology were to fail or disengage, it could
lead to catastrophic outcomes.
The faults obtained during the validation of autonomous vehicles need to be reproduced and after
fixing the fault, the same scenario validation has to be repeated. This is a direct increase in the
effort and time required for validation.
To train and validate scenarios in a fleet of the autonomous vehicle, a huge number of scenarios
need to be created and these have to be repeated multiple times, which is economically infeasible.
Figure 4 compares the miles per disengagement for AD test vehicles in California during the period
from December 2017 to November 2018. This comparison along with the number of
disengagements from Figure 3 provides us with an estimate of the amount of distance that was
covered for the testing of these vehicles.
0
100
200
300
400
500
600
700
800
900
1000
1100
1200
1300
Art of Autonomous Driving Validation
[email protected] © Tata Elxsi 2019 9
Figure 5. Miles per disengagement for AD vehicles tested in California from Dec 2017 to Nov 2018
THE NEED FOR PRECISE VISION There are many development steps involved for performing simulated tests in the automotive industry.
Early tests are performed as SIL (Software In the Loop)-tests. SIL-tests represent the environment of the
tested software as a software simulation. This allows early dynamic testing of the software component.
HIL (Hardware In the Loop)-tests add more realism by executing parts of the system on real hardware.
A more practical solution to Autonomous Driving validation and testing would be to train, test and validate
these algorithms in a virtual vehicle in a simulated environment. This will enable the creation of test
scenarios to be created in a controlled virtual world, which would otherwise be impossible to recreate in
the real world.
A game engine would give a near real-word environments, which can be used to create multiple scenarios.
This ground truth data which is much more reliable, can be used in a closed loop validation of autonomous
driving algorithms in which a virtual vehicle can be exposed to.
This closed-loop simulation approach could generate virtual world model and driving scenarios, physically
accurate sensor models, autonomous driving algorithms and vehicle dynamics. This allows virtual testing
in HiL, SiL, ViL, PiL, MiL and DiL applications.
Art of Autonomous Driving Validation
[email protected] © Tata Elxsi 2019 10
CONCLUSION Autonomous driving which is mostly depended upon artificial intelligence, needs to be tested rigorously well before it is exposed to the public. Validation of AD algorithms is not only proven to consume enormous effort and time but also possess health and safety risk elements. This is due to various elements such as huge amount of data required, scenarios which need to be created & recreated in the real-world, rare case generation etc. These challenges may be overcome by making use of a game engine technology to create near real world scenarios and vehicle dynamics. This enables rendering of 3D simulation, which helps attain higher efficiency and accuracy. Sensor models and vehicle dynamics, which is simulated would reduce the real world risk to minimal. This results in an accelerated, economical, efficient, accurate and risk-free solution to AD testing and validation The autonomous driving validation suite developed by Tata Elxsi, helps to create simulated environment and vehicle dynamics. It helps OEM’s and Tier 1s in saving effort and time for AV testing with paramount precision. The solution has the ability to automatically create scenarios based on a set of rules or parameters defined by the user. The tool does so using a generation algorithm, which is fine-tuned so that each generated scenario is logical, realistic and unique. The solution facilitates realistic and ideal sensor modeling of multiple sensors types – Ultrasonic, RADAR, LIDAR, Camera, GPS, IMU and Wheel-tick.
Art of Autonomous Driving Validation
[email protected] © Tata Elxsi 2019 11
ABOUT TATA ELXSI
Tata Elxsi is a global design and technology services company, headquartered in Bangalore. It addresses the automotive, broadcast and communications, consumer electronics and healthcare industries. This is supported by a network of design studios, development centers, and offices worldwide.
Tata Elxsi works with leading OEMs and suppliers in the automotive and transportation industries for R&D, design and product engineering services from architecture to launch and beyond. It brings together domain experience across Infotainment, Autonomous Driving, Telematics, Powertrain, and Body electronics, along with technologies such as Artificial Intelligence, Analytics, Cloud and IoT.
Tata Elxsi has been investing in research and development for Advance Driver Assisted Systems and
Autonomous Driving for over a decade and have developed multiple solutions which accelerates the
realization of autonomous driving system.
Art of Autonomous Driving Validation
[email protected] © Tata Elxsi 2019 12
REFERENCES
1. Accelerating the Future: The Economic Impact of the Emerging Passenger Economy, Roger
Lanctot, June 2017
2. The Last Driving License Holder, Mario Herger, December 2019
3. Will autonomous cars change the role and value of public transportation?, Freemark, 2017
The Transport Politic
4. Autonomous Driving, McKinsey Center of Future mobility, 2019
5. An Efficient Method for Testing Autonomous Driving Software Against Nondeterministic
Influences, Pascal Marcel Minnerup, May 2017
6. Automotive Software Engineering, 5th ed. Wiesbaden: Springer Fachmedien, J. Schauffele and T.
Zurawka, 2013