www.tassinternational.com

MODEL BASED INTELLIGENT VEHICLE SYSTEMS DEVELOP & VALIDATE METHODOLOGY 基于模型的智能驾驶系统的开发和测试方法

TASS International China Ltd 天欧汽车工程软件(上海)有限公司

Confidential

Contents 内容

• Short introduction to TASS International

TASS International公司简单介绍

• Challenges in intelligent vehicle systems development & evaluation

智能驾驶系统开发与测试的挑战

• Model based development & evaluation methodology

开发和测试方法论：基于模型的仿真方法

- SiL, software-in-the-loop

- HiL, hardware-in-the-loop

- ViL, vehicle-in-the-loop

- Indoor ADAS test facility VeHIL

- ADAS and AD test track

- Road test

3

Confidential

Short introduction to TASS International TASS International公司简单介绍

20-12-2016 4

Confidential

Introduction to TASS TASS简介

TASS International is one of TNO companies, and TASS International Co. Ltd is located in Shanghai

TASS International 是TNO的全资子公司，总部位于荷兰，TASS 中国分公司(天欧)位于上海

5

Independent Dutch Public R&D organization

Focus on a wide variety of themes / topics

5000 employees

5 commercial TNO spin-off companies

100% focus on transportation industry

200 employees

Global presence through 12 Offices

Consolidation and integration into commercial products/services

Global sales & support organization

Research

Innovative solutions

Confidential

International TASS Offices TASS 全球分公司

6

Yokohama, Japan

Wheelers Hill, Vic. Australia

Coventry, United Kingdom

Detroit, USA

Seoul, Korea

Rijswijk + Helmond,

The Netherlands

Paris, France

Frankfurt, Germany

Shanghai, China

Bangalore, India

São Paulo, Brazil

Confidential 7

Products 产品线

Crash lab

VeHIL active safety lab

DITCM intelligent driving facility

Tire testing

Powertrain lab

New energy lab

Homologations

Crash safety engineering

ADAS HiL development

ADAS system development

Intelligent driving research

Chassis control development

Engine emission development

Engineering & Solutions 工程开发

Testing & Services 测试服务

Software Products 仿真软件

Confidential 8

Passive, active and real-world safety

Connected driving (V2X) for safety and mobility

Powertrains: diesel, hybrid and electric

Products MADYMO PreScan Delft-Tyre Services Crash Lab – Safety Center Active safety lab (VeHIL) EEMC lab Homologations Engineering

Products SIMCAT Services Powertrain lab EEMC lab Homologations Engineering

Products PreScan SIMCAT Services DITCM test bed VeHIL Engineering

Products 产品线

Confidential

Challenges in intelligent vehicle systems

development & evaluation 智能驾驶系统开发与测试的挑战

20-12-2016 10

Confidential

The testing challenge 智能车辆测试的挑战

13

Question:

“How many kilometers of driving would it take to prove autonomous vehicle reliability?”

“需要多少公里数的路试，才能验证自动驾驶车辆的可靠性?”

Confidential

The testing challenges 智能车辆测试的挑战

“Unlimited” number of possible scenarios to deal with 测试里程和工况场景，几乎是无限的

System-critical situations and variant rarely happen 严苛工况及全面工况，在道路测试中可遇不可求

Scenarios often not reproducible 即使能遇到严苛工况，也不可重现，无法重复验证

“Ground-truth data” often unknown/inaccurate 在道路上，缺乏真值数据，来对比检验系统性能

Scenarios often too dangerous/complicated to test 部分测试工况，具有危险性，复杂性

15

Confidential

Our Answer: Model Based Design & Test 我们的建议 : 基于模型仿真的设计和测试流程

16

1. Concept phase 5. Validation

2. Design 4. Verification

3. Implementation

• TASS International toolsuite: TASS仿真和测试工具链 • Reduces development lead time 加快开发进度 • Improves product quality 提高产品性能

Confidential

PreScan is physics-based simulation platform that is used in the automotive industry for development of Advanced Driver Assistance Systems (ADAS), that are based on sensor technologies such as radar, laser/lidar, camera and GPS. PreScan is also used for designing and evaluating vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication applications

PreScan是以物理模型为基础的汽车智能驾驶仿真软件，支持基于摄像头、雷达、激光雷达、GPS、V2V车车通讯等多种高级驾驶辅助功能、智能驾驶功能的开发和测试。

18

PreScan PreScan软件 http://v.youku.com/v_show/id_XNTA4OTkzOTg0.html?from=y1.7-1.2

Confidential

Intelligent driving systems 智能驾驶系统的一般构成

19

ADAS systems

Adaptive Cruise Control (ACC)

Forward Collision Warning (FCW)

Collision Mitigation Braking (CMB)

Lane Departure Warning (LDW)

Blind Spot Warning (BLSW)

Lane Keeping Assistance (LKA)

Pedestrian Avoidance (PA)

Intelligent Headlight Systems (IHS)

Cooperative Driving Systems (CDS)

… …

Scenarios 场景工况

1 Algorithms 算法

3 Sensors & V2X 传感器

2 Actuators & HMI 执行器&人机界面

4

Confidential

ADAS systems in PreScan simulation

Adaptive Cruise Control (ACC)

Forward Collision Warning (FCW)

Braking (CMB)

Lane Departure Warning (LDW)

Blind Spot Warning (BLSW)

Lane Keeping Assistance (LKA)

Pedestrian Avoidance (PA)

Intelligent Headlight Systems (IHS)

Cooperative Driving Systems (CDS)

… …

Intelligent driving systems in PreScan 智能驾驶系统在PreScan中的仿真实现

20

Scenarios 场景工况

1 Algorithms 算法

3 Sensors & V2X 传感器

2 Actuators & HMI 执行器&人机界面

4

Traffic element database 128 scenario demos

Euro NCAP ×8 NHTSA ×12 ISO ×19 ADAC ×9 Real life ×48 ACC scenario ×18 AEB scenario ×15 LDW scenario ×27 TSR scenario ×7 PA scenario ×2 V2X scenario ×10 ……

19 sensor types Mono camera Stereo camera Fisheye camera Radar Lidar Ultrasonic Antenna Radio DSRC V2X Lane marker sensor Object sensor Depth sensor

Vehicle dynamics model PreScan model Coupling to 3rd party model ………

Matlab Simulink

Confidential

SiL HiL ViL VeHIL Test

Track Public Roads

1，PreScan software-in-the-loop

PreScan虚拟仿真SiL

24

Development & evaluationmethodology in

model based simulation 开发和测试方法论：基于模型的仿真方法

Confidential

Example 1: Run massive simulation with PreScan(European OEM) 案例1：大规模工况自动化仿真测试(欧洲某OEM)

Scenarios 场景工况

1 Algorithms 控制算法

3 Sensors & V2X 传感器&通讯

2 Actuators & HMI 执行器&人机界面

4

Massive Scenarios Inputs(thousands?) 大规模场景工况输入(成千上万种工况)

Regulation scenarios ISO

NHTSA NCAP

…

Accident database GIDAS CIDAS

…

Recorded scenarios Road test data IBEO scanned

…

Critical scenarios OEM database

Supplier database …

Use massive scenarios and test automation to validate systems in batch and effective model, to select most critical cases for road test

使用大规模测试工况库，利用自动化测试功能，高效验证系统和算法。

Confidential

Engineer client

Run batch command

Select worker node

Run massive experiments

1

2

3

~ 2.5 minutes per experiment with a 10 node cluster ~ 240 tests per hour ~ 3000 tests per night ~ 15000 tests per weekend

Example: Run massive simulation in cluster 大规模工况自动化仿真测试：高性能服务器方案

Confidential

Q: how to get LDW parameter limit in the specification according to certain vehicle mounting and alignment ? Or , find the best alignment to get optimized limit?.

问题：如何在光学、成像、算法、安装位置等诸多参数中，寻求最优的参数组合，实现LDW最佳性能？

30

Example 2: LDW performance analysis(Janpanese OEM) 案例2：LDW摄像头最优参数分析(某日系OEM)

光学 成像 算法

安装位置

检测精度 弯道半径极限 车道宽度极限

最优产品参数配置

Confidential

PreScan SiL Example 仿真案例 Camera LDW/LKA ：摄像头LDW/LKA(车道保持辅助)仿真

31

http://v.youku.com/v_show/id_XNTEwNjY4NTg4.html?from=y1.7-1.2

Confidential

Q: How to optimize ACC comfort considering the performance of each individual component - radar sensor, ACC algorithm and ECU, brake and engine actuation, as well as chassis and tire dynamics?

问题：把ACC系统分解为四个部分：传感器/控制算法/电子制动/车辆底盘，如何分配各部分的响应和延迟等参数特性，来优化ACC系统间减速过程中的舒适性(jerk)？

32

PreScan SiL Example 3: ACC comfort PreScan软件在环仿真案例3：ACC舒适性分析(自主OEM)

Detection delay False positive False negative Detection of cu tin & cut out

ACC algo Parameter tuning

Acceleration profile Deceleration profile

Chassis dynamics Tire dynamics

Radar 雷达 ACC控制器和算法 Actuation 执行器 底盘/轮胎 Chassis/tire

Confidential

PreScan SiL Example 仿真案例 Radar ACC and TJA：雷达ACC(自适应巡航)和TJA(堵车辅助)仿真

33

http://v.youku.com/v_show/id_XNTA5MTA4MDQw.html?from=y1.7-1.2

Confidential

SiL HiL ViL VeHIL Test

Track Public Roads

2，PreScan硬件在环台架HiL

PreScan hardware-in-the-loop

35

Development & evaluation methodology in

model based simulation

开发和测试方法论：基于模型的仿真方法

Confidential

HW/SW architecture Honda LKA HiL setup

36

LDW/FCW HiL Example 4: camera in the loop(Magna) PreScan hardware-in-the-loop：硬件在环仿真试验台案例4(Magna)

Confidential

HW/SW architecture Honda LKA HiL setup

37

LDW/FCW HiL Example 4: camera in the loop(Magna) PreScan hardware-in-the-loop：硬件在环仿真试验台案例4(Magna)

Confidential

HW/SW architecture Honda LKA HiL setup

RT PC PC

Vehicle Dynamics

Power steering logic

Camera Sensor model

CAN

Video output

Vehicle x,y,θ

Camera Image

@ Vehicle x, y, θ

Vehicle speed ↑

Yaw rate

Steering torque

Trajectory

x,y,θ

Steering angle

Vehicle speed x,y,θ

Steering angle, vehicle speed

Front wheel angle

Vehicle speed

Vehicle x,y,θ

Vehic

le s

peed

+ y

aw

rate

Steering angle

Vehicle speed

Steering

torque

Driver model

World Scenarios

Environment info

LKA Camera ECU

CAN

CAN

PreScan LKA HiL Example 5(Japanese OEM/tier) 案例5 : 车道保持辅助HiL (日系OEM/供应商)

Confidential

Q: how can OEM/tier fine tune steering ADAS for various lane curvature and width?

问题：如何充分测试LKA在不同的车道工况下，车道保持的最优性能呢？

39

海量LKA工况组合

不同宽度车道 不同转弯半径 各种车道组合 多种标线形态 高效仿真测试

PreScan LKA HiL Example 5(Japanese OEM/tier) 案例5 : 车道保持辅助HiL (日系OEM/供应商)

Confidential 41

AEB HiL Example 6: AEB ECU the loop(Daimler) 示例：AEB HiL案例6(Daimler)

Confidential

PreScan Example 仿真案例 Radar AEB-Pedestrian：雷达AEB-行人仿真

42

http://v.youku.com/v_show/id_XNTA5MTA4MDQw.html?from=y1.7-1.2

Confidential

World simulation Application simulation

Message encoding

Geo-networking algorithm

802.11p message scheduling

Radio transmitter

Message data (e.g road works message)

Vehicle/infrastructure GPS

Vehicle GPS or attenuation from radio channel layer

Complete customer on-board unit

RF signal attenuated RF signal

ADAS functional output

TASS Communication unit (e.g. alix)

Vehicle GPS

Octobox Octobox

Example 7, V2X HiL(hardware in the loop) 案例7: V2V硬件在环测试台

44

Confidential

HW/SW architecture Honda LKA HiL setup

Example 7, V2X HiL(hardware in the loop) 案例7: V2V路口碰撞预警功能测试

45

Confidential

SiL HiL ViL VeHIL Test

Track Public Roads

3，PreScan车辆在环台架ViL

PreScan vehicle-in-the-loop

46

Development & evaluationmethodology in

model based simulation

开发和测试方法论：基于模型的仿真方法

Confidential

Real Vehicle

HW/SW architecture Honda LKA HiL setup

PreScan PC @ 100Hz

World scenarios

Intervention acc./dec. request

Radar model

Target information

Target detection

Radar (radar antenna is bypassed so only ACC&AEB logic is tested)

Echo target detection

47

Vehicle velocity

Yaw rate

Steering angle

Vehicle Chassis/Brake/Engine actuation response to

ACC/AEB intervention request

Vehicle motion message

ACC HMI

chassis dyno & Open road

Radar ViL(vehicle-in-the-loop) Example 8 (China OEM) 雷达ACC/AEB车辆在环仿真试验台案例8 (自主某OEM)

CAN

Confidential

Q: How to run massive scenarios test in an effective way as an OEM, to evaluate and accept ACC/AEB systems delivered by supplier, rather than time consuming road test?

问题：供应商交付ACC/AEB的功能样件后，如何高效的开展大规模测试，找出性能上的问题，要求整改？

48

ACC/AEB ViL Example 8: radar and vehicle in the loop PreScan车辆在环仿真案例8：ACC/AEB性能评估

Confidential

Benefits and Conclusion 系统应用及结论

• Controllability, repeatability of running critical scenarios

• 可控、重复、批量化运行路试中难以遇到的场景

• VIL helps to find the boundary condition of the ADAS system

• VIL 可以帮助用户找出系统算法的临界值

• For example, what is the minimum curve radius that ACC system is able to follow under certain speed?

• 例如：VIL可以方便的测试出ACC系统弯道下的最小跟车半径

• VIL helps user to find the real reason behind a “strange” behavior and make improvements to the system

• VIL可以帮助用户找出系统问题并优化系统算法

• For example, ACC is likely to lose the target when entering into a curve, and how long it takes to get back to the target? Will this cause an impact?

• 例如：ACC系统在进入弯道时经常会出现目标丢失情况，丢失时间会持续多久，何种车速/HWT，目标丢失会导致系统加速从而发生碰撞或者触发CIB?

54

Confidential

The testing challenge 智能车辆测试的挑战

“Unlimited” number of possible scenarios to deal with 测试里程和工况场景，几乎是无限的

System-critical situations and variant rarely happen 严苛工况及全面工况，在道路测试中可遇不可求

Scenarios often not reproducible 即使能遇到严苛工况，也不可重现，无法重复验证

“Ground-truth data” often unknown/inaccurate 在道路上，缺乏真值数据，来对比检验系统性能

Scenarios often too dangerous/complicated to test 部分测试工况，具有危险性，复杂性

55

利用仿真软件提供道路“真值”数据，作为客观参照值，可以定量测试系统性能； HiL类台架相比路试，测试效率得到提高 HiL and ViL setup offers advantages that road test could not make

Confidential

SiL HiL ViL VeHIL Test

Track Public Roads

4，室内主动安全试验室VeHIL

Indoor ADAS test facility VeHIL

56

Development & evaluationmethodology in

model based simulation

开发和测试方法论：基于模型的仿真方法

Confidential

VeHIL facility, system assessment(FAW H7) 车辆在环智能驾驶试验室案例9: ACC系统级测试(FAW红旗)

58

Vehicle-Hardware-In-the-Loop (in short: VeHIL) laboratory, we offer a unique environment for the development and testing of intelligent vehicles and transport systems, safely, cost-effectively and manageably, by using moving robots to represent road traffic

VeHIL试验室支持主动安全、智能车辆等多种系统测试，通过在室内使用机器人小车呈现道路 交通场景，用一种安全、精确、高效的方式支持开发和测试验证

Confidential

VeHIL facility: sensor benchmark 车辆在环试验室: 传感器零部件级测试

59

VeHIL use moving robot and ground truth data to benchmark sensor detection performance in a very unique manner

VeHIL试验室使用可控、位置速度状态可知的机器人小车，对雷达传感器性能进行深入测试。试验数据可用于雷达等传感器PreScan模型的改进。

Confidential

SiL HiL ViL VeHIL Test

Track Public Roads

5，ADAS和自动驾驶测试场

ADAS and automated vehicle test track

60

Development & evaluationmethodology in

model based simulation

开发和测试方法论：基于模型的仿真方法

Confidential

LDW&AEB Test Track LDW&AEB试验场

61

TASS test track is accredit Euro NCAP test facility for new AEB/LDW testing item, which is located in Aldenhoven, as well as crash safety.

TASS天欧的Euro NCAP AEB/LDW试验场位于Aldenhoven，是首批获得Euro NCAP主动安全系统试验资质的试验场。

Confidential

Example 10, AEB-Cyclist test protocol research for Euro NCAP 案例10，AEB-自行车, Euro NCAP测试规范研究

62

Confidential 63

Example 11 : V2V Cooperative AEB-Cyclist 案例11 : 对基于车车通讯的 AEB-自行车 的测试

Confidential

Example 12, Asta-Zero test facility for truck AEB 案例12，Asta-Zero 试验场的商用车AEB试验支持

64

使用仿真技术，在实际测试之前，进行试验分析和摸底。

TASS provide PreScan software to customer to run virtual simulation before actual field test

https://youtu.be/LLK5m8HLMGs?t=1m

Confidential

Dome camera (safety)

Fixed camera (vehicle localization)

Communication unit (ITS G5)

DITCM = Dutch Integrated Test-site for Cooperative Mobility(荷兰协同驾驶集成试验场)

Real-life validation of Connected Systems on a network-level Public Roads A270 between Helmond and Eindhoven operated by TASS

使用A270高速Helmond和埃因霍温段建立的智能驾驶测试场，长达7公里。由TASS运营，执行过多个荷兰政府和欧盟组织的智能驾驶、自动驾驶项目的测试

DITCM facilities test site A270 DITCM自动驾驶测试场 – 荷兰A270高速公路

http://pan.baidu.com/s/1dDdaFSH

65

Confidential

Example program – Test facility for 2016 GCDC(Grand Cooperative Driving Challenge协同式驾驶竞赛), a European research project, supported by the European Commission

测试项目一例： 2016年5月完成的欧盟GCDC协同式驾驶竞赛的比赛场, GCDC是欧盟委员会支持的欧盟研究项目

DITCM facilities test site A270, Example 13 案例13 : DITCM自动驾驶测试场– GCDC大赛

67

Confidential

Example program – Ten teams from six European courtiers using different cars to make cooperative self-driving test , the Swedish team from Halmstad University won the first prize 测试项目一例：不同厂商的V2V车辆，如何完成协同式自动驾驶？

来自瑞典的Halmstad大学赢得了第一名。

68

DITCM facilities test site A270, Example 13 案例13 : DITCM自动驾驶测试场– GCDC大赛

Confidential https://www.youtube.com/watch?v=duCtiLJ_iCE 70

MCity powered by PreScan, Example 14 案例14 : MCity使用PreScan进行测试场设计支持

https://www.youtube.com/watch?v=duCtiLJ_iCE

Confidential 72

The development & evaluation tool chain 开发&测试工具链

仿真 场地 路试

场景数量巨大 场景数量有限 场景数量可遇不可求

场景可控 ，可重现 场景可控 ，可重现 不可控，不可重现

提供”真值”，客观评价 提供”真值”，主客观评价 缺乏”真值”，主观评价

高效率 较高效率 低效率

真实度 < 100% 真实度接近100% 真实度 = 100%

SiL HiL ViL VeHIL Test

Track Public Roads

Confidential

Summary 总结

73

Model based simulation methodology offers effective tool chain for ADAS /AD systems performance development and evaluation, from simulation, to hardware-in-the-loop test, to close track and public roads

PreScan is the core of the whole tool chain with multiple benefits.

ADAS/自动驾驶系统日渐复杂，基于模型的仿真开发和测试方法，更加高效。使用从虚拟仿真->硬件在环台架->室内受控试验室->场地测试->路试的工具链，是一条必由之路。

PreScan是测试工具链的核心，提供卓越仿真能力

Confidential

Testimonial

客户评价

20-12-2016 74

Confidential

Testimonial Honda

“We applied PreScan real-time HIL for testing our camera – radar system. Extensive test scenarios were simulated using test automation to maximize the efficiency of development cycle. We used it to test our ADAS applications such as LDW, FCW and TSR, and are now extending to AEB and ACC along with precise vehicle dynamics models of CarSim."

Masahito Shingyoji

Chief Engineer

Honda R&D Co., Ltd.

75

Confidential

Testimonial Hyundai Motor Company

"Hyundai Motor Company (HMC) has been using PreScan software for several years now to develop and verify new ADAS functions such as pre-crash. We particularly value the capabilities of the software in the area of scenario definitions, the easy interface to our existing vehicle dynamics models, the broad database of different sensing technologies and the professional support that we receive from the local TASS team. As a result we recently planned to extend our PreScan usage to our autonomous driving

development and research."

Mr. Yongsun Kim

Hyundai Motor Company

76

Confidential

Testimonial Denso: LKA/LDW

"We are using PreScan extensively for Lane Keeping Assist and Lane Departure Warning (LKA/LDW) applications. It allowed us to achieve a major reduction in road testing while investigating a much wider range of scenario variations, which would never have been possible by using road testing alone."

Mr. Takahito Nakano

DP-iSafety Center

DENSO CORPORATION

77

Confidential

Testimonial Daimler Trucks

“When upgrading our truck driving simulators for ADAS and HMI studies, we selected the PreScan software from TASS International for the world, scenario and sensor simulation. Its accurate sensor models and flexible scenario definition enable us to do active safety system analysis and HMI studies with a driver in the loop. Their engineering team did a great job connecting PreScan to our existing control and vehicle dynamics models"

Mr. Thomas Passegger

Truck Product Engineering

Daimler AG

78

Confidential

Testimonial VisLab: Autonomous driving

" The PreScan software is used at VisLab during the algorithm design phase of our autonomous vehicles. Its usage allowed us to make very fast design iterations and contributed to a robust algorithm design and a successful road demonstration of our autonomous vehicles.”

Prof. Dr. Alberto Broggi

VisLab, Parma University

www.vislab.it

79

Confidential 80

Testimonial FAW: ACC

“The results from radar sensor benchmarking in TNO’s VeHIL facility enabled us to make an objective selection of different sensor suppliers for our new ACC/AEB system. Moreover, the PreScan sensor model that was derived from these tests allowed us to complete large parts of the algorithm development process by simulation instead of on the road. We also used VeHIL to validate the final system under controlled yet realistic conditions. This offered us valuable insight into the system’s real-world behavior and performance.”

Mr. Shaobo Qiu

Technical Director Vehicle Safety

FAW Group R&D Center

Confidential

Testimonial SAIC: active safety systems

81

“We successfully used PreScan for design and evaluation of active safety system concepts in typical Chinese accident scenarios. PreScan enabled us to achieve our project targets in a cost- and time-effective manner… this makes that we will be able to save a considerable amount of time, hardware investments, and road-testing efforts.”

Dr. Xiaodong Tang

Director Safety Integration

SAIC Motor

Confidential

Thanks for your attention !

谢谢

黄汉知 137 6428 1890 hanzhi.huang@tassinternational.com www.tassinternational.com TASS International China Co. Ltd 天欧汽车工程软件（上海）有限公司