Research Project PEGASUS. - pegasusprojekt.de · Cross-Verification Assessment of AD –...

Research Project PEGASUS. Test Case Variation and Execution.

Cross-Verification

Assessment of AD – Methodology.

4 © PEGASUS | Goals and work contents of PEGASUS

Simulation

FOT / NDS

Proving Ground

Scenarios

Scenarios

Scenarios

Extracting

Input

Input

PEGASUS Database

Linking KPIs and Test Instance to

Scenario File

Evaluation

Traces KPIs

Evaluation

Traces KPIs

Evaluation

Traces KPIs

Cross-Verification



Simulation

FOT / NDS

Proving Ground

Scenarios

Scenarios

Scenarios

Extracting

Input

Input

PEGASUS Database


Scenario File

Evaluation

Traces KPIs

Evaluation

Traces KPIs

Evaluation

Traces KPIs

Evaluation

Traces KPIs

Evaluation

Traces KPIs

Evaluation

Traces KPIs

Cross-Verification



Simulation

FOT / NDS

Proving Ground

Scenarios

Scenarios

Scenarios

Extracting

Input

Input

PEGASUS Database


Scenario File

1 2

Information Basis for AD Assessment.


FOT / NDS

Accident Database

Public Testing Fields

Expert Knowledge

Laws/ Standards/ Guidelines

PEGASUS Database

Test Case Data:

- logical scenario spaces,

- parameter lists with distributions,

- known critical concrete scenarios labelled within the logical spaces,

- metrics / KPIs,

- pass/fail criteria.

Scenario data model enrichment with a-priori knowledge about reality:

A-priori probabilities and correlations.

1

Scenario File

Input KPIs

Traces

Evaluation

AD function connected

Dynamic Parameter Space

Logical Scenario File

Parameter Variation

New Scenario Files

Simulation Framework

AD function connected

Stochastic Variation – Methodology. 2

Scenario Data Model for exchangeable and reproducible Scenarios.

1

EGO

1

2

3

4

5

6

…011010…

EGO

Concrete Scenario:

• Highway, 3 lanes, speed limit 80 km/h, • EGO drives const. 80 km/h, • Vehicle 1 flashes and cuts in, • sunshine, dry road, etc.

1

What is varied? – Example.

1

2

3

4

5

Levels of PEGASUS Scenario Data Model.

6

EGO

1


Concrete Scenario:

• Highway, 3 lanes, speed limit 80 km/h, • EGO drives const. 80 km/h , • Vehicle 1 flashes and cuts in at different starting positions with different trajectories.

1

2

3

4

5


6

Concrete Scenario:

• Highway, 3 lanes, speed limit 80 km/h, • EGO drives const. 80 km/h, • Vehicle 1 flashes and cuts in, • new static elements.


1

2

3

4

5


6

EGO

1

Concrete Scenario:

• Highway, 3 lanes, speed limit 80 km/h, • EGO drives const. 80 km/h, • Vehicle 1 flashes and cuts in with 100 km/h, • lane closes.


EGO

1

1

2

3

4

5


6

Concrete Scenario:

• Highway, 3 lanes, speed limit 80 km/h, • EGO drives const. 80 km/h, • Vehicle 1 flashes and cuts in with 100 km/h, • snow.


EGO

1

1

2

3

4

5


6

Stochastic Variation – Goal.


Main Goal:

Find all relevant critical parameter subsets within each logical scenario space, that are within the use-case of the AD function. The stochastic variation within the PEGASUS project has two tasks:

- Exploration: Find critical parameter sets in the sense of a certain safety metric in the defined subspace.

- Characterization: Goal is to deliver a description of the parameter subspace where a metric reports a safety violation.

Stochastic Variation – Exploration.


PEGASUS focused on two approaches of parameter space exploration:

1. Explore known critical scenarios, also with local variation of parameters.

param1

param2

2. Explore a given abstract parameter subspace (logical scenario) with optimization methods in order to find one or more worst cases of the safety metrics.

Starting points for iterative search may be chosen as known critical parameter subsets from real traffic data.

Stochastic Variation – Characterization.


1. Characterization means assessing the probability of safety violations for a logical scenario space with a given confidence measure.

2. Test result is a description of the critical parameter subspace where a metric reports a safety violation including an approximation of probability.

Next Processing Step – Risk Assessment.


Test Results:

- concrete test results per critical scenario

or

- description of the parameter subspace where a metric reports a safety violation including an approximation of extension and probability.

Contact:

BMW Group

Dr. Mark Schiementz

[email protected]

+49 151 60135903

www.pegasusprojekt.de


Resimulation Architecture 2

• Real measurement data of relevant scenarios.

system under test

Actuator

Simulation Model

Environment-/ Situation-analysis

Fusion

Localization

AD Core-Function

Planning

Driving Strategy

Control

longitudinal

lateral

lights

Open Simulation Interface (OSI) OpenScenario / OpenDRIVE

NDS-Format

actuator model

OSI

Einviron-

ment Represen-tation

OSI


Te

st R

esu

lt E

valu

ati

on

system under test (AD Function) sensor model environment model

Actuator

Simulation Model

Vehicle-Simulation Model

USS

Map

Synth. Backend

Radar

Sensor Model

Lidar

Lidar Sensor Model

Camera

Sensor Model

Environment-/ Situation-analysis

Fusion

Localization

AD Core-Function

Planning

Driving Strategy

Control

longitudinal

lateral

lights

Open Simulation Interface (OSI) OpenScenario / OpenDRIVE

OSI

Obj-List

OSI Raw Data

Einviron-

ment Represen-tation

NDS-Format

Ground Truth

Traffic

Agents

Open-

SCENARIO

Open-

SCENARIO

Open-

SCENARIO

Open-

SCENARIO

Open-SCENARIO

actuator model

23

Simulation Architecture – Full Parameter Variation

OSI OSI

Obj-List

OSI Obj-List

OSI Raw Data

OSI Raw Data

OSI Data

OSI Data

OSI Data

2


Sto

cha

stic

Va

ria

tio

n

Open-

SCENARIO Open-SCENARIO

•Logical scenario •Parameter distributions •Metrics •Pass/fail criteria

Open-

DRIVE Open-

DRIVE

Open-

DRIVE

Te

st R

esu

lt E

valu

ati

on

24

PEGASUS Simulation Architecture

Layer 2: Traffic Infrastructure • Boundaries (structural) • Traffic signs, elevated barriers

Layer 1: Road-Level • Geometry, topology • Quality, boundaries (surface)

Layer 3: Temporary manipulation of Layer 1 and Layer 2 • Geometry, topology (overlaid) • Time frame > 1 day

Layer 4: Objects • Static, dynamic, movable • Interactions, maneuvers

Layer 5: Environment • Weather, lighting and other

surrounding conditions

Layer 6: Digital Information • (e.g. )V2X information,

digital map

Research Project PEGASUS. - pegasusprojekt.de · Cross-Verification Assessment of AD –...

Documents

Transcript of Research Project PEGASUS. - pegasusprojekt.de · Cross-Verification Assessment of AD –...