Copyright MIRA 2008

Testing and Validation ofPre-Crash Sensing

Testing and Validation ofTesting and Validation ofPrePre--Crash SensingCrash Sensing

Jonathan Moore and Tim EdwardsMIRA Ltd, UK

Pre-Crash-Sensorik im Automobil22 April, 2008. Munich

Copyright MIRA 2008

IntroductionIntroductionIntroduction

An overview of test and validation approaches for pre-crash systemsThe sensor technologies and example test cases will largely be focused on systems to detect and respond to vulnerable road users (VRUs)

PedestriansCyclistsMotorcyclists

This ties in to our work on a number of EU funded research projects, such as WATCH-OVER and SAVE-U

Copyright MIRA 2008

Why do we need pre-crash test facilities?

Why do we need preWhy do we need pre--crash test crash test facilities?facilities?

A safe environment for:Research and development

Currently new tests are being designed for each projectComponents and sub-systems often tested in isolationFuture refinement activities would benefit from established and repeatable test procedures/environments

Demonstration activitiesProof-of-concept tests to showcase technologies, encourage public acceptance and develop commercial support.

Certification and ValidationCommercial systems will require validation against a set of specific requirements, and therefore a defined set of tests.

Copyright MIRA 2008

Image from the United Nations Population Fund (UNPFA) web site: http://www.unfpa.org/swp/2007/presskit/images/japan.jpg

Tokyo, Japan

Copyright MIRA 2008

Recreating realistic scenariosRecreating realistic scenariosRecreating realistic scenarios

The requirements for test scenarios are linked to the systems under test. In most cases it is desirable to have minimal test equipment in view of the vehicle sensors

Current automotive tests such as crash, EMC, and emissions are very established, and specialist facilities exist. The body and safety tests are the closest to replicating pre-crash scenarios

However, in these tests there are often systems such as rails and winches which are not acceptable for some pre-crash sensor tests

Copyright MIRA 2008

Body and Safety TestingBody and Safety TestingBody and Safety Testing

Copyright MIRA 2008

Controlling test conditionsControlling test conditionsControlling test conditions

The test environment should be controlled to allow for safe, repeatable testsOutdoor tests introduce a number of challenges in this area. We can’t control nature, but we can…

Automate as much as possibleTest vehicle controlData synchronisation

Monitor environmental conditionsProvide clear signals to co-ordinate “actors”

Copyright MIRA 2008

Testing safelyTesting safelyTesting safely

Risk assessment procedure for any proving ground testDetailed FMEA (Failure Mode and Effects Analysis) for test equipmentRigorous maintenance procedures for test equipmentStrict operating procedures and checksRedundancy of safety mechanismsFail safe modes in control electronics

Copyright MIRA 2008

Current/Future ADAS SensorsCurrent/Future ADAS SensorsCurrent/Future ADAS Sensors

24GHz RadarMono and Stereo vision camerasPassive Infra-RedWireless communications

2.4Ghz Chirp Spread Spectrum (CSS)

LIDARUltra-wide band (UWB)

Copyright MIRA 2008

Data Acquisition RequirementsData Acquisition RequirementsData Acquisition Requirements

Quiet wireless environmentTherefore no wireless data streams during the test

On-board storageDurableLightHigh capacityFast, shared, accessQuick “data dump” or drive switch between tests

Interfacing with proprietary hardware / softwareAccurate time stamping

Required for synchronising all logged data (on and off-board the test vehicle)

Copyright MIRA 2008

Ground TruthWhere was everything?

Ground TruthGround TruthWhere was everything?Where was everything?

Very important that we can compare real events with sensor readingsKey element is timing

External ground truth monitoring must support synchronisation with on-vehicle data

Options include:Positioning devices on every “actor” in the scenario

High accuracy required and therefore high costVideo footage and image processing

Suitable observation positions can be hard to findDistorting lens increase coverage but complicate image processing

Copyright MIRA 2008

Case Studies: SAVE- U and WATCH-OVER

Case Studies: Case Studies: SAVESAVE-- U and WATCHU and WATCH--OVEROVER

BackgroundTest conceptResults / VideoConclusions http://www.watchover-eu.org/

http://www.save-u.org/

Copyright MIRA 2008

SAVE-USAVESAVE--UU

SAVE-U was a large European project that ended in 2005.

It was aimed at developing an integrated safety concept for VRUs such as pedestrians and cyclists.

An innovative sensor platform was developed with three different technologies (24 GHz radar, IR and video cameras).

The maximum vehicle speed considered was 40 km/h

Copyright MIRA 2008

WATCH-OVERWATCHWATCH--OVEROVER

WATCH-OVER started in January 2006. It is a collaborative project co-funded by the European Commission under FP6.

The innovative concept is represented by an on board platform and by a vulnerable user module. The system is based on short range communication and vision sensors.

Maximum vehicle speedincreased to 50 km/h

Copyright MIRA 2008

Project RequirementsProject RequirementsProject Requirements

MIRA is responsible for dynamic evaluation of the system – especially at close range (highest risk)

Target must:Look like a moving VRU (video)Look like a moving VRU (IR)Reflect like a moving VRU (radar)

Hence target must BE a VRU – not a picture or a dummy…but absolutely no risk of actual impact

MIRA solution is to mount the entire system on a tethered dynamic test rig, not a car

Copyright MIRA 2008

MIRAMIRA’’ss HeadquartersHeadquarters

Copyright MIRA 2008

Test site configurationTest site configurationTest site configuration

105m

5.5m

27m

25m

10m

50m

19m29m

3.2m

1.6m OK

No2 Circuit

No2 Bend No3North Straight

Observation RoadNo15 CorrugationsNo13 Resonance Road

Parking

Entrance

ExitConcrete

wall

Crash barrier

Grass

T2

Created:20070601Revised:20070611

To Control Tower

N

Tarmac

Store

Tether point

Copyright MIRA 2008

Pre-crash test vehiclePrePre--crash crash

test vehicletest vehicle

Copyright MIRA 2008

Restraint MechanismsRestraint MechanismsRestraint Mechanisms

Test vehicle

Falling weight

Braking Drum

Energy absorbers

Deformed steel bar

Copyright MIRA 2008

Braking system in use during SAVE-U tests

Copyright MIRA 2008

SAVE-U test with pedestrians crossing

Copyright MIRA 2008

Ground TruthGround TruthGround TruthTo allow accurate post-test data analysis, onboard sensors logged

rig velocity and yaw, and “ground truth” information is recorded using an overhead camera.

On board cameras

Overhead “ground truth” camera

Copyright MIRA 2008

Overhead ground truth camera capturing a SAVE-U test scenario

Copyright MIRA 2008

Data acquisitionData acquisitionData acquisition

Copyright MIRA 2008

Un-tethered speed measurement

UnUn--tethered speed tethered speed measurementmeasurement

Max: 50.4km/h

Target: 50km/h

8 seconds60m

Braking took 3 seconds.With tether ropes this is reduced to 1 second.

0

2

4

6

8

10

12

14

16

0 5 10 15

Time (s)

Vel

ocity

(m/s

)

Copyright MIRA 2008

Test scenariosTest scenariosTest scenarios

A variety of scenarios were run in SAVE-U with vehicles, pedestrians and cyclists

In WATCH-OVER the scenario complexity will be increased, with more actors and night tests

Copyright MIRA 2008

ConclusionsConclusionsConclusionsIncreasing scenario complexity desired by system designersSafety issues need to be tightly monitored and controlledTest vehicle positioning

Manual/Semi-Autonomous/Fully-Autonomous vehicles all require accurate monitoring and control

Actor control is very importantRepeatable test scenarios Safety

Ground truthThe capture and synchronisation of position information for all actors in a test is non-trivial.This must be considered as a design requirement at an early stage of the test design

Copyright MIRA 2008

Specifically the future of pre-crash testing…

Autonomous vehiclesAutonomous VRUs?Purpose built test sites

with protective barriers and fail safe restraintsComplex test scenariosEnvironmental conditions – Fog/Rain etcCornering, urban/built up areas

The future of pre-crash sensing

The future of preThe future of pre--crash crash sensingsensing

Copyright MIRA 2008

Improved test scenarioImproved test scenarioImproved test scenario

Test Scenario

Test Vehicle

Restraint Mechanism

Copyright MIRA 2008

Autonomous Test VehicleAutonomous Test VehicleAutonomous Test Vehicle

+ Complex and repeatable test scenarios- High levels of reliability need to be assured

for use around VRUs

Copyright MIRA 2008

http://www.mira.co.uk

MIRA LtdWatling Street

NuneatonWarwickshire

CV10 0TUUnited Kingdom