1

Application specific low cost positioning for vehicles

Erwin Rademakers

Senior Research Engineer

28 March 2017

Low cost Positioning

AGENDA

Motivation & user needs

Absolute & relative positioning concepts and results

Computing platform

B2B project: Low cost and accurate positioning for cycling

WHY POSITIONING?

Persistent need for low cost accurate, reliable and robust positioning systems

Room for innovation in low cost positioning systems tailored per applicationdomains

Current price factor is 100 going from 2m to 2cm

APPLICATION REQUIREMENTS

On-road applications: Automated/Autonomous driving

➢ Sub-meter accuracies for safety, efficiency and comfort

➢ Precise HAD map through crowdsourcing

Off-road applications

➢ dm to cm range accuracies

➢ Agricultural vehicles, special purpose vehicles

POSITIONING CONCEPTS

Absolute earth coordinatesresulting from

• Satellite systems (e.g. GPS)

• Matching (land)markerswith an accurate map

Relative positioning tolimit the delta againstlast absolute coordinatesthrough Odometry

• Inertia odometry (IMU)

• Visual odometry (camera)

Fusion techniques for accurate, robust and reliable positioning system

FUSION

Positioning

Absolute Relative

MAP

INS

Odometry

Visual

Beacons

GPS

Visual

Odometry

FUSION

Positioning

Absolute Relative

Map

Visualodometry

INS

odometry

Visualbeacons

GPS

Receiver

ionosphere

troposphere

ABSOLUTE POSITIONING: GNSS SATELLITESGNSS RECEIVERS

• Single frequency - Dual frequency

• Dual frequency allows for removal of the first ordereffect of the ionosphere

• Single – Multiple concurrent constellations

• GNSS constellations of interest: GPS, Glonass, Galileo (y2020)

Error Component Description Potential Improvement

Ionosphere Free electrons (sun) influence satellite signal (EM waves), above > 1 km

200 cm

Troposphere Temperature, humidity, pressureinfluence, below < 1 km

200 cm

Ephemeris data Satellite position data 100 cm

Satellite clock drift Clock drift 150 cm

Dcb Differential code bias, hardware delay 10 cm

WindUp Rotation of the antennas 10 cm

Sagnac effect Light travels in a spiral path 5 cm

ROA Satellite orbit correction (radial, allong, crossing direction)

5 cm

Solid Earth Tides Tides: Moon – earth interaction 2 cm

Relativistic effect Relativistic clock correction 5 cm

Available solutions: GPS ≈ 25 € (10 m acc)

RTK > 5 k€ (2 cm acc)

Solution: SF-PPP* ≈ 100 € (sub-meter

accurate)

SELECTION GNSS SOLUTION

Ref ESA navipedia (modified)

NEW

In cooperation with:

• Single Frequency Precise Point Positioning based• Low cost because of single constellation (GPS) - single frequency• Error corrections from internet

BENCHMARK

Effect from non-line of sight and reflections (e.g. by trees)

Driven trajectory Error against RTK as reference

(snapshot)

Low cost GPS 25 € Low cost PPP 100 €RTK-reference > 5k €

t

error

sub

-met

er r

equ

irem

ent

RESULTS “OPEN AREA” ENVIRONMENT

No obstructions from trees or buildings.

RTKPPP

RMSe North RMSe East

Avg σ Min Max Avg σ Min Max

0.53 m 0.22 m 0.17 m 0.98 m 0.48 m 0.15 m 0.24 m 0.77 m

RESULTS “HIGHWAY” ENVIRONMENT

In this environment we have less satellites due to obstructions from trees.

RTK

Waiting @traffic light (trees)

RMSe North RMSe East

Avg σ Min Max Avg σ Min Max

1.15 m 0.30 m 0.67 m 1.73 m 0.62 m 0.28 m 0.36 m 1.25 m

WHAT IF THERE IS NO GPS OR LIMITED AVAILABLE?

FUSION

Positioning

Absolute Relative

Map

Visualodometry

INS

odometry

Visualbeacons

GPS

RELATIVE POSITIONINGINS ODOMETRY

Continuously calculate position based on MEMS IMU

• Gyroscope: degrees/s in 3D

• Accelerometer: g in 3D

• Compass (useless close to metal)

• 100 Hz

SENSOR FUSION

Kalman Based

Filtering

Sensor fusionGPS PPP

Position

Course

Velocity

IMU

Yaw rate

Acceleration

PoseYaw

Velocity

Ext. Kalman based filtering

?ˆ,ˆ ,N,E V

GPS 10 HzIMU 100 Hz

FUSION 100 Hz

DRIFT

Absolute coordinates available

Drift / Integration errorCorrection moment

to available absolute

coordinate

Errors in measurements of gyro- & accelerometer resulting in a drift.

WHAT ABOUT CORNERSIn a corner: vehicle heading ≠ position heading

Sideslip angle estimation

• Measuring wheel speed, steering angle

• Vehicle model (bicycle model, 2 wheel comparable against 4 wheel)

SENSOR FUSION

Sensor fusionGPS PPP

Position

Course

Velocity

IMU

Vehicle

Yaw rate

Acceleration

Steering angle

Wheel speeds

PoseYaw

Velocity Side slip?ˆ,ˆ ,N,E V

Ext. Kalman based filtering

RESULTS URBAN ENVIRONMENT PPP ONLY

In an urban environment we have less satellitesavailable due to obstructions from buildings.

RTK

Buildings blockingSatellite reception

Open area

RMSe N RMSe E

PPP only 3.90 m 3.24 m

RESULTS URBAN ENVIRONMENT PPP AIDED WITH IMU

PPP used in combination with IMU (gyro and accel)▪ Extended kalman filter (EKF)

RMSe N RMSe E

PPP only 3.90 m 3.24 m

PPP+IMU 1.16 m 1.14 mmore than times improved

EKF (PPP+IMU)RTKPPP

Content

HARSH ENVIRONMENT

GPS outage

EKF (PPP+IMU)

SBG

PPP

PPP large error

RMSe N E

PPP+IMU 1.08 m 0.96 m

FUSION

Positioning

Absolute Relative

Map

Visualodometry

INS

odometry

Visualbeacons

GPS

Vehicle

Camera

Road Surface

RELATIVE POSITIONINGVISUAL ODOMETRY – GROUND FACING CAMERA

Height (𝐻)

Camera

Displacement (𝐷∆𝑡)Road Surface

////////////////////////////////////////////////////

𝐶𝑡1𝐶𝑡2

Have potential 𝑚𝑚 resolution accuracy (depending on calibration, resolution …)

Using a mono camera

Finding the maximum phase correlation between two consecutive frames

A wheel encoder suffers from wheel slippage, GFC not

Independent of the kinematics of the vehicle

Closing Trajectory Length ≅ 𝟐𝟎𝒎

Error Distance = 𝟎. 𝟐𝟔𝟕𝒎 (𝟏. 𝟑𝟑 %)

Error Orientation = 𝟏𝟕. 𝟐𝟎𝟓𝟕°

EXPERIMENTAL RESULTS GFC

Data sample (Octinion)

RELATIVE POSITIONINGVISUAL ODOMETRY – STEREO CAMERA

Have potential c𝑚 resolution accuracy (depending on calibration, resolution …)

Using stereo camera

Matching visual feature points between two consecutive frames

Extracting 3D information from multiple 2D views, which provides relative depth information of a scene

• In GFC the height (depth) is already known

Independent of the kinematics of the vehicle

Matching features in a line/circle, colors encode disparities.

EXPERIMENTAL RESULTSSTEREO VISUAL ODOMETRY

Distance: 60m

Matches: 126

Inliers: 50%

Failed frames: 3

Good frames: 755

GPS

VO

FUSION

Positioning

Absolute Relative

Map

Visualodometry

INS

odometry

Visualbeacons

GPS

MAP FOR POSITIONING

Multi layer approach

• Static layer: existing environmental data

• Dynamic layer: temporal object information

• Has confidence, accuracy and reliability parameters

Positioning approach using

• Visual Beacons

• Road Signs

• Landmarks

• Potholes

• Road bumps

• ...

BOOTSTRAPPING THE MAP

DetectionVisual Beacons

Road Signs

LandmarksPotholes

Road bumps

Positioningfusion

D

relative positioning

visual odometry

Map1. Visual Beacons2. Traffic Signs3. Re-visit-able

Landmarks4. Road surface

x,y,z,rotx,roty,rotz

PPP

absolute position bootstrapping of unreliable

landmarks and road surface imperfections

FUSION

Positioning

Absolute Relative

Map

Visualodometry

INS

odometry

Visualbeacons

GPS

Content

ABSOLUTE POSITIONINGVISION BASED – PASSIVE BEACONS – ROAD SIGNS

Absolute position based on detection of road signs• Mono camera• Relative position from detection algorithm: 3D camera position (relative) from the road

sign detection algorithm and the absolute position of the speed sign.• The absolute position of the road signs are retrieved from the dynamic map

Can also be used to create a High Definition MAP• Can also be lane markers, buildings, traffic lights, obstacles (depending on the detector)• Map update via crowdsourcing to obtain an accurate position

RESULTS ROAD SIGNS

Camera resolution 1280x960 @ 20HzVelocity speed 70 – 50 – 70 – 50 kph

RMSE < 0.5 m

° Ground Truth° RS

50

70

70

50

ABSOLUTE POSITIONINGVISION BASED – ACTIVE BEACONS

Where GPS is limited available, to bootstrap the creation of a MAPControlled lighting beaconsTriangulation approach with heading fusionMotion compensationLow speed

DeltaBeacon OFF Beacon ON

Size and deformation are measures for distance

RESULTS VISUAL BEACONS

Detection accuracy related to optics & resolution

For <40m: ±25cm accurate (RMS error E/N)

For >40m: 25 – 50cm accurate (RMS error E/N)

Position Error

FUSION

Positioning

Absolute Relative

Map

Visualodometry

INS

odometry

Visualbeacons

GPS

RTK Low Cost-GPS PPP PPP/IMU Beacons Road Signs

Accuracy 1-2cm 2-10m 0,2-2m 0,2-2m < 20cm 0,1-50cm

Update rate 200Hz 1 Hz 10Hz 100Hz 20Hz 20 Hz

GFC

Road Signs

VO (stereo)

gg,V

vv,V

rrN,E

PPP/IMU

Vehicle Model

Beacons

Sen

sor

fusi

on

?ˆ,ˆ ,N,E V

V, N,E,

,,yx

aa

bbN,E

MULTI SENSOR FUSION

To obtain robust and accurate position in different situations/environments

Centralized Kalman filterTaking into account for each sensor

• Measurement noise• Weight factor

Fusion with GFC/VO status experimentalFusion with Map conceptual status

MAP

Results from real time multi sensor:• PPP/IMU – VM – Beacons – Road Signs.

Simulation of a harsh environment:• Obstruction from bridges (indoor-outdoor), trees, …

The reference systems is also suffering in this harsh environment (RTK system aided with IMU).

RESULTS MULTI SENSOR FUSION

RTK drift

RESULTS MULTI SENSOR FUSION

EKF

RTK

PPP

GPS outage

bad GPS reception

suffering from trees

PPP+IMU PPP+IMU+Road Signs

EKF

RTK

PPP

RSR

drift corrected

by road sings

PPP+IMU+Road Signs+Beacons

EKF

RTK

PPP

RSR

BEACONS

position accuracy

improved by beacons

GENERIC MULTI SENSOR FUSION

Research PlatformEmbedded platform

RESEARCH PLATFORM

+GPU

MB

Router+ 4G

SSD

CAN

Distributed architecture: open platform (extendable, flexible)

Outstanding performance using fast hardware CPU/GPU.CPU Intel 64 bit, 4x core, 3.5 GHzGPU Nvidia GTX 780

+

Trunk Ford S-MAX

MULTI SENSOR FUSION

TCP

TCP

TCP

Visual OdometryRoad Signs

Fusion

Beacons

TCP TCP

GFC

TCP

TCP

PPP

IMU

Client

>>

>>

>>

>> <<

>>

>>

>>

SOFTWARE DEVELOPMENT ENVIRONMENT

Enterprise architect for OOAD (UML)

• PlantUML in the code with doxygen

IDE eclipse CDT

• Scons (build tool)

Programming language C++11 (multi-arch, 32-64 bit), python (scripts)

Linux operating system (Ubuntu, with pre-emptive patch in development)

Main 3th party libraries (Open source)

• Boost (XML, UDP/TCP/serial communication …)

• OpenCV (computer vision, basic functions)

• Newmat (matrix operations)

• Google test/mock

Profilers: memory check, performance, code coverage, … (GNU-tools, Google)

EMBEDDED PLATFORM

Research platformCPU Intel 64 bit, 4x core, 3.5 GHz

GPU Nvidia GTX 780

Nvidia Jetson Tk1 (Tegra)CPU 2.32GHz ARM quad-core Cortex-A15 (32 bit)

GPU with 192 CUDA cores

Rpi 3CPU 1,2 GHz ARM quad core Cortex-A53 (32/64-bit)

GPU Broadcom VideoCore IV

+

3x 2x

Less

per

form

ance

COST PROTOTYPE

Embedded platform (mono camera)

Indicative costs

• HW/SW costs (depending on the use case varying from 6 MM to 12 MM)

• BOM: 300 to 400 €

Computing platform(with GPU)

Camera(low cost variant)

IMUGPSData connection

50 € 10 €50 € 150 €100 €

TEST SETUP

Platform

Ground Facing Camera

LOW COST & ACCURATE POSITIONING FOR CYCLING

Company:

Problem statement (summary):

Current products have low accuracy and low update rates.

Design of the bike computer doesn’t fit the design and “aero-dynamics” of the bike

Additional sensors required on the bike

POSITIONING FOR CYCLING

RTK

m distance

m height

kph average

kph

SUMMARY

SUMMARY

Low cost and accurate techniques can deliver high accuracy• GPS with PPP < 50 cm (open air)

• INS odometry < 2 % (error distance)

• Visual beacons < 50 cm (low-mid speed)

• Visual odometry < 1.5 % (error distance)

• Pitfalls:

• high drift in case of long outage of an absolute position (i.e. GPS outage)

• inaccuracy of vision based positioning due to light conditions, shocks, needs texture, imperfect camera calibration, dust/dirt, low vehicle speed (framerates)

BOM Cost of these techniques can stay below 400 € (prototype)

Autonomous vehicle positioning does require fusion of techniques especially in harsh environments

50

erwin.rademakers@flandersmake.be