mapping Smart-Grid modernization in Power Distribution Systems
Multi-Objekt-Tracking und Grid-Mapping für die … · 2017. 9. 22. · Seite 2 Tracking and...
Transcript of Multi-Objekt-Tracking und Grid-Mapping für die … · 2017. 9. 22. · Seite 2 Tracking and...
1
Multi-Objekt-Tracking und Grid-Mapping für die
Fahrzeugumfelderfassung unter Verwendung von Random-Finite-Sets
Stephan Reuter, Karl Granström, Dominik Nuss, Alexander Scheel
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 2
Motivation
Multi-object tracking and grid mapping are widely used approaches for
environment perception
Complexity increases with level of automation
Requirements
Real-time capability
Integrated existence estimation for objects / obstacles
High object density
level of automation
2
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 3
Sensors for Environment Perception
Mono-Camera
Lidar
Radar
Versuchsträgerfahrzeuge der Universität Ulm
Stereo-Camera
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 4
Bayes Filter
Implementation
Kalman Filter
Sequential Monte-Carlo Methods (Particle Filter)
motion
state space
measurement space
xk
xk+1
zk
zk+1
updateprediction
3
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 5
Multi-object tracking often realized using several independent Kalman filters
Heuristics between the individual steps lead to loss of information
Joint Probabilistic Data Association (JPDA) filter
Based on computing marginal data association probabilities
Multi-Hypothesis Tracker (MHT)
Based on maintaining different hypotheses about data association.
Pruning and merging of hypotheses
Classical Approach: Multi-Instance Kalman Filter
Raw data Tracks
? ?
?
?
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 6
Multi-object tracking often realized using several independent Kalman filters
Heuristics between the individual steps lead to loss of information
Joint Probabilistic Data Association (JPDA) filter
Based on computing marginal data association probabilities
Multi-Hypothesis Tracker (MHT)
Based on maintaining different hypotheses about data association.
Pruning and merging of hypotheses
Classical Approach: Multi-Instance Kalman Filter
Raw data Tracks
Random-Finite-Set
Approach
4
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 7
Random Finite Set
Set-valued random variable
Realization represents multi-object state
Modelling of object interdependencies possible
Random Finite Sets (RFS)
state vectors RFS
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 8
Multi-Object BayesFilter
updateprediction
state space
measurement space
XkXk+1
Zk Zk+1
k k+1
Motion
Disappearance
Appearance
multi-object measurement model
Detection
Missed detection
Clutter
multi-object Markov density
5
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 9
Multi-Object Bayes Filter – Implementation
Sequential Monte-Carlo Methods
(Vo 2005)
Probability Hypothesis Density (PHD) Filter
(Mahler 2003)
Cardinalized PHD Filter
(Mahler 2007)
Cardinality Balanced Multi-Bernoulli Filter
(Vo 2009)
-Generalized Labeled Multi-Bernoulli Filter
(Vo 2011)
Labeled Multi-Bernoulli Filter
(Reuter 2014)
Filter Algorithm
Moment
Approximation
Parameter
Approximation
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 10
Multi-Object Bayes Filter: Moment-Approximations
PHD-Filter (Mahler2003): Approximation using first statistical moment
CPHD-Filter (Mahler2007): first statistical moment & cardinality distribution
PHD or
intensity function
Cardinality
distribution
pdf over number of
objects
first moment of
multi-object distribution
vkvk-1
rkrk-1
6
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 11
Parameter-Approximation: Labeled Random Finite Sets
Labeled Multi-Bernoulli (LMB) Random Finite Set
r=0.99 r=0.05r=0.79r=0.79existence
probability
Track-IDs
spatial
distribution
-Generalized Labeled Multi-Bernoulli (-GLMB) Random Finite Set
w=0.2 w=0.74 w=0.05
Transformation Approximation
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 12
Multi-Sensor Multi-Object Tracking
Sensor-independent fusion using Labeled-Multi-Bernoulli-Filter
Facilitates sensor replacement
Classification of objects (car, bike, pedestrian,…)
Generic
Sensor-
Inte
rface
LMB Filter
-
LMB Prediction
Gating
Birth model
Track-Management
LMB -GLMB
-GLMB-Update
-GLMB LMB
Camera Preprocessing
Lidar Preprocessing
Radar Preprocessing
7
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 13
Video Urban Traffic
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 14
Extended Object Tracking
State of the Art: point-target tracking using heuristic preprocessing
algorithms
Extended Object Point-Object Unresolved Objects
Raw data Tracks
Random-Finite-Set
Approach:
Labeled-Multi-
Bernoulli-Filter
8
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 15
Extended Object Tracking
State of the Art: point-target tracking using heuristic preprocessing
algorithms
Extended Object Point-Object Unresolved Objects
Raw data Tracks
Random-Finite-Set-Filter for Extended Objects
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 16
Modeling of Extended Objects
Complexity
Point Basic Shape Arbitrary Shape
Possible Reflection Centers (in 2D)
Contour Surface
9
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 17
Gamma Gaussian Inverse Wishart Model
Elliptical shape
Assumption: Measurements are generated by a multi-variateGaussian distribution with mean at the objects‘ position andcovariance matrix
Number of measurements isPoisson with mean
Combined state vector
Gaussian distributed kinematic state
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 18
Results
LMB filter with joint prediction and update implementation (red)
Computation times (Intel i7 2600, C++, no parallelization)
Average of ~100 targets
Mean: 7.0 ms
Max: approx. 25 ms
Comparison: standard
implementation using
Murty’s algorithm (blue)
Mean: 622.0 ms
Max: aprox. 10000 ms
10
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 19
Modeling of Extended Objects
Complexity
Point Basic Shape Arbitrary Shape
Possible Reflection Centers (in 2D)
Contour Surface
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 20
Gaussian Process Object Model
• Use Gaussian process to describe the radius function
• Consider combined state vector for kinematics and radius functiong mit
EKF
Combined
State
Kinematic
State
11
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 21
Gaussian-Process
Definition:
with first two moments
• Mean function
• Covariance function:
Representation as a multivariate Gaussian:
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 22
Gaussian Process
3-Sigma-Bounds
12
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 23
Recursive Gaussian Process
3-Sigma-Bounds
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 24
Extended Object Tracking Using Gaussian Processes
13
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 25
Intersection Scenario Using Gaussian Processes
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 26
Intersection Scenario Using Gaussian Processes
14
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 27
Challenges
Precise shape representation requires high number of inputs (sampling
points)
Matrix dimension: number of kinematical states + number of inputs
Numerical issues during calculation of matrix inverse
Rao-Blackwellized implementation
Representation of kinematic state using particles (approx. 1000)
Each particle holds a GP or a mixture of GPs for the objects’ extent
Parallelization is required
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 28
Environment Representation using Grid Maps
Separation of environment into discrete cells
For each cell: infer on cell state (occupied or free) using sensor data
Assumption: statistically independent cells
[1] Thrun, S.: Probabilistic Robotic. The MIT Press, 2005.
[2] Elfes, A.: Using occupancy grids for mobile robot perception and navigation. IEEE Computer, 1989, Vol. 22.6, pp. 46-57.
15
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 29
Static Grid Map
Assumes stationary environment
Dynamic objects lead to artefacts in grid map
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 30
Robust estimation of dynamic objects without error-prone clustering
Independent of object shape
and class
Fusion of heterogenous sensor
data possible
Based on PHD and Bernoulli filter
Dynamic Grid Map
Laser Raw Data
Dynamic Grid Mapping
Clustering
Object-Tracking
Radar Raw Data
16
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 31
Dynamic Grid Map – Principle
Occupancy for current timerepresented by particles
“Motion” of occupancy inprediction step,uncertainty about actualmovement leads to blurring
Update of particle weightsusing measurement grid,new cells may be occupiedafterwards
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 32
Current Implementation
Random-Finite-Set based particle filter implementation
Parallelization using CUDA (GTX970+)
Huge amount of particles required (3-10M)
Prediction step facilitates embarrassingly parallel computation
Update step
Association of particles to cells using sorting algorithm
Update of particles in cell embarrassingly parallel
Resampling based on sorting
Real-time capable for up to 10M particles
17
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 33
Dynamic Grid Map – Video
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 34
Conclusions
Random Finite Sets provide a rigorous mathematical framework for object
tracking and grid mapping
Multi-Object Tracking
Very efficient implementation of point-target tracking and extended object
tracking with basic shapes
Tracking of arbitrarily shaped objects requires parallelization and
numerical issues need to be considered
Dynamic Grid Mapping
Well suited to represent arbitrarily shaped objects
Parallelization required to handle huge number of particles
18
Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 35
Additional References / Matlab-Code
B.-N. Vo et. al: „Multitarget Tracking“, http://ba-ngu.vo-au.com/vo/VMBCOMV_MTT_WEEE15.pdf
Multi-Object-Tracking using Random Finite Sets:
R. Mahler: „Statistical Multi-Source Multi-Target Information Fusion“
R. Mahler: „Statistics 101“ / „Statistics 102“
S. Reuter: „Multi-Object Tracking Using Random Finite Sets“
https://oparu.uni-ulm.de/xmlui/handle/123456789/3231
K. Granström, M. Baum, S.Reuter: „Extended Object Tracking: Introduction, Overview and
Applications“, https://arxiv.org/abs/1604.00970
S. Hörmann, M. Bach, K. Dietmayer: „Dynamic Occupancy Grid Prediction for Urban Autonomous
Driving: A Deep Learning Approach with Fully Automatic Labeling”,
https://arxiv.org/abs/1705.08781
D. Nuss et. al: „A Random Finite Set Approach for Dynamic Occupancy Grid Maps with Real-Time
Application”, https://arxiv.org/abs/1605.02406
Matlab Code Multi-Objekt-Tracking: http://ba-tuong.vo-au.com/codes.html