Multitarget therapy versus intravenous cyclophosphamide in ...
Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8....
Transcript of Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8....
![Page 1: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/1.jpg)
Data fusion and multitarget tracking:
some interests for military and
automotive applications.
Evangeline POLLARD
![Page 2: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/2.jpg)
Just a few words about me…
One year internship
DLR Munich, Germany
Master and PhD degree
Paris/Grenoble,France
Post-doc
Versailles and Sherbrooke (Canada)
Visitor researcher
Berkeley, CA 2
![Page 3: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/3.jpg)
Outlook
1. Military application: convoy detection and tracking
1. Multi-target tracking, a brief overview
2. Hybridization of CPHD filter and MHT
3. Bayesian network for convoy detection
2. Multi-target detection and tracking with uncalibrated
aerial videos
1. Detection
2. Tracking
3. Automotive applications
1. Multi-lane detection and tracking
3
![Page 4: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/4.jpg)
Outlook
1. Military application: convoy detection and tracking
1. Multi-target tracking, a brief overview
2. Hybridization of CPHD filter and MHT
3. Bayesian network for convoy detection
2. Multi-target detection and tracking with uncalibrated
aerial videos
1. Detection
2. Tracking
3. Automotive applications
1. Multi-lane detection and tracking
4
![Page 5: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/5.jpg)
Battlefield surveillance
zone of interest
GMTI data
Videos SAR images
Geographical information System (GIS)
5
![Page 6: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/6.jpg)
General problem
Goal
6
Situation assessment
How many targets on the scene ?
What is their behavior ?
Are they objects of interest ?
Methods:
• 1st step: Using GMTI sensor to detect agregates
• Algorithm weaknesses for closely spaced target tracking
• Use a promising algorithm: the PHD filter (Probability Hypothesis Density)
• 2nd step: Integrate other data types to determine if the detected
aggregates are convoys or not.
Convoy detection
and if so, how many targets are in
Convoy detection
Methods
![Page 7: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/7.jpg)
GMTI data
• GMTI (Ground Moving Target Indicator) data:
• High traffic density
• High maneuverability of ground targets
• Environment complexity (roads, mountains, ...)
• Sensor limitations: measurement noise, spatial and temporal bias,…
• False alarms , PD<1 and spawned targets
7
![Page 8: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/8.jpg)
Single object tracking
solved by Kalman filter equation
with linear/Gaussian assumption
8
![Page 9: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/9.jpg)
Optimal Bayesian Filter: Kalman filter
1
1
11111
1
1)()()(
k
k
kkkkkkk
k
kkkdxZxfxxfZxf
Propagation of the probability density function (pdf) of xk
[Mahler01] : Detecting, tracking and classifying group targets: a unified approach, Proc.
of SPIE Vol. 4380
)(
)()(
)(1
1
1
k
kkk
k
kkkkkkkk
kkk
Zzf
Zxfxzf
Zxf
Prediction
Update
Estimator )(ˆ supargk
kk
x
kkZxfx
a priori pdf
a posteriori pdf
motion model previous pdf
normalization
likelihood a priori pdf
9
![Page 10: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/10.jpg)
Multi-target tracking
State space
• Varying number of targets:
• Birth targets
• Stationary targets
• Output of the observation
zone
• False alarm
• Non-detection
Observation space
solved by MHT,
JPDAF, particle
filter…
and CPHD filter
10
![Page 11: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/11.jpg)
Random Finite Set (RFS)
Target set Xk modeled as a RFS
k
X
kk
X
kkk
kk
BSX
11
)()(11
• Measurement set Zk modeled as a RFS
k
Xx
kk
k
xZ
)(θ
• : Survival targets between iteration k and iteration k-1
• : Spawned targets
• : Birth targets
1kk
S
1kk
B
k
• : Target originated measurement
• : false alarms k
θ
k
11
![Page 12: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/12.jpg)
Multi-sensor/Multi-target Bayes filter
1
1
11111
1
1)()()(
k
k
kkkkkkk
k
kkkdXZXfXXfZXf
Propagation of the joint probability density function (jpdf) of RFS Xk
[Mahler03] : Multitarget Bayes Filtering via First-Order Multitarget Moments, IEEE AES, Vol. 39, No 4
)(
)()(
)(1
1
1
k
kkk
k
kkkkkkkk
kkk
ZZf
ZXfXZf
ZXf
Prediction
Update
Estimator )(ˆ supargk
kk
X
kkZXfX
a priori jpdf
a posteriori jpdf
motion model previous jpdf
normalization
measurement likelihood a priori jpdf
12
![Page 13: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/13.jpg)
13
PHD definition
S
kdxxvSX )(E
• : first-order statistical moment of the multitarget posterior, also
called intensity function or Probability Hypothesis Density k
v
![Page 14: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/14.jpg)
PHD filter principle
(x))()().((x)111 kkkkskk
dvxfPv
kZz
kkdk
kkd
kkdk
dvzPz
vxzP
vPxv
)(.)g()(
(x)).g(.
(x)1)(
1
1
1
• : survival probability between iteration k and iteration k-1
• : transition function knowing the previous state
• : birth intensity
.1kk
f
k
• : detection probability
• : measurement likelihood
• : clutter intensity
sP
dP
)g( xz
k
Prediction
Estimation
14
![Page 15: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/15.jpg)
Several implementation
[Vo06] : Analytical implementation of the Gaussian Mixture Probability Hypothesis Density
Filter, IEEE SP, 2006
15
![Page 16: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/16.jpg)
Le Cardinalized PHD: principle the number of targets is considered as a random variable p
the corresponding pdf is conjointly propagated over time
[Mahler07] : PHD filters of higher order in target number, IEEE AES, 2007
Prediction
Update
Estimator )(ˆ suparg npN
kk
n
kk
a posteriori pdf normalization
Measurement likelihood a priori pdf
Sum of hypotheses for the n targets to be
(n-j) birth l survival (l-j)non survival
16
![Page 17: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/17.jpg)
Labeling
labeling
17
kkN̂
11
ˆ kk
N
![Page 18: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/18.jpg)
Labeled GM-CPHD (1/2)
• : Gaussian set of size
GG
kNiikikikk
Pmw,...,1,,,
,,
Principle
G
kN
kG
• : track set of size describing the target trajectory kk
N̂k
T
kk
NijkikikikksPx
ˆ,...,1,1,,,,,,ˆ
TT
Evaluate the track-to-Gaussian association matrix of size k
A
0
1),( nm
kA
If the Gaussian component n is associated to the track m
otherwise
G
kkkNN ˆ
weight state covariance
state covariance score
Principle
Goal
18
![Page 19: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/19.jpg)
19
• Maximization of the weight matrix 3 tracks
4 Gaussian components
kW
• Minimization of the cost matrix
Contributions to the labeled GMCPHD
kC
If the Gaussian n can be associated to the predicted track m
Otherwise
0
, n k
k
w W
0 1 . 0 7 . 0 0
6 . 0 0 0 0
0 0 0 9 . 0
k W
0 6 . 0 7 . 0 0
1 . 0 6 . 0 7 . 0 0
0 0 0 9 . 0
k W If the Gaussian n can be associated to the predicted track m
Otherwise
0
) , ( n m c k C
0 18 . 5 67 . 3 0
52 . 0 85 . 2 83 . 4 0
0 0 0 55 . 4
k C
Labeled GMCPHD (2/2)
Means
![Page 20: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/20.jpg)
Comparison between the IMM-MHT and the
GMCPHD filter
IMM-MHT GMCPHD
Target position
estimation ++ +
Target velocity
estimation ++ -
Number of targets
estimation - ++
Computational
complexity + ++
Hybridization
++
++
++
+
Creation of a hybrid algorithm which would combine the
advantages of both of them
• IMM-MHT: Interacting Multiple Model – Multiple Hypothesis Tracker
• GMCPHD: Gaussian Mixture Cardinalized Probability Hypothesis Density
20
![Page 21: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/21.jpg)
Hybridization
[Pollard11] : E. Pollard, B. Pannetier, M. Rombaut, “Hybrid algorithms for Multitarget tracking using the
MHT and the GMCPHD”, IEEE Aerospace and Electronic Systems
21
![Page 22: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/22.jpg)
22
Scenario
Convoy velocity: 10m/s
Isolated target velocity: 15m/s
![Page 23: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/23.jpg)
Root Mean Square Error in position
Convoy of 6 targets Independent
target
quality
23
![Page 24: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/24.jpg)
Root Mean Square Error in velocity
Convoy of 6 targets Independent
target
quality
24
![Page 25: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/25.jpg)
Track length ratio
Convoy of 6 targets Independent
target
quality
25
![Page 26: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/26.jpg)
A convoy detection process
Elaboration of a new algorithm combining the advantages of the
GMCPHD and the IMM-MHT with road constraints
+ No performance drop when targets are close together
Aggregate detection
26
![Page 27: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/27.jpg)
27
Convoy: definition and analysis
• Convoy definition
• Number of targets > 2
• Low and constant velocity
• Military type
• Stay on sight
• On the road
• System analyze
• Asynchronous data
• Heterogeneous data
• Random variables
• Missing data
• Temporal evolution
Dynamic Bayesian Network
X1 X2
X3
3
1
321))((),,(
i
iiXPaXPXXXP
![Page 28: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/28.jpg)
Convoy modeling by using Dynamic
Bayesian Network
k-1 k
[Pollard09a] : E. Pollard, B.
Pannetier, M. Rombaut, “Convoy
detection processing by using the
hybrid algorithm (GMCPHD/VS-
IMMC-MHT) and Dynamic
Bayesian Networks ”, Fusion 2009,
Seattle
28
![Page 29: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/29.jpg)
29
Convoy probablity
Time (in s)
![Page 30: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/30.jpg)
30
Number of target estimation
: set of unique value of
![Page 31: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/31.jpg)
Outlook
1. Military application: convoy detection and tracking
1. Multi-target tracking, a brief overview
2. Hybridization of CPHD filter and MHT
3. Bayesian network for convoy detection
2. Multi-target detection and tracking with uncalibrated
aerial videos
1. Detection
2. Tracking
3. Automobile applications
1. Multi-lane detection and tracking
2. Ego-localisation by fusing GPS and proprioceptive data
31
![Page 32: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/32.jpg)
General problem
Detection
Camera motion
Parallax effects with
urban objects
Low image parameters
Unknown camera
parameters
Tracking Extended targets
Probability of detection <1
Hidden zone in urban areas
Spawned targets
High false alarm rate
[Pollard09b]: E. Pollard, A. Plyer, B.
Pannetier, F. Champagnat, G.
Lebesnerais, “GM-PHD Filters for
Multi-Object Tracking in
Uncalibrated Aerial Videos”, Fusion
2009, Seattle
32
![Page 33: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/33.jpg)
Image motion decomposition
Image motion = ground plane motion
+ parallaxe motion
+ moving object motion
residuals
filtered using
size requirements
Optical flow
algorithm
parametric registration
(homography)
33
![Page 34: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/34.jpg)
Ground plane motion (1/3)
Ransac
Region segmentation
Region selection
Dense estimation
mask
Grond plane motion
regions
inliers
Images
Point tracking
Initial transformation
34
![Page 35: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/35.jpg)
Ground plane motion (2/3)
Ransac
Region segmentation
Region selection
Dense estimation
mask
parametric registration
regions
inliers
Images
Point tracking
Initial transformation
[FOLKI]: G. Le Besnerais, F. Champagnat, “Dense
optical flow estimation by iterative local window
registration”, in ICIP’05, IEEE, vol. 1, p. I-137-140
35
![Page 36: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/36.jpg)
Ground plane motion (3/3)
Ransac
Region segmentation
Region selection
Dense estimation
mask
regions
inliers
Images
Point tracking
Initial transformation
parametric registration
36
![Page 37: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/37.jpg)
Ground plane motion (2/3)
Ransac
Region segmentation
Region selection
Dense estimation
mask
regions
inliers
Images
Point tracking
Initial transformation
parametric registration
37
![Page 38: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/38.jpg)
Postprocessing
Image segmentation based
on the norm of the residual motion
After object selection
Selection of moving objects • Edge detection: select region with high density of edges
• Morphological processing: regularize region shape
• Final selection on area (use prior information on object’s size)
38
![Page 39: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/39.jpg)
Detection results
39
![Page 40: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/40.jpg)
GM-CPHD tracking
40
![Page 41: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/41.jpg)
Outlook
1. Military application: convoy detection and tracking
1. Multi-target tracking, a brief overview
2. Hybridization of CPHD filter and MHT
3. Bayesian network for convoy detection
2. Multi-target detection and tracking with uncalibrated
aerial videos
1. Detection
2. Tracking
3. Automotive applications
1. Multi-lane detection and tracking
41
![Page 42: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/42.jpg)
Intelligent vehicle
processing
unit
laser
actuators HMI
camera radar GPS
gyrometer accelerometer
exteroceptif
proprioceptif odometer
42
![Page 43: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/43.jpg)
X
Y
local map
model
• ego-vehicle
• obstacles
• environment - marking lanes
camera
laser
com
scene
Situation assessment
43
![Page 44: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/44.jpg)
Goals
Multi lane
detection
Multi-camera system
X
Y
2
210xaxaay
Number of marking lines
Shape of marking lines
44
![Page 45: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/45.jpg)
Issues
Missing marking line
Identification ambiguity
Curves
Texture changes
Line width change
Shadow
Light condition change
False alarm
45
![Page 46: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/46.jpg)
General scheme 1. Marking feature extraction
Correction
Prediction Association
Propagation
Kalman Filter
Transferable Belief Model
Estimated
marking
Estimated
marking
Classification
Estimated
marking
Road shape
2. Multi-lane detection 3. Estimation
46
![Page 47: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/47.jpg)
Extraction of road markings primitives
Computed only in a region of interest to limit false points
Horizon
Car cover
(height, variance)
Maximum height
[Pollard11a] : Lane Marking Extraction with Combination Strategy and Comparative
Evaluation on Synthetic and Camera Images, ITSC, 2011 47
![Page 48: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/48.jpg)
Local threshold extractor – LT: Local Threshold
– SLT: Symmetrical Local Threshold
Tg
width
LT
SLT
Drawback
• Depends on the width of
the neighborood
• Only horizontal markings
• Sensitive to marks on the
road
48
![Page 49: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/49.jpg)
Background detection extractor – MLT: Median Local Threshold (50th percentile)
– PLT: Percentil Local threshold (43th percentile)
Median filter
Median value 43th percentile value
49
![Page 50: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/50.jpg)
Multi-lane detection
50
![Page 51: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/51.jpg)
Projection according to the road shape
51/52
![Page 52: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/52.jpg)
Multi-lane tracking
Marking lane 2
210xaxaay
state vector
state eq.
Observations
observation eq
solved by Kalman filter
perspective: use of an IMM for dealing with high curve situation [PollardXX]Road Lane Marker Detection and Estimation: a new algorithm and its complete
Evaluation Process, submitted to IEEE ITS
![Page 53: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/53.jpg)
Performance evaluation
53/52
![Page 54: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/54.jpg)
Conclusion
Multitarget Tracking, Situation Assessment,
Object of Interest Detection
Data Fusion, Uncertainty Management,
Evidence Theory, Bayesian Networks
Particle filtering
GMTI, SAR, GPS, GIS, video images
Thank you for your attention!
54
![Page 55: Data fusion and multitarget tracking: some interests for military and automobile … · 2013. 8. 22. · Automobile applications 1. Multi-lane detection and tracking 2. Ego-localisation](https://reader035.fdocuments.in/reader035/viewer/2022071517/613bf14bf8f21c0c826948e2/html5/thumbnails/55.jpg)
The Joint Directors of Laboratories
[Steinberg98] : Revisions to the JDL data fusion model, Proceedings of SPIE
Multi-target tracking Data fusion
55