UGV & AUV SLAM and Mapping
Transcript of UGV & AUV SLAM and Mapping
![Page 1: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/1.jpg)
UGV & AUVSLAM and Mapping
Simon Lacroix Robotics and AI
LAAS/CNRS, Toulouse, France
1
With contributions from: Anthony Mallet, Il-Kyun Jung, Thomas Lemaire, Sébastien Bosch, Joan Sola, Cyrille Berger and Teresa Vidal … and others from the robotics and vision communities
![Page 2: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/2.jpg)
UGV & AUVSimultaneous Localization
and Mappingand Mapping
with vision
2
![Page 3: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/3.jpg)
Contents
• UGVs & UAVs : outdoor (field) robotics • Some computer vision • Thoughts, beliefs, feelings, claims,
(parentheses)…
3
![Page 4: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/4.jpg)
Why do we need maps ? • To plan motions
– evaluate/quantify possible motions • To plan tasks
– perception, exploration, surveillance – in a multi-robot context
• To achieve motions – e.g. servo along a path/road/wall…
• To interact – With users and operators – With other robots
• To localize the robot (yes, this is the SLAM summer school) – Missions defined in localization terms (“reach goal”,
“explore area”…) – To ensure the proper execution of motions – To ensure the spatial consistency of the maps
![Page 5: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/5.jpg)
Why do we need maps ?
• Geographic information systems: multi-layered maps
![Page 6: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/6.jpg)
Why do we need maps ?
• “Robot GIS”
…
n
Threats, targets…
6
Landmarks
5
Road network
4
Color and texture
3
3D geometric model
2
Digital terrain model
1
![Page 7: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/7.jpg)
Why vision ? • Cameras : low cost, light
and power-saving
• Perceive data – In a volume – Very far – Very precisely
1024 x 1024 pixels 60º x 60º FOV
⇓ 0.06 º pixel resolution
(1.0 cm at 10.0 m)
• Stereovision – 2 cameras
provide depth ⊕ ⇒
• Images carry a vast amount of information, at high rates
• A vast know-how exists in the computer vision community
Micro UAVs, planetary rovers
![Page 8: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/8.jpg)
Vision-based SLAM
– Landmark detection – Relative observations (measures)
• Of the landmark positions • Of the robot motions
– Observation associations – Estimation: refinement of the
landmark and robot positions
Functions required by any SLAM implementation :
Plus: – Choice of the landmark representation
Vision brings something
![Page 9: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/9.jpg)
Vision-based SLAM maps
9
Landmarks for estimation purposes: visual features (interest points, patches – here represented with the associated normal)
⊕
View-based representation for loop-closing detection purposes (image or places “indexes”)
![Page 10: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/10.jpg)
Vision-based SLAM maps
⊕
View-based representation for loop-closing detection purposes (image or places “indexes”)
Landmarks for estimation purposes: visual features (interest points, patches – here represented with the associated normal)
![Page 11: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/11.jpg)
Vision-based SLAM maps
⊕
Who wants such maps ?
![Page 12: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/12.jpg)
Outline
UGV & AUV SLAM and Mapping
• UGV Mapping
• UAV Mapping
• “&”
![Page 13: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/13.jpg)
(A few words on stereovision)
13
• The way humans perceive depth
Stereo image pair Stereo images viewer Stereo camera
• Very popular since the early 20th century
• Anaglyphs
Polarization Red/Blue
![Page 14: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/14.jpg)
(A few words on stereovision) • In 2 dimensions (two linear cameras):
14
Right camera
β
b
Right image
Disparity x
€
x =b
tan(α) + tan(β)Left camera
Left image α
Stereovision = depth by triangulation Two problems at hand:
• Finding matches • Determining the system geometry
![Page 15: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/15.jpg)
(A few words on stereovision) • Now with real images
15
Le) image Right image
Where is in ?
![Page 16: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/16.jpg)
pr2 pr1
(A few words on stereovision)
• Geometry of stereovision
16
x
y
z x
y
z
Ol Or
P
pl
P1
P2
pr
![Page 17: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/17.jpg)
(A few words on stereovision)
• Geometry of stereovision
17
x
y
z x
y
z
Ol Or
P
pl
pr
“Epipolar geometry”
![Page 18: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/18.jpg)
(A few words on stereovision)
• Finding matches
18
… 3 6 3 7 9 2 8 7 6 8 9 6 4 9 0 9 9 0 …
… 3 5 7 4 9 6 3 9 6 5 8 6 3 0 1 9 7 5 …
Le) line
Right line
???
The matches are computed on windows
Several ways to compare windows: “SAD”, “SSD”, “ZNCC”, Hamming distance on census‐transformed images…
![Page 19: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/19.jpg)
(A few words on stereovision)
19
Original image Disparity image 3D image
640x480 @ 30Hz on a standard CPU
![Page 20: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/20.jpg)
(A few words on stereovision)
• Error model
20
)( cfd =σempirical analysis:
Maximal errors : 0.4m baseline: 2310 xx−≤σ
1.2m baseline: 2410.3 xx−≤σ
Online estimation of the errors
• Errors on the 3D coordinates : 2xd
x dx α
σσ
α=⇒=
• How to get an estimate of ? Analyze the correlation curves
€
σ d
![Page 21: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/21.jpg)
(A few words on stereovision)
21
• Difficulties
1. “obstacle growing”
2. “wavelets”
Left mage 3D points
3. Discontinuity smoothing 4. Lack of data 5. Calibration 6. …
![Page 22: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/22.jpg)
Digital terrain map • DTM: on a regular Cartesian grid
22
€
z = f (x,y)
Ground rover case:
• Varying resolution
• Imprecision on the data uncertainties in the values volumic occupancy grid ?
![Page 23: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/23.jpg)
(Occupancy grids) • Mapping with “occupancy grids”:
– 2D regular discretization of the environment – Encodes the probability of presence of an
obstacle (“occupied”)
23
Example with sonar data • Sensor model: probability of range reading given known occupancy at a known distance :
€
P(m = dOi)
€
P(Oi m =d) =P(m = d Oi)P(Oi)
P(m = d Oi)P(Oi) + P(m = d O i)P(O i)
Sensor model Initial knowledge • Bayes theorem
![Page 24: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/24.jpg)
(Occupancy grids) Well suited for 2D mapping indoor environments with telemetry
![Page 25: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/25.jpg)
Digital terrain map
• Confidence model of stereo:
• Model parametrization: (this is “only” a model) • Turning the model into a pdf:
€
P(m /O)
€
P(O /m) =P(m /O)P(O)
P(m)=
P(m /O)P(O)P(m /O)P(O) + P(m /¬O)P(¬O)
• Probability updates:
€
P(O /m1,m2) =P(m2 /O)P(O /m1)
P(m2 /O)P(O /m1) + P(m2 /¬O)P(¬O /m1)
![Page 26: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/26.jpg)
Digital terrain map
Simulation results on a 2D profile:
One acquisition
Fusion of 36 acquisitions
Update strategy Bayes Dempster-Shafer
Not (yet) tractable for large 3D environments
![Page 27: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/27.jpg)
Digital terrain map
Practical implementation 1: • simple statistics on the “population” of 3D points that fall in a cell • a cell is : (z, σz)
Practical implementation 2: • Each 3D point is associated to a surfacic patch, fused in the model according to a confidence value (Dempster-Shafer like approach) • a cell is : (z, c, σz)
![Page 28: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/28.jpg)
Digital terrain map • Dealing with dynamic elements and verticals
DTMs are suited to represent the ground : z=f(x,y) (“2.5 D”) Label the patches according to , monitor the patches evolution over time
€
σ z
![Page 29: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/29.jpg)
Digital terrain map • Dealing with dynamic elements and verticals
DTMs are suited to represent the ground : z=f(x,y) (“2.5 D”) Label the patches according to , monitor the patches evolution over time
€
σ z
![Page 30: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/30.jpg)
Digital terrain map • But we took localization for granted ! What if the pose is
refined when closing a loop ?
Before loop closing A2er loop closing
?
Applying a deformaZon on the grid ? (what deformaZon ?) Storing all the acquired 3D images and re‐merging them ?
![Page 31: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/31.jpg)
Digital terrain map • But we took localization for granted ! What if the pose is
refined when closing a loop ? “DenseSLAM”: Hybrid metric maps (Nieto&Nebot@ACFR)
Principle: anchor local dense maps to highly correlated landmarks
Global map Local region
![Page 32: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/32.jpg)
Exploiting DTMs
• To localize the rover – Extracting landmarks from the DTM ? (e.g. peaks)
– By correlating local DTMs (or raw 3D data wrt. the current DTM) ?
32
t t+1
Exemple: minimizing the distance between the new 3D image and the current DTM (simplex algorithm, ICP…)
Distance as a function of X and Y
![Page 33: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/33.jpg)
Exploiting DTMs
• To determine/plan feasible trajectories
33
![Page 34: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/34.jpg)
Exploiting DTMs
• To determine/plan feasible trajectories
34
Evaluation of one position: convolution of the robot chassis wrt. the DTM
![Page 35: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/35.jpg)
(parenthesis: this is “perception”)
• To determine/plan feasible trajectories
35
Evaluation of one position: convolution of the robot chassis wrt. the DTM
![Page 36: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/36.jpg)
Traversability maps
36
![Page 37: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/37.jpg)
Traversability maps
37
1. Discretisation of the perceived area
2. Probabilistic labelling
Correlated pixels Labeling
(top view) labeling
(sensor view)
Flat Obstacle Unknown
![Page 38: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/38.jpg)
Traversability maps
2. Probabilistic labelling • Attributes computed for each cell :
• Point density • Mean elevation and standard deviation • Mean normal vector • …
• Bayesian classification
€
P(Ci /A) =P(A /Ci)P(Ci)
P(A)
€
P(A /Ci) (usually not Gaussians)
![Page 39: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/39.jpg)
Traversability maps 3. Merging maps: application of the Bayes rule (3 classes here)
![Page 40: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/40.jpg)
Traversability maps 4. Extension: introduction of color/texture attributes
![Page 41: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/41.jpg)
(“map-free” motion generation)
• Servo on particular perceived (“mapped”) elements – e.g. path following
41
Perception = trail detection, from texture and color segmentation (Cf “LAGR” program)
![Page 42: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/42.jpg)
Exploiting traversability maps
• Trajectory determination • Potential fields navigation
42
![Page 43: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/43.jpg)
Exploiting traversability maps • Long term navigation. Knowing:
• Where I have to go (my goal, my mission) • What I know on the environment • How I can know more on the environment • How I can move (my motion modes)
Where should I head to ? How ? What for ?
43
NavigaZon vs. ExploraZon ? EssenZal informaZon in maps: the amount/quality of encoded informaZon
![Page 44: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/44.jpg)
Outline
UGV & AUV SLAM and Mapping
• UGV Mapping – DTM – Traversability maps
• UAV Mapping
• “&”
![Page 45: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/45.jpg)
DTM from aerial stereo imagery
• Good projection characteristics (wrt. UGVs)
45
2.4m stereovision bench, mounted on a blimp
![Page 46: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/46.jpg)
DTM from aerial stereo imagery
46
![Page 47: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/47.jpg)
DTM from aerial imagery • Binocular stereo is a bad idea for UAVs “Multi-view stereovision” (@Onera)
47
1. Data acquisiZon
2. Data registraZon (full SLAM, bundle adjustment)
![Page 48: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/48.jpg)
DTM from aerial imagery “Multi-view stereovision” (@Onera)
48
1. Data acquisiZon
2. Data registraZon (full SLAM, bundle adjustment)
3. Building a DTM: • For each DTM patch, discreZze
the possible heights • For each height hypothesis,
recover the associated pixels in the sequence
• Analyse the pixels sequence to declare the “likelihood” of the hypothesis
• Detect occlusions, apply regularizaZon…
![Page 49: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/49.jpg)
DTM from aerial imagery “Multi-view stereovision” (@Onera)
49
![Page 50: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/50.jpg)
DTM from aerial imagery “Multi-view stereovision”
50
3cm resoluZon DTM from low alZtude UAV (far from real‐Zme) (M.P. Deseilligny @ TeledetecZon lab, Montpellier)
![Page 51: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/51.jpg)
Traversability maps from aerial imagery
![Page 52: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/52.jpg)
(what is a homography ?)
TransformaZon in the 2D plane:
The relaZon between the images of coplanar points viewed from arbitrary camera posiZons is a homography
If there is a homography that relates two regions of two images acquired with t≠0, the region is planar
![Page 53: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/53.jpg)
Traversability maps from aerial imagery
53
1st approach Relies on homography computaZon ‐ no assumpZon (neither esZmate)
of the camera moZon
img1
img2
![Page 54: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/54.jpg)
Traversability maps from aerial imagery
54
1st approach Relies on homography computaZon ‐ no assumpZon (neither esZmate)
of the camera moZon
img1
img2
![Page 55: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/55.jpg)
Traversability maps from aerial imagery
55
1st approach Homography computaZon
![Page 56: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/56.jpg)
Traversability maps from aerial imagery
56
1st approach Relies on homography computaZon ‐ no assumpZon (neither esZmate)
of the camera moZon
![Page 57: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/57.jpg)
Traversability maps from aerial imagery
57
Merging traversability map: use the associated orthoimage
“True” orthoimages
![Page 58: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/58.jpg)
Traversability maps from aerial imagery
58
2nd approach (@Onera)
OpZc flow on the “H‐registered” images
img1
H(im
g2)
![Page 59: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/59.jpg)
Traversability maps from aerial imagery
59
img1
H(im
g2)
2nd approach (@Onera)
OpZc flow on the “H‐registered” images
• Hints on verZcal disconZnuZes • Towards a DTM ?
![Page 60: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/60.jpg)
Traversability maps from aerial imagery 2nd approach (@Onera)
OpZc flow on the “H‐registered” images
![Page 61: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/61.jpg)
Outline
UGV & AUV SLAM and Mapping
• UGV Mapping
• UAV Mapping – DTM – Traversability maps
• “&”
![Page 62: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/62.jpg)
Air/Ground cooperation • Field robotics main missions / tasks:
• Exploration, reconnaissance (information gathering)
• Monitoring, surveillance • Intervention (rescue, fire
fighting…)
62
• For all these tasks, air/ground robotics systems bring forth several operational and robotics capacities
![Page 63: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/63.jpg)
Air/Ground cooperation
63
• Ground robots
Good at: Precise informaZon gathering Physical intervenZon Long duraZon missions Heavy load carrying
Not so good at: Global informaZon gathering Self localizaZon High speed mobility Avoiding hazards
• Aerial robots
Good at: Global informaZon gathering High speed mobility Avoiding hazards CommunicaZon relaying
Not so good at: Long duraZon missions Physical intervenZon Heavy load carrying
![Page 64: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/64.jpg)
Air/Ground cooperation schemes
64
UAVs assist UGVs • LocalizaZon • CommunicaZon relay • Environment modeling
UGVs assist UAVs • Detect clear landing areas • Carry on UAVs • Provide energy support
UAVs and UGVs cooperate to achieve a task • ExploraZon • Monitoring • IntervenZon • …
![Page 65: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/65.jpg)
Air/Ground cooperation: issues
65
“Usual” mulZ‐robot issues: • Task allocaZon • Task planning (incl. communicaZons) • CoordinaZon (supervision) • Inter‐robot servoing • … Managing informaZon
and decision sharing (incl. the operators)
Environment modeling: at the heart of cooperaZon
![Page 66: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/66.jpg)
Air/Ground cooperation Illustration: rover navigation assisted by aerial observation
66
![Page 67: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/67.jpg)
Air/Ground cooperation Illustration: rover navigation assisted by aerial observation
67
![Page 68: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/68.jpg)
Air/Ground cooperation • Environment modeling: at the heart of cooperation Key prerequisite: register aerial and ground data/maps
68
![Page 69: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/69.jpg)
Air/Ground traversability map registration ?
69
From the ground
From the air
![Page 70: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/70.jpg)
Air/Ground orthoimages registration ?
70
Aerial orthoimage Orthoimage derived from ground DTM
![Page 71: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/71.jpg)
Air/Ground orthoimages registration ?
71
Aerial orthoimage Orthoimage derived from ground DTM
![Page 72: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/72.jpg)
Air/Ground data registration ?
72
Air Ground
![Page 73: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/73.jpg)
Air/Ground DTM registration ?
73
Air Ground
Nice soluZon in [Vandapel‐Hebert‐IJRR‐2006] (spin images approach – cf difficulZes with peaks)
![Page 74: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/74.jpg)
Common landmarks
Air/Ground registration ?
74
Landmarks Landmarks
UGV maps UAV maps
![Page 75: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/75.jpg)
Air/Ground registration
• Main issue: find common landmarks What information is really invariant wrt. viewpoints, camera
characteristics, environment type, sensor type ?
75
geometry
Building 3D models in a wholesome way
![Page 76: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/76.jpg)
Towards richer 3D SLAM
• Visual line segments – Robust extraction remains a
bit challenging – (model-driven approaches
seem better than data driven approaches)
76
![Page 77: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/77.jpg)
Towards richer 3D SLAM
• Visual line segments – Robust extraction remains a
bit challenging – (model-driven approaches
seem better than data driven approaches)
– A good tracker/matcher is so desirable (SLAM will help)
77
![Page 78: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/78.jpg)
Towards richer 3D SLAM
• Visual line segments – Robust extraction remains a
bit challenging – (model-driven approaches
seem better than data driven approaches)
– A good tracker/matcher is so desirable (SLAM will help)
78
![Page 79: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/79.jpg)
Towards richer 3D SLAM
• Monocular line segments SLAM – Estimate the supporting line (not the endpoints) – Plücker parameterization
– Good properties for SLAM: simple expressions for frame transformations and observations
79
![Page 80: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/80.jpg)
Towards richer 3D SLAM
• Monocular line segments SLAM – Landmark initialization
80
![Page 81: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/81.jpg)
Towards richer 3D SLAM
• Monocular line segments SLAM
81
![Page 82: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/82.jpg)
(monocular visual SLAM) • Early work (Davison, 2003): delayed landmark
initialization
82
![Page 83: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/83.jpg)
(monocular visual SLAM) • Early work (Davison, 2003): delayed landmark
initialization
83
![Page 84: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/84.jpg)
(monocular visual SLAM) • Early work (Davison, 2003): delayed landmark
initialization A much better approach: inverse-depth parameterization
(undelayed initialization)
84
![Page 85: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/85.jpg)
Towards richer 3D SLAM
• Monocular line segments SLAM – Early work: delayed landmark initialization A much better approach: “pseudo” inverse-depth
parameterization (undelayed initialization) (on going work)
85
![Page 86: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/86.jpg)
Towards richer 3D SLAM
• Monocular line segments SLAM – Early work: delayed landmark initialization A much better approach: “pseudo” inverse-depth
parameterization (undelayed initialization) (on going work)
86
![Page 87: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/87.jpg)
Towards richer 3D SLAM
• Stereo segments • Planar patches
– Extracted from stereo data using homographies
87
![Page 88: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/88.jpg)
Towards richer 3D SLAM
• Stereo segments • Planar patches • Planes
• Extracted from monocular sequences using homographies
88
[SILVEIRA‐ICRA‐07]
![Page 89: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/89.jpg)
Towards richer 3D SLAM
• Stereo segments • Planar patches • Planes • Collapsing landmarks into 3D structures
89
+ + + …
![Page 90: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/90.jpg)
Towards air/ground 3D SLAM • Multi-robot SLAM: “various” loop closure events
1. “Rendez-vous”: inter-robot relative localization 2. Map matching 3. Absolute localization: GPS fix 4. Absolute localization: localization wrt. an initial map
3D geometry is a key for 2. and 4.
• On the estimation side: – Distributed management of various maps
90
![Page 91: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/91.jpg)
Towards air/ground 3D SLAM • Illustration. A mix of:
– Multi-robot multiple local maps (akin to hierarchical SLAM)
– Point landmarks, inverse depth parameterization
– Line segments, stereovision
91
![Page 92: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/92.jpg)
Outline
UGV & AUV SLAM and Mapping
• UGV Mapping
• UAV Mapping
• “&”
![Page 93: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/93.jpg)
Maps, maps, maps…
93
• Geographic information systems
Maps of everywhere, available everywhere ! (googleEarth, virtualEarth, geoportail, Nasa WorldWind, flashEarth…)
![Page 94: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/94.jpg)
Maps, maps, maps… Mars images @ 0.25m resoluZon, DTM @ 1m (hirise.lpl.arizona.edu)
![Page 95: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/95.jpg)
Maps, maps, maps !
![Page 96: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/96.jpg)
Maps, maps, maps…
!!
!!!!!!!!!!!"#$"%!&'$$"("!)!*+,-".+!/00'11'-2+3'.!
!
! 456!)!!7"8"!+7"!$"0+,8"#!28"!7"$-!3.!+7"!1'8.3.(!
! !
&'19,+"8!:2;#!)!<'8!+7"!9820+302$!#"##3'.#!3.!+7"!2<+"8.''.!
!
/&=6!>,3$-3.(! !
! !
! ?"-3.2!@'+"$!)!48"#".+"8#!/00'11'-2+3'.!
We must endow our robots with the ability to use these 3D geometry in one of the key
Summer School maps
![Page 97: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/97.jpg)
Questions ?
![Page 98: UGV & AUV SLAM and Mapping](https://reader034.fdocuments.in/reader034/viewer/2022042520/586e17781a28ab01648b8987/html5/thumbnails/98.jpg)
Answers !