Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym...
Transcript of Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym...
![Page 1: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/1.jpg)
TEK5030
Lecture 10.1 Introduction to Visual SLAM
Trym Vegard Haavardsholm
![Page 2: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/2.jpg)
TEK5030
What is SLAM?
![Page 3: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/3.jpg)
TEK5030
What is SLAM?
• Simultaneous localization and mapping
![Page 4: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/4.jpg)
TEK5030
What is SLAM?
• Simultaneous localization and mapping
• Mapping – Continuously expanding a map while exploring the environment
• Localization
– Localization within the map
![Page 5: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/5.jpg)
TEK5030
What is Visual SLAM?
• Visual simultaneous localization and mapping
• Mapping – Continuously expanding a map while exploring the environment
• Localization (tracking)
– Localization within the map = tracking the map in image frames
![Page 6: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/6.jpg)
TEK5030
Example
![Page 7: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/7.jpg)
TEK5030
How do we track a map?
![Page 8: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/8.jpg)
TEK5030
Pose from known 3D map
![Page 9: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/9.jpg)
TEK5030
Pose from known 3D points
![Page 10: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/10.jpg)
TEK5030
Pose from known 3D points
![Page 11: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/11.jpg)
TEK5030
Pose from known 3D points
• Minimize geometric error
2argmin ( )
cw
wcw cw i i
iπ∗ = −∑
TT T x u
![Page 12: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/12.jpg)
TEK5030
How do we build a map?
![Page 13: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/13.jpg)
TEK5030
Relative pose and 3D from two views
![Page 14: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/14.jpg)
TEK5030
Map initialization and tracking
![Page 15: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/15.jpg)
TEK5030
Map reinitialization and tracking
![Page 16: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/16.jpg)
TEK5030
Map reinitialization and tracking
![Page 17: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/17.jpg)
TEK5030
Map reinitialization and tracking
![Page 18: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/18.jpg)
TEK5030
Map reinitialization and tracking
Drift
![Page 19: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/19.jpg)
TEK5030
Map reinitialization and tracking
Drift
![Page 20: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/20.jpg)
TEK5030
Map reinitialization and tracking
Drift Very naïve Visual Odometry (VO)
![Page 21: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/21.jpg)
TEK5030
Multi-view mapping
![Page 22: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/22.jpg)
TEK5030
Multi-view mapping
![Page 23: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/23.jpg)
TEK5030
Multi-view mapping
![Page 24: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/24.jpg)
TEK5030
Multi-view mapping
Tracking frame
![Page 25: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/25.jpg)
TEK5030
Sliding window mapping
![Page 26: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/26.jpg)
TEK5030
Sliding window mapping
![Page 27: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/27.jpg)
TEK5030
Sliding window mapping
Local map
![Page 28: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/28.jpg)
TEK5030
Sliding window mapping
Global drift
![Page 29: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/29.jpg)
TEK5030
Sliding window mapping
Global drift
![Page 30: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/30.jpg)
TEK5030
Sliding window mapping
No longer map
Global drift
![Page 31: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/31.jpg)
TEK5030
Sliding window mapping
No longer map Typical VO
Global drift
![Page 32: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/32.jpg)
TEK5030
Monocular Visual SLAM
![Page 33: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/33.jpg)
TEK5030
Monocular Visual SLAM
![Page 34: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/34.jpg)
TEK5030
Monocular Visual SLAM
![Page 35: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/35.jpg)
TEK5030
Monocular Visual SLAM
Drift
![Page 36: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/36.jpg)
TEK5030
Monocular Visual SLAM
Drift
![Page 37: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/37.jpg)
TEK5030
Monocular Visual SLAM
Loop closure detection
Drift
![Page 38: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/38.jpg)
TEK5030
Monocular Visual SLAM
Loop closure correction
![Page 39: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/39.jpg)
TEK5030
Monocular Visual SLAM
![Page 40: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/40.jpg)
TEK5030
Visual SLAM vs visual odometry
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015). ORB-SLAM: A Versatile and Accurate Monocular SLAM System. IEEE Transactions on Robotics, 31(5), 1147–1163
![Page 41: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/41.jpg)
TEK5030
Visual SLAM vs visual odometry
Cadena, C., et al. (2016). Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age. IEEE Transactions on Robotics, 32(6), 1309–1332
![Page 42: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/42.jpg)
TEK5030
Components of SLAM
Cadena, C., et al. (2016). Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age. IEEE Transactions on Robotics, 32(6), 1309–1332
![Page 43: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/43.jpg)
TEK5030
Components of VSLAM
• Short-term tracking – Pose estimation given the map – Keyframe proposals
• Long-term tracking
– Visual place recognition – Loop closure detection over keyframes
• Mapping
– Optimizing the map over keyframes – Data fusion
Lowry, S. et al. (2016). Visual Place Recognition: A Survey. IEEE Transactions on Robotics, 32(1), 1–19.
![Page 44: Lecture 10.1 Introduction to Visual SLAM€¦ · Lecture 10.1 Introduction to Visual SLAM Trym Vegard Haavardsholm . TEK5030 What is SLAM? TEK5030 ... (ISMAR'07, Nara), 2007 ... •](https://reader033.fdocuments.in/reader033/viewer/2022052009/601e9119bac64453426e7fb6/html5/thumbnails/44.jpg)
TEK5030
Supplementary material
• “Parallel Tracking and Mapping for Small AR Workspaces”, Klein and Murray, In Proc. International Symposium on Mixed and Augmented Reality (ISMAR'07, Nara), 2007 https://www.robots.ox.ac.uk/~vgg/rg/papers/klein_murray__2007__ptam.pdf
• “Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age”, Cadena et al., IEEE Transactions on Robotics 32 (6) pp 1309-1332, 2016 https://ieeexplore.ieee.org/document/7747236
• “Visual Place Recognition: A Survey“, Lowry, S. et al., IEEE Transactions on Robotics, 32 (1), pp 1–19, 2016 https://ieeexplore.ieee.org/document/7339473