Introduction to Mobile Robotics...
Transcript of Introduction to Mobile Robotics...
![Page 1: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/1.jpg)
1
Wolfram Burgard, Michael Ruhnke,
Bastian Steder
Welcome
Introduction to Mobile Robotics
![Page 2: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/2.jpg)
Today
This course
Robotics in the past and today
2
![Page 3: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/3.jpg)
3
Organization
Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures, discussions
Fr 15:00 – 16:00 homework, practical exercises (Python/Octave)
Web page: www.informatik.uni-freiburg.de/~ais/
Exam: Written
![Page 4: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/4.jpg)
4
Goal of this course
Provide an overview of problems / approaches in mobile robotics
Probabilistic reasoning: Dealing with noisy data
Hands-on experience
![Page 5: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/5.jpg)
Content of this Course
1. Linear Algebra
2. Wheeled Locomotion
3. Sensors
4. Probabilities and Bayes
5. Probabilistic Motion Models
6. Probabilistic Sensor Models
7. Mapping with Known Poses
8. The Kalman Filter
9. The Extended Kalman Filter
10.Discrete Filters
11.The Particle Filter, MCL
5
12. SLAM: Simultaneous Localization and Mapping
13. SLAM: Landmark-based FastSLAM
14. SLAM: Grid-based FastSLAM
15. SLAM: Graph-based SLAM
16. Techniques for 3D Mapping
17. Iterative Closest Points Algorithm
18. Path Planning and Collision Avoidance
19. Multi-Robot Exploration
20. Information-Driven Exploration
21. Summary
![Page 6: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/6.jpg)
Reference Book
Thrun, Burgard, and Fox: “Probabilistic Robotics”
![Page 7: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/7.jpg)
Relevant other Courses
Foundations of Artificial Intelligence
Computer Vision
Machine Learning
and many others from the area of cognitive technical systems.
7
![Page 8: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/8.jpg)
Opportunities
Project
Practical
Seminar
Thesis
… your future!
8
![Page 9: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/9.jpg)
Autonomous Robot Systems
perceive their environment and
generate actions to achieve their goals.
environment
sense
act
model
![Page 10: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/10.jpg)
Tasks Addressed that Need to be Solved by Robots
Navigation Perception Learning Cooperation Acting Interaction Robot development Manipulation Grasping Planning Reasoning
…
![Page 11: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/11.jpg)
12
Robotics Yesterday
![Page 12: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/12.jpg)
13
Current Trends in Robotics
Robots are moving away from factory floors to
Entertainment, toys
Personal services
Medical, surgery
Industrial automation (mining, harvesting, …)
Hazardous environments (space, underwater)
![Page 13: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/13.jpg)
14
Shakey the Robot (1966)
![Page 14: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/14.jpg)
15
Shakey the Robot (1966)
![Page 15: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/15.jpg)
Robotics Today
Lawn mowers
Vacuum cleaners
Self-driving cars
Logistics
…
17
![Page 16: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/16.jpg)
18
The Helpmate System
![Page 17: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/17.jpg)
Autonomous Vacuum Cleaners
![Page 18: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/18.jpg)
Autonomous Lawn Mowers
20
![Page 19: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/19.jpg)
21
DARPA Grand Challenge
[Courtesy by Sebastian Thrun]
![Page 20: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/20.jpg)
Die DARPA Urban Challenge
![Page 21: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/21.jpg)
Walking Robots
[Courtesy by Boston Dynamics]
![Page 22: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/22.jpg)
Humanoids Climbing Staircases
![Page 23: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/23.jpg)
Androids
Overcoming the uncanny valley
[Courtesy by Hiroshi Ishiguro]
![Page 24: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/24.jpg)
Driving in the Google Car
![Page 25: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/25.jpg)
Autonomous Motorcycles
[Courtesy by Anthony Levandowski]
![Page 26: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/26.jpg)
The Google Self Driving Car
28
![Page 27: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/27.jpg)
Folding Towels
![Page 28: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/28.jpg)
30
Rhino (Univ. Bonn + CMU, 1997)
![Page 29: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/29.jpg)
31
Minerva (CMU + Univ. Bonn, 1998)
Minerva
![Page 30: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/30.jpg)
Robotics in Freiburg
32
![Page 31: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/31.jpg)
Autonomous Parking
![Page 32: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/32.jpg)
Autonomous Quadrotor Navigation
Custom-built system:
laser range finder
inertial measurement unit
embedded CPU
laser mirror
![Page 33: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/33.jpg)
Precise Localization and Positioning for Mobile Robots
![Page 34: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/34.jpg)
Obelix – A Robot Traveling to Downtown Freiburg
![Page 35: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/35.jpg)
The Obelix Challenge (Aug 21, 2012)
![Page 36: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/36.jpg)
The Tagesthemen-Report
![Page 37: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/37.jpg)
Brain-controlled Robots
39
![Page 38: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/38.jpg)
Teaching: Student Project on the Autonomous Portrait Robot
![Page 39: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/39.jpg)
Final Result
![Page 40: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/40.jpg)
Other Cool Stuff from AIS
42
![Page 41: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/41.jpg)
Accurate Localization
KUKA omniMove (11t)
Safety scanners
Error in the area of millimeters
Even in dynamic environments
![Page 42: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/42.jpg)
Fuselage assembly
20 vehicles to transport industrial robots for drilling and filling of 60,000 fasteners in
6 vehicles for logistics of parts, work stands and fuselages
26 Units installed at Boeing
![Page 43: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/43.jpg)
![Page 44: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/44.jpg)
Deep Learning to Manipulate from Parallel Interaction
Source: Google Research Blog
![Page 45: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/45.jpg)
Learning User Preferences
Task preferences are subjective
Fixed rules do not match all users
Constantly querying humans is suboptimal
How to handle new objects?
Where does this
go?
![Page 46: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/46.jpg)
?
Collaborative Filtering
•…
•-
•-
•-
![Page 47: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/47.jpg)
Collaborative Filtering
•…
•-
•-
•-
![Page 48: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/48.jpg)
Online Prediction of Preferences
![Page 49: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/49.jpg)
Localization in Urban Environments
Inaccurate (if even available) GPS signal
No map
Limited Internet
![Page 50: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/50.jpg)
Motivation
![Page 51: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/51.jpg)
Example
![Page 52: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/52.jpg)
Example contin.
Text: irpostbankfmarzcenter tllgi
Matched Landmarks:
Postbank finanzcenter
Text: melange
Matched Landmarks:
Melange
Melange
Text: casanova
Matched Landmarks:
Casanova
![Page 53: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/53.jpg)
Example
![Page 54: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/54.jpg)
Deep Learning Applications
RGB-D object
recognition
Images human part
segmentation
Sound terrain
classification
![Page 55: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/55.jpg)
Fusion layers automatically learn to combine feature responses of the two network streams
During training, weights in first layers stay fixed
DCN for Object Recognition
![Page 56: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/56.jpg)
Learning Results
Method RGB Depth RGB-D
CNN-RNN 80.8 78.9 86.8
HMP 82.4 81.2 87.5
CaRFs N/A N/A 88.1
CNN Features 83.1 N/A 89.4
This work, Fus-CNN 84.1 83.8 91.3
•Category-Level Recognition [%] (51 categories)
•[Lai et. al, 2011]
![Page 57: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/57.jpg)
Network Architecture
Fully convolutional network
Contraction and expansion of network input
Up-convolution operation for expansion
Pixel input, pixel output
![Page 58: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/58.jpg)
Deep Learning for Body Part Segmentation
•Input Image •Ground Truth •Segmentation mask
![Page 59: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/59.jpg)
Deep Learning for Terrain Classification using Sound
![Page 60: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/60.jpg)
Network Architecture
Novel architecture designed for unstructured sound data
Global pooling gathers statistics of learned features across time
![Page 61: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/61.jpg)
Data Collection
Asphalt
Wood
Offroad
CobbleStone
Paving Grass Mowed Grass
Carpet Linoleum
P3-DX
![Page 62: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/62.jpg)
Results - Baseline Comparison
16.9% improvement over the previous state of the art
99.41% using a 500ms window
(300ms window)
[1]
[2]
[3]
[4]
[5]
[6]
[1] T. Giannakopoulos, K. Dimitrios, A. Andreas, and T. Sergios, SETN 2006
[2] M. C. Wellman, N. Srour, and D. B. Hillis, SPIE 1997.
[3] J. Libby and A. Stentz, ICRA 2012
[4] D. Ellis, ISMIR 2007
[5] G. Tzanetakis and P. Cook, IEEE TASLP 2002
[6] V. Brijesh , and M. Blumenstein, Pattern Recognition Technologies and Applications 2008
![Page 63: Introduction to Mobile Robotics Welcomeais.informatik.uni-freiburg.de/teaching/ss16/robotics/slides/00-intro.pdf · 3 Organization Wed 14:00 – 16:00 Fr 14:00 – 15:00 lectures,](https://reader033.fdocuments.in/reader033/viewer/2022051606/601d6d722408bf712d0bf3c2/html5/thumbnails/63.jpg)
Thank you
… and enjoy the course!
66