Robot Vision SS 2007 Matthias Rüther 1 710.088 ROBOT VISION 2VO 1KU Matthias Rüther.
-
Upload
kathlyn-walton -
Category
Documents
-
view
227 -
download
2
Transcript of Robot Vision SS 2007 Matthias Rüther 1 710.088 ROBOT VISION 2VO 1KU Matthias Rüther.
Robot Vision SS 2007 Matthias Rüther 1
710.088 ROBOT VISION 2VO 1KU
Matthias Rüther
Robot Vision SS 2007 Matthias Rüther 2
Administrative Things
VO: Tuesday 14:30-16:00 HS i11
Strongly coupled with KU!!
www.icg.tu-graz.ac.at/courses
Exam: Written Exam Oral Exam if Requested
KU: Groups of three students Each group does the same project Effort: ~1week per student
Robot Vision SS 2007 Matthias Rüther 3
Time Table
Robot Vision SS 2007 Matthias Rüther 4
Literature
• Sciavicco, L., Siciliano, B., Modelling and Control of Robot Manipulators 2nd Ed., Springer, 2000
• Sonka M., Hlavac V., Boyle Image Processing, Analysis and Machine Vision, Chapman Hall, 1998
• Hartley R., Zissermann A., Multiple View Geometry in Computer Vision, Cambridge, 2001.
Robot Vision SS 2007 Matthias Rüther 5
Student Project
Solve a Computer Vision Problem– From Hardware selection over 3D Measurement to Live Test
Robot Vision SS 2007 Matthias Rüther 6
Goal
Measure 3D Geometry of Electrical Discharges
Impact Area
C1
C2 C3
C4
Robot Vision SS 2007 Matthias Rüther 7
Tasks
Workpackage 1: select hardware, acquire images, segment flash
(xi, yi)
Robot Vision SS 2007 Matthias Rüther 8
Tasks
Workpackage 2: camera calibration and pose estimation
Impact Area
C1
C2 C3
C4
RW, TW
R21, T21
R31, T31
R41, T41
K1
K2K3
K4
Robot Vision SS 2007 Matthias Rüther 9
Tasks
Workpackage 3: correspondence & triangulation
(xi, yi) (xj, yj)
3D
Robot Vision SS 2007 Matthias Rüther 10
Organization
The Project is divided in three workpackages which have to be delivered during the term:– 30.3.2007
– 1.6. 2007
– 22.6. 2007
Each group (3 students) does all three workpackages.
The workpackages build on top of the previous ones. After submission, the workpackages are published.
Each group is allowed to use previous workpackages of any other group.
Robot Vision SS 2007 Matthias Rüther 11
Example
WP1:
WP2:
WP3:
Group 1 Group 2 … Group n
NO Collaboration during workpackage
Group 1 Group 2 … Group n
Group 1 Group 2 … Group n
YES
Each group may reuse previous workpackages of other groups
Robot Vision SS 2007 Matthias Rüther 12
Rules
No collaboration between groups during a workpackage. Copying groups are removed from the KU.
Every group member is held responsible for every task in every workpackage.
Code reuse has no influence on the grade.
Each group must deliver at least two workpackages.
A “Sehr Gut” on the Project gives a 25% Bonus on the Lecture exam on 3.6.2007.
Robot Vision SS 2007 Matthias Rüther 13
Robotics
What is a robot?"A reprogrammable, multifunctional manipulator designed to move
material, parts, tools, or specialized devices through various programmed motions for the performance of a variety of tasks"
Robot Institute of America, 1979
… in a three-dimensional environment.
Industrial– Mostly automatic manipulation of rigid parts with well-known shape in a
specially prepared environment.
Medical– Mostly semi-automatic manipulation of deformable objects in a
naturally created, space limited environment.
Field Robotics– Autonomous control and navigation of a mobile vehicle in an arbitrary
environment.
Robot Vision SS 2007 Matthias Rüther 14
Experimental/Industrial/Commercial Robots
Robot Vision SS 2007 Matthias Rüther 15
Industrial Robots
Robot Vision SS 2007 Matthias Rüther 16
Challenging Environments
Robot Vision SS 2007 Matthias Rüther 17
Service and Assistance
Robot Vision SS 2007 Matthias Rüther 18
FRIEND Project
Robot Vision SS 2007 Matthias Rüther 19
Robot vs Human
Robot Advantages:
– Strength
– Accuracy
– Speed
– Does not tire
– Does repetitive tasks
– Can Measure
Human advantages:
– Intelligence
– Flexibility
– Adaptability
– Skill
– Can Learn
– Can Estimate
Robot Vision SS 2007 Matthias Rüther 20
Robotics: Goals and Applications
Goal: combine robot and human abilities.
Applications: – Automation (Production)
– Inspection (Quality control)
– Remote Sensing (Mapping)
– Man-Machine interaction („Cobot“)
– Robot Companion (Physically challenged people)
– See [Brady, M. et. al. (eds). „Robot Motion: Planning and Control“]
Robot Vision SS 2007 Matthias Rüther 21
Statistics
Yearly installations of industrial robots, 2003-2004 and forecast for 2005-2008
Robot Vision SS 2007 Matthias Rüther 22
Statistics
Estimated operational stock of industrial robots 2003-2004 and forecast for 2005-2008
Robot Vision SS 2007 Matthias Rüther 23
Statistics
Number of robots per 10,000 production workers in the motor vehicle industry 2002 and 2004
Robot Vision SS 2007 Matthias Rüther 24
Statistics
Service robots for professional use. Stock at the end of 2004 and projected installations in 2005-2008
Robot Vision SS 2007 Matthias Rüther 25
Statistics
Service robots for personal and domestic use. Stock and value of stock at the end of 2004 and projected installations in 2005 -2008
Robot Vision SS 2007 Matthias Rüther 26
What can Computer Vision do for Robotics?
Accurate Robot-Object Positioning
Keeping Relative Position under Movement
Visualization / Teaching / Telerobotics
Performing measurements
Object Recognition (see LV „Bildverarbeitung u. Mustererkennung“, „Bildverstehen“, „AK Computer Vision“)
Registration
Visual Servoing
Robot Vision SS 2007 Matthias Rüther 27
Computer Vision
What is Computer Vision?
"Computer Vision describes the automatic deduction of the structure and the properties of a (possible dynamic) three-dimensional world from either a single or multiple two-dimensional images of the world" [Nalva VS, "A Guided Tour of Computer Vision"]
Measurement– Measure shape and material properties in a 3D environment. Accuracy
is important.
Recognition– Cognitive systems interpret a 3D environment (object classification,
categorization). Systems are allowed to fail to a certain extent (similar to humans).
Navigation– Navigation Systems orient themselves in a 3D environment.
Robustness and time are important.
Robot Vision SS 2007 Matthias Rüther 28
Shape from Stereo
Robot Vision SS 2007 Matthias Rüther 29
Shape from Stereo
Robot Vision SS 2007 Matthias Rüther 30
Shape from Focus
Robot Vision SS 2007 Matthias Rüther 31
Shape from Structured Light
Structured Light Sensor
Figures from PRIP, TU Vienna
Robot Vision SS 2007 Matthias Rüther 32
Shape from Shading
Robot Vision SS 2007 Matthias Rüther 33
Navigation
SLAM: Simultaneous Localization and Mapping. – Where am I on my map?
– If the place is unknown, build a new map, try to merge it with the original map.
Visual Odometry: calculate the relative motion of the camera between two frames. Summing up the motion gives the camera path. Error propagation!
Visual Servoing: move to / maintain a relative position between robot end effector and an object.
Tracking: continuously measure the position of an object within the sensor coordinate frame.
Robot Vision SS 2007 Matthias Rüther 34
SLAM
Mapping:
Robot Vision SS 2007 Matthias Rüther 35
SLAM
The final map:
Robot Vision SS 2007 Matthias Rüther 36
SLAM
Navigation:
Robot Vision SS 2007 Matthias Rüther 37
Visual Odometry
Robot Vision SS 2007 Matthias Rüther 38
Visual Servoing
Robot Vision SS 2007 Matthias Rüther 39
Tracking
Robot Vision SS 2007 Matthias Rüther 40
Registration
Registration of CAD models to scene features:
Figures from P.Wunsch: Registration of CAD-Models to Images by Iterative Inverse Perspective Matching