Vision-Based Robot Control

Bax SmithTerm 8 Electrical (CC)

Outline

• How I came up with this project• Components of the project• Visual Demo• Conclusion

Vision-Based Robot Control• Person: 5 Senses

Robot: 1 Sense• Person: Mobile

Robot: Mobile• Person: Can Grasp

ObjectsRobot: Can Grasp Objects

• Person: Intelligent• Robot: ?

The Result: Sambuca

Applications for Sambuca

• Get me beer from fridge

• Vacuum carpet• Mow the lawn• Give tours• Recycle!

Illustrates the point and Saves the environment!

Components of Sambuca

• Vision System• Mobility System

• Intelligence System• Arm System

Sambuca - Vision System

• Hardware: Webcam• The Problem:

Distinguishing Cans/Bottles from other things

• The Solution: Image Processing and Feature Extraction

Sambuca - Mobility System

• Hardware: Electric Wheelchair Base

• Turns by reversing direction of wheel rotations

Sambuca - Intelligence System

• What to do if it sees an object?• How far away is the object?• Is on the left or right?• What about multiple objects?• How to get to the object?

Sambuca - Intelligence - FSM

• Scavenger - look for object

• Retrieval - grasp object

• Deliver - put in bin

START DETECT

GRABRELEASE

DONE

Scavenger Retrieval

Delivery

Sambuca - Arm System

• RT200 Robotic Arm• Go to Point - Acknowledge

PC -Intelligence

RT200 Robotic Arm

Serial

Sambuca in Action

Conclusion

• Finds and distinguishes objects very well• Tracks and follows targets very well• Arm could do with its own vision system• Mobile Base could do with its own Control

System/Encoders