If you have any questions please feel free to interrupt me.
-
Upload
solomon-brown -
Category
Documents
-
view
212 -
download
0
Transcript of If you have any questions please feel free to interrupt me.
If you have any If you have any questions questions
please feel free please feel free to interrupt meto interrupt me
The vision The vision system for Marie system for Marie
CurieCurie
Main Tasks in our Main Tasks in our systemsystem
Image recognition
Machine Learning
Control of robot’s behavior
Environment
Marie Curie will be communicating Marie Curie will be communicating with Schroedinger’s Cat robotwith Schroedinger’s Cat robot
Interaction of Marie Curie and Interaction of Marie Curie and CatCat
1. Marie Curie does not change its main coordinates, she can only move her hands, head and legs, but she remains attached to the desk.
2. Cat can move freely in the area of the stage.3. Cat should not bump into Marie or furniture
of the lab.4. Marie should know where the cat is located
and look to him5. Cat should know where Marie is located and
talk to her.
Ideal view of the ceiling Ideal view of the ceiling cameracamera
Marie Curie
shelf
equipment
equipment equipment
Black curtainCat
There will be a Kinect camera looking There will be a Kinect camera looking from the ceiling to the stage from the ceiling to the stage
The role of the ceiling cameraThe role of the ceiling camera
1. The camera will be attached to the ceiling or will be in some position very high, as high as we can.
2. We have done something similar but the robots were small.
3. The camera should know x,y coordinates of every robot and its orientation (pose)
4. Marie Curie does not change its main coordinates, so it is easy
5. Cat is fast so we have to track the triplet (x, y, (x, y, ))
There is nothing like There is nothing like that in Disneylandthat in Disneyland
All behaviors of robots in commercial theatres are strictly scripted, robots move on rails, they cannot make an error.In our case we have interaction, improvisation, and robots are subject to noisy behaviors.This task is more similar to robot soccer than to existing robot theatres in the world.
There will be another There will be another camera looking to faces of camera looking to faces of
the audiencethe audienceWe will call it the human-control camera or a front camera
The role of the front cameraThe role of the front camera1. The camera is attached to the wall near the
glass window of the theatre, looking towards humans located in the corridor.
2. This camera will look at the audience3. There are several goals of having this camera
1.1. Recognizing (x, y, Recognizing (x, y, ) of every person that ) of every person that looks at the performance (perhaps not more looks at the performance (perhaps not more than 5).than 5).
2.2. Recognizing the emotion on the faces of these Recognizing the emotion on the faces of these people.people.
3.3. Recognizing their gestures with hands and Recognizing their gestures with hands and legs, full bodies and faces.legs, full bodies and faces.
4.4. Use these data to control the behavior of the Use these data to control the behavior of the robots, songs selected, slides selected, lights robots, songs selected, slides selected, lights and other effects.and other effects.
There is nothing like There is nothing like that in Disneylandthat in Disneyland
All behaviors of robots in Disneyland are strictly scripted.
Rarely humans can change robots’ behaviors.
This is a new task for our This is a new task for our teamteam
Marek Perkowski has never done anything like this beforePerhaps nobody in the world has done something like this.This is good as we are doing something new.Hopefully we have done something similar and have a good experience from the past.We were doing ROBOT SOCCER.We will try now to use our past experience and theory for a new task.
Ideal view of the ceiling Ideal view of the ceiling cameracamera
Marie Curie
shelf
equipment
equipment equipment
Black curtainCat
This is ideal, in reality the image will be much This is ideal, in reality the image will be much distorted with noise and lightning and geometrydistorted with noise and lightning and geometry
Marie Curie
shelf
equipment
equipment equipment
Black curtainCat
X axis
Y axis
c
Xc
Yc
The idea of The idea of Robot Robot SoccerSoccer
3. Robot Soccer and 3. Robot Soccer and Similar TasksSimilar Tasks
• Robot Soccer Competition– RoboCup– FIRA– Remote controlled systems– Autonomous robots
• Clustering
3.1 Robot Soccer3.1 Robot Soccer“RoboCup is an international joint project to promote AI,robotics, and related fields.
It is an attempt to foster AI andintelligent robotics research by providing a standard problemwhere a wide range of technologies can be integrated andexamined.
RoboCup chose to use the soccer game as a centraltopic of research, aiming at innovations to be applied forsocially significant problems and industries.
The ultimate goalof the RoboCup project is: By 2050, develop a team of fullyautonomous humanoid robots that can win against the humanworld champion team in soccer.” [RoboCup 1998]
As you see, it is difficult to approximate every robot with a rectangle. It will be even more difficult in our case.
Overhead Overhead VisionVision
• Our goal is to start with Overhead Vision (Ceiling Camera) and check how it will work.
• We may move to more cameras if necessary.
Local Local VisionVision
Design Criteria for robot Design Criteria for robot soccersoccer
• Controller Hardware: Enable on-board image processing– Interface to digital camera– Incorporate graphics LCD– Incorporate user buttons– Wireless communication between robots
• Sensors: Allow variety of additional sensors:– Shaft encoders– Infra-red distance measurement sensors– Compass module
• Software: Flexibility to accommodate for different robot equipment– Operating system RoBIOS– Hardware description table HDT
What is AI?What is AI?Research in Artificial Intelligence (AI)
includes: design of intelligent machines formalization of the notions of intelligence
and rational behavior understanding mechanisms of intelligence interaction of humans and intelligent
machines.
Objectives of AIObjectives of AI
Engineering : costruct intelligent machines
Scientific : understand what is intelligence.
Can a robot do these?Can a robot do these?Understand?Simulate its environment?Act rationally?Collaborate and compete?Display emotions?
A team of Robots will beat the FIFA World Cup A team of Robots will beat the FIFA World Cup champions by 2050! champions by 2050!
A bold claim:A bold claim:
RoboCup - AimRoboCup - Aim
”pushing the state-of-the-art”
”By mid-21st century, a team of fully autonomous humanoid robot soccer players shall win the soccer game, comply with the official rule of the FIFA, against the winner of the most recent World Cup.
TO BOLDLY GO WHERE MAN HAS GONE BEFORE (cf. Star Trek)
Formalised Testbed
Do you really believe that a team of Do you really believe that a team of Robots could beat the FIFA World Cup Robots could beat the FIFA World Cup
champions by 2050? champions by 2050?
By all accounts this may sound overly ambitious. In fact, if you compare this goal to other ground breaking achievements it is not ambitious at all. The Wright brothers' first airplane was launched and 50 years later man landed on the moon. Even more recently Deep Blue the computer programmed to play chess, played chess grand master Garry Kasparov and won -- roughly 50 years after the deployment of the first computer. It's a long time. Think what has happened since 1950.
Power of AIPower of AI
In 1997 a computer, Deep Blue, won a chess match with world champion Kasparov. Accident? IBM paid Kasparov to loose? Brute force with no intelligence?
So, what is intelligence?
Is the following AI?Is the following AI?
SimulationSimulation Turing test (1950)
Chess versus soccer robotChess versus soccer robot
Difference of domain characteristics between computer chess and soccer robots
DistributedCentralControl
Non-symbolicSymbolicSensor Readings
IncompleteCompleteInfo. accessibility
Real timeTurn takingState ChangeDynamicStaticEnvironment
RoboCupChess
Intelligent Agents Intelligent Agents Agents are situated Perception of environment Execution of actions
Agents can communicate and collaborate they can differ than can compete and be more or less
egoistic/altruistic
The agents have: objectives, communications, intentions.
Professor Professor Kim from Kim from
KAISTKAISTThe founder of Robot Soccer and FIRA president
A New ApproachA New Approach
Two organizations:
1. FIRA (earlier)
2. RoboCup (larger)
Four Blocks in two PCBs (Printed Four Blocks in two PCBs (Printed Circuit Boards)Circuit Boards)
Micro-controller (upper PCB) Communication module (upper
PCB) Motor and driving circuits (lower
PCB) Power (lower PCB)
top view front view side view
Importance of Robot Importance of Robot SoccerSoccer
Communication CooperationCoordination LearningCompetenceReal Time
Robot Soccer EvolutionComputer simulationsWheeled brainless robotsWheeled autonomous robotsLegged autonomous robots
Robot Soccer PurposeRobot Soccer Purpose
“The number one goal of [robot soccer] is not winning or losing, but accumulating diverse technology.” - Mr. Dao (Senior VP of Sony
Corporation).
MiroSotMiroSot
3 robots on 1 teamSize : 7.5cm * 7.5cm * 7.5cmBall : orange golf ballPlayground : black wooden rectangular playground
(150cm * 130cm * 5cm)
Vision : global vision system
(more than 2m above playground)
FIRAFIRAcategorycategory
Experimental SetupExperimental Setupof the Vision Systemof the Vision System
Control panel
NaroSotNaroSot5 robots on 1 teamSize : 4cm * 4cm * 5.5cmBall : orange table-tennis ballPlayground , Vision : same as Mirosot
FIRAFIRAcategorycategory
KheperaSotKheperaSot3 robots on 1 teamBall : yellow tennis ballPlayground : green playground (105cm * 68cm * 20cm)Robot : Khepera RobotVision : K213 Vision Turret
FIRAFIRAcategorycategory
RoboSotRoboSot3 robots on 1 teamSize : 15cm * 15cm * 30cmBall : red roller-hockey ballPlayground : black wooden rectangular playground
(220cm * 150cm * 30cm)Vision : on the robot
Under preparation
FIRAFIRAcategorycategory
Small-Small-Size Size
LeagueLeague
Small-Size League (F-180)Small-Size League (F-180)
Field: 2.7 m x 1.5 m
teams of autonomous small size robot play soccer game on a field equivalent to a ping-pong table. Each team consists of 5 robots.
SizeArea : 18cm rule (fit inside in 18cm diameter cylinder)Height : 15cm (global vision), 22.5cm (otherwise)
Small size leagueSmall size league
The field is the size and color of a Ping Pong table
orange golf ballorange golf ball
Robots move at speeds as high as 2 meters/second Global
vision is allowed
Robot Soccer InitiativeRobot Soccer Initiative
Basic Architecture for Robot Soccer Systems
Robots on the playing field
Host computer
Host computer
Vision system
Communication System Communication
System
“Brainless” System
Vision SystemVision SystemVision : global vision system(more than 3m above ground)
Each team has its own camera and PC
Small-Size LeagueSmall-Size League20 minutes, 2 breaks
Real Robot Small-Size League CompetitionReal Robot Small-Size League Competition
Middle-Middle-Size Size
LeagueLeague
Middle-size Real Robot Middle-size Real Robot League (F-2000): League (F-2000): Local Local
VISIONVISION The field is the size and color of a 3 x 3 arrangement of Ping
Pong tables (9-3 5-meter field)
Each team consists of 5 robots playing with a Futsal-4 ball (4 players, one goal-keeper)
Larger (50 centimeters in diameter) robots
Global vision is not allowed. Each robot has its own vision system
Goals are colored
Field is surrounded by walls to allow for distributed localization through robot sensing
Rule structure based on the official FIFA rules
Medium size leagueMedium size league
Teams of autonomous mid size robots
Real Robot Middle-Size League Real Robot Middle-Size League CompetitionCompetition
Ball : red small soccer ball (FIFA standard size 4 or 5)Playground : green playground (10m * 7m * 0.5m)
Medium Size Medium Size LeagueLeague
Medium Size LeagueMedium Size League
Robots can be heterogenousRobots can be heterogenous
Middle-Middle-Size Size
LeagueLeague
Sony Sony Legged Legged Robot Robot
LeagueLeague
Sony Legged Robot LeagueSony Legged Robot League
3 robots on 1 team (including the goalkeeper).Robot : AIBO ERS-110 (provided by Sony)
No communication, autonomous robots, No communication, autonomous robots, software only. Legged Robot Leaguesoftware only. Legged Robot League. 2.8 m x 1.8 . 2.8 m x 1.8
mm2 players and 1 goal-keeper in a team2 players and 1 goal-keeper in a team
Sony Legged Robot LeagueSony Legged Robot LeagueIs played on a field, approx 3x2 meter Sony develops the robots, and provides a interface for the programming of the robots.
•No Hardware modification is allowed
Playing time is 10 minutes per half, with a 10 minute break at halftime
Do different Robots have different personalities?
Some teams have robots with very different capabilities.
But it is hard to think of them as having personalities; rather the robots have different playing
styles.
Early Sony prototypeEarly Sony prototype
Robot movements closely mirror those of animals
•The winner is the team that scores the most goals.• In the event of a tie, a sudden death penalty kick competition will determine the winner
The Legged Robot LeagueThe Legged Robot League
The Legged Robot LeagueThe Legged Robot League
If opposing teams' robots are damaged or play is excessively rough (whether intentional or not), penalties may be assessed to the offending robot
HumanoiHumanoid Leagued League
Starting 2002, the humanoid Starting 2002, the humanoid leagueleague
Humanoid LeagueHumanoid League
Bi-Ped League (Humanoid) Australia Japan
Where is the science Where is the science in these robot in these robot competitions?competitions?
Global vision Local vision Other sensors Cooperation Sensor fusion Strategy Learning
Sensors and Actuators Sensors and Actuators for Robot Soccerfor Robot Soccer
Local and Global Local and Global VISIONVISION
Sensors for Robot Sensors for Robot SoccerSoccer
• Shaft Encoders– PI controller to maintain wheel speed– PI controller to maintain path curvature– Dead reckoning for vehicle position + orientation
• Infrared Distance Measurement– Avoid Collision– Navigate and map unknown environment– Update internal position in known environment
• Compass– Update orientation independent of shaft encoders– Fault-tolerance in case robot gets pushed or wheels slip
Sensors for Robot Sensors for Robot SoccerSoccer
• Digital Camera– Low resolution, 60x80 pixels, 24bit color
(Braunl)– Color or shape recognition
• Communication– Sharing information among robots– Receiving commands from human operator
VISION: VISION: Color DetectionColor Detection• In robot soccer, objects are color coded:
ball, goals, opponents, team mates, walls, etc.
Teach ball and goal color (hue) before starting the game
Match colors in HSI space→ Better in changing lighting conditions
Robots The robots have functions such as velocity control, position control, obstacle
avoidance, etc.
Host computer The host computer processes vision dataprocesses vision data and calculates next behaviors of robots
according to strategies and sends commands to the robots using RF modem.
Role of Vision
Brain-on-board Brain-on-board system system
2.2 Robot-based system2.2 Robot-based system
Suitable when the large number of agents existComplex and expensiveNeed communication among robots
Distributed system
Intelligent part is implemented in the robots.
Robots The robots decide their own behavior autonomously using the
received vision data, own sensor data and strategies.
Host computerThe host computer processes only vision datacan be considered as a kind of sensor.
Role of Role of visionvision
Robot-based Robot-based systemsystem
System System ComparisonsComparisons
Brain-on-board system
Merits Demerits Research purpose
Robot -based
system
Low cost Easy to
develop
Cannot use local sensors High computing power
& fast sampling time
Suitable for many agents Can use local information
Complex and expensive robots.
Hard to build the system
Suitable to modularize
Risk of inconsistent property between host computer and robot system
Vision system Multi-agent theory
Robot system Multi-agent system
development
Robot-based and vision-based systems
Remote-brainless system
VXD: role of VXD: role of colorcolorInitialization
Click ‘Load VXD’ in the Initialize group box Click ‘Start Grab’
Configuration ‘Load Conf.’: load a configuration file ‘Save Conf.’: save current configuration ‘Set Robot Size’: set the robot size in number of pixels ‘Set Pixel Size’: set the size of each color (ball, team, robot, opponent) patch in number of
pixels ‘Set Boundary’: set the field boundary on the screen ‘Change Color’: change the color setting of each color
patch ‘Set Color’: set the range of tolerance of each color
Subsystems and Subsystems and VisionVision
Serial Port Select the serial communication port
Home Goal Select the home side on the screen
Find Objects Check the box of which you like to find on the field
Initial Position: tell the vision system the initial position
of each object E.g.) for the ball
i) turn on the radio button of ‘Ball’ ii) place the mouse on the ball and press the left button
Repeat above procedure for another object
Commands for Commands for VisionVision
Select Situation The situation in which the game is about to start
Command Click ‘Ready’: the vision system starts finding the
objects on the field
Click ‘Start’ : the vision system starts sending commands
to the robots Click ‘Stop’ : the vision system stops finding objects
and sending commands
Tasks for usTasks for usHow to organize the ceiling camera system?How to describe (learn?) the shapes of robots?How to find the (x,y,) for each robot?How to modify the scripted behavior when the triplets for each robot are known?How to design the interactive behaviors?
Task for Robot Theatre Task for Robot Theatre TeamTeam
Write a half-page essay about the vision system that we discussed today.Use you knowledge from other lecture of today.Add your imagination and crazy ideas about what the robots should see and know for our particular scene of Marie Curie and Schroedinger’s Cat.
For next weekFor next week