Post on 29-Jan-2016
Multi-Agent Exploration in Unknown Environments
Changchang WuNov 2, 2006
Outline
• Why multiple robots• Design issues• Basic approaches
– Distributed– Centralized– Market-based
Why Multiple Robots
• Some tasks require a robot team
• Have potential to finish tasks faster
• Increase robustness w/ redundancy
• Compensate sensor uncertainty by merging overlapping information
• Multiple robots allow for more varied and creative solutions
A Good Multi-Robot System Is:
• Robust: no single point of failure
• Optimized, even under dynamic conditions
• Quick to respond to changes
• Able to deal with imperfect communication
• Able to avoid robot interference
• Able to allocate limited resources
• Heterogeneous and able to make use of different robot skills
Basic Approaches
• Distributed – Every robot goes for itself
• Centralized– Globally coordinate all robots
• Market-based– Analogy To Real Economy
Distributed Methods
• Planning responsibility spread over team
• Each robot basically act independently• Robots use locally observable
information to coordinate and make their plans
Example: Frontier-Based Exploration Using Multiple Robots (Yamauchi 1998)
• A highly distributed approach
• Simple idea: To gain the most new information about the world, move to the boundary between open space and uncertainty territory
• Frontiers are the boundaries between open space and unexplored space
Occupancy Grid
• World is represented as grid• Each cell in the grid is assigned with a
probability of being already occupied/observed• The initial probability is all set to .5• Cell status can be Open (<0.5), Unknown
(=0.5) or Occupied (>0.5)• Bayesian rule is used to update cells by
merging information from each sensor reading (sonar)
Frontier Detection• Frontier = Boundary between open and unexplored
space.
• Any open cell adjacent to unknown cell is frontier edge cell.
• Frontier cells grouped into frontier regions based on adjacency.
• Accessible frontier = Robot can pass through opening.
• Inaccessible frontier = Robot cannot pass through opening.
Multi-Robot Navigation
• Simple algorithm: Each robot goes along the shortest obstacle free path to a frontier region
• Robots share a common map: All information obtained by any robot is available to all robots
• Robots are planning path independently • Use reactive strategy to avoid collisions• Robots may waste time for the same frontiers
An Exploration Sequence
Distributed Methods: Pros & Cons
• Pros– Very robust. No single point failure– Fast response to dynamic conditions– Little or no communication is required– Easy….Little computation required
• Cons– Plans only based on local information– Solutions are often sub-optimal
Centralized Methods
• Robot team treated as a single “system” with many degrees of freedom
• A single robot or computer is the “leader”
• Leader plans optimal tasks for groups
• Group members send information to leader and carry out actions
Example: Arena (Jia 2004)
• Robots share a common map and only communicate with a leader
• Robots compete for resources by their efficiency
• leader greedily assigns the most efficient tasks
• Leader coordinate robots to handle interference
Background
• World representation– Occupancy grid
• Cost unit– Moving forward one step = Turning 45
degrees
• Cost overflow– Similar to minimum cost spanning tree– Easy to compute the shortest path– Easy to handle obstacle
Cost Overflow
Cost of 45° turning = Cost of one cell’s step
1 2
2 3
3
3
3
4
4
4
44
4
5
5 5 5 5
5
5
5
Direction
priority
6
6
6 6 6 6 6
7
7
7
77
8
8
8
8
88
8
9 9
9
9
9
9
9
10
10
10
10
10
10
11
11
11
11
11
11
11
12
12
12
12
12
12
12
12
Goal Candidates Detection
• A goal point P should satisfyi. P is passable (Mark the cells in warning range or
obstacles/Wall/Unknown cells as impassable)ii. Some unexplored cells lie in the circle with P as the
center and (R + K) as the radium, where R is the warning radius and K is usually 1
Robot cellpaths cellobservation cell
candidate goal
Goal Resource
• Reserved goal candidates– Robots obtained by competition
• Recessive goal candidates– The goal points in a given range
to a reserved goal point– This distance can be adjusted
Goal candidates
Recessive goals candidates
Path Resource
• Path resource is a time-space term
• For a given time, the cells close to any robot are marked off for safety
• Looks just like a widened path
• Basically a reactive strategy
goal
path
resource
Revenue and Utility
• Revenue – The expected gain of information that
robots observe at a goal point
• Utility used by many other approaches– Utility = revenue – cost
• Utility in this paper– Utility = Revenue / Cost– Better connected to purpose of smallest
cost– No need to care about unit conversion
Greedy Goal Selection
• Try to maximize the global utility
• Coordination: robots obtain goal and path resources exclusively
• Competition: repetitively select the pair of free agent and goal with highest utility
• Sub-optimal
Simple Algorithm
• Repeat until map is complete– Repeat #free robots times
1. Cost computation (Also make sure no interference with the busy robots)
2. Select the highest utility task (Compete)3. Mark off the associated robot and goal
points, and nearby goal points
1st Competition:
2
2 2
2
2
2
2
2
2
2 2 2 2
2
2
2
2
2
2
3 3 3
3
3
3
3
3
3
3 3 3 3 3 3
3
3
3
3 3 34 4 4
4
4
4
4 4 4 4 4 4 4 4
1
1
1 1
1
1
1
1
1
11
55
5 55 5 5 5 55
6 6 6 6 6
6
6
6
6
6
Interval = 3 Competitor:
1st Competition Result:
4 4
6 6
5 5
3
2
2
1
4
4
1
2 2
4 4
3 3
2 2
4
Interval = 3 Competitor:
2nd Competition
1
2
1
2 2
3 3 3
4 4
4
4 4 4
44
1
11
1
1 1
Interval = 3 Competitor:
Satisfied:
2 2 2
2
2
2
2
2 2 2
3 3 3 3
3
3
3
3
3
3
3 3 3 3444 4 4 4
4
4
4
4
4
4
4 4 4 4 4
5 5 5 5 5
5 5 5
5
5
5 5
6
6
6
6 6 66 6
6
6
6
6
2nd Competition Result
1
2
1
2 2
3 3 3
4 4
4
4 4 4
44
4
1
2
3
4
5
1
1
2
2
22
2
333
4 44
5 5 5
6 6 6
6
666
6
Interval = 3 Competitor:
Satisfied:
3rd Competition
1
2
1
2 2
3 3 3
4 4
4
4 4 4
44
4
1
2
3
4
5
1
2
2
22
2
333
4 44
5 5 5
6 6 6
6
666
6 6
Competitor:
Satisfied:
Interval = 3
1
1 1
2
2
222
3 3 3 3
3
3
3
4 4 4 4 4
4
4
5 5 5 5 5
6 6
6
7 7
7
7
7
7
7
7
7
8
8
8
8
8
8
8
9
9
9
9
9
9
9
10
10
10
10
10
10
10
10
11 11
11
11
11
11
11
11
11
11 12
12
12
12
12
12
12
12
13
13
13
13
13
13
13
3rd Competition Result
1
3 2
4
4
1
2
3
4
5
1
2
3
4
6 6 6
Competitor:
Satisfied:
Interval = 3
1
2
3
4 7
8
9 10 11 12
5 6
13
4
6
Planning Issues
• Do not transfer a reserved goal point to another free agent (unless necessary). Frequent change of tasks can cause localization error.
• Quit an assigned task when the goal point is unexpectedly observed by other robots
• Schedule at most one task for each agent
Possible Variations• Still keep busy agents in competition.
Remove the goal resources they win from competition.– This prevents those goal resources being
assigned to other agents– It is too early to burden a new task on a
robot who has not achieved it current task• No need to schedule them.
– New resources probably will be found when they reach the goals
Handling Failure of Planning
• It may fail to plan safe paths– When some robot get to a place where
• it is almost too close to other robot • it has no good space to detour
– And it choose to just wait there for other robots to move away, which is not known by other robots
• Avoidance of unexpected obstacle– Robots have simple reactive mechanism– Release resources and try to gain new task
Fail to plan safe paths
Competitor:
Satisfied:
Interval = 3
1
2
3 4 5 6 7 8
9
10
1
2
3410111213141516
collision
Reactive Mechanism
Competitor:
Satisfied:
Interval = 3
Exchange Tasks
Competitor:
Satisfied:
Interval = 3
1234
5
1
2
3
Some Statistics
Demo
Centralized Methods : Pros
• Leader can take all relevant information into account for planning
• Optimal s islution possible! • One can try different approximate
solutions to this problem
Centralized Methods: Cons
• Optimal solution is computationally hard– Intractable for more than a few robots
• Makes unrealistic assumptions:– All relevant info can be transmitted to leader
– This info doesn’t change during plan construction
• Vulnerable to malfunction of leader
• Heavy communication load for the leader
Market-Based Methods
• Based on market architecture• Each robot seeks to maximize individual
“profit”• Robots can negotiate and bid for tasks• Individual profit helps the common good• Decisions are made locally but effects
approach optimality– Preserves advantages of distributed
approach
Why Is This Good?
• Robust to changing conditions– Not hierarchical– If a robot breaks, tasks can be re-bid to others
• Distributed nature allows for quick response• Only local communication necessary• Efficient resource utilization and role adoption• Advantages of distributed system with
optimality approaching centralized system
Architecture• World is represented as a grid
– Squares are unknown (0), occupied (+), or empty (-)• Goals are squares in the grid for a robot to explore
– Goal points to visit are the main commodity exchanged in market
• For any goal square in the grid:– Cost based on distance traveled to reach goal– Revenue based on information gained by reaching goal
• R = (# of unknown cells near goal) x (weighting factor)
• Team profit = sum of individual profits– When individual robots maximize profit, the whole team
gains
Example World
Goal Selection Strategies
• Possible strategies:– Randomly select points, discard if
already visited– Greedy exploration:
•Choose goal point in closest unexplored region
– Space division by quadtree
Exploration Algorithm
Algorithm for each robot:1. Generate goals (based on goal selection
strategy)2. If OpExec (human operator) is reachable,
check with OpExec to make sure goals are new to colony
3. Rank goals greedily based on expected profit
4. Try to auction off /bid goals to each reachable robot
– If a bid is worth more than you would profit from reaching the goal yourself (plus a markup), sell it
Exploration Algorithm
5. Once all auctions are closed, explore highest-profit goal
6. Upon reaching goal, generate new goal points
– Maximum # of goal points is limited
7. Repeat this algorithm until map is complete
Bidding Example
• R1 auctions goal to R2
Expected vs. Real
• Robots make decisions based on expected profit– Expected cost and revenue based on
current map
• Actual profit may be different– Unforeseen obstacles may increase cost
• Once real costs exceed expected costs by some margin, abandon goal– Don’t get stuck trying for unreachable goals
Information Sharing
• If an auctioneer tries to auction a goal point already covered by a bidder:– Bidder tells auctioneer to update map– Removes goal point
• Robots can sell map information to each other– Price negotiated based on information gained– Reduces overlapping exploration
• When needed, OpExec sends a map request to all reachable robots– Robots respond by sending current maps– OpExec combines the maps by adding up cell
values
Advantages of Communication
• Low-bandwidth mechanisms for communicating aggregate information
• Unlike other systems, map info doesn’t need to be communicated repeatedly for coordination
What Is a Robot Doing
• Goal generation and exploration• Sharing Information with other
robots• Report information to OpExec at
some frequency
Experimental Setup
• 4 or 5 robots– Equipped with
fiber optic gyroscopes
– 16 ultrasonic sensors
Experimental Setup
• Three test environments– Large room cluttered with obstacles– Outdoor patio, with open areas as well as walls and tables– Large conference room with tables and 100 people
wandering around• Took between 5 and 10 minutes to map areas
Experimental Results
Experimental Results
Experimental Results
• Successfully mapped regions• Performance metric (exploration efficiency):
– Area covered / distance traveled [m2 / m]– Market architecture improved efficiency over no
communication by a factor of 3.4
Conclusion
• Market-based approach for multi-robot coordination is promising– Robustness and quickness of distributed system– Approaches optimality of centralized system– Low communication requirements
• Probably not perfect– Cost heuristics can be inaccurate– Much of this approach is still speculative
• Some pieces, such as leaders, may be too hard to do
In Sum
• Distributed vs. centralized mapping• Distributed vs. centralized planning• Revenue/Cost vs. Revenue – Cost
• Often sub-optimal solutions• No common evaluation system for
comparisons
References
• Yamauchi, B., "Frontier-Based Exploration Using Multiple Robots," In Proc. of the Second International Conference on Autonomous Agents (Agents98), Minneapolis, MN., 1998.
• Menglei Jia , Guangming Zhou ,Zonghai Chen, "Arena—an Architecture for Multi-Robot Exploration Combining Task Allocation and Path Planning,“ 2004
• Zlot, R., Stentz, A., Dias, M. B., and Thayer, S. “Multi-Robot Exploration Controlled By A Market Economy.” Proceedings of the IEEE International Conference on Robotics and Automation, 2002.
• http://voronoi.sbp.ri.cmu.edu/presentations/motionplanning2001Fall/FrontierExploration.ppt
• http://www.ai.mit.edu/courses/16.412J/lectures/advanced%20lecture_11.6.ppt
• http://mail.ustc.edu.cn/~jml/jml.files/Arena.ppt