Post on 19-Dec-2015
September 7, 2005 1
Control Agents making Robotics a reality
Dr. Reuven GranotFaculty of Science and Scientific Education
University of Haifa, Israel
Visiting scholar at UBC/Mining Eng. Dept.
rgranot@inter.net.il
September 7, 2005 2
The need for unmanned systems.
Why tele-robotics?
A new layer of software should be developed in order to support application users in fields of robotics and other semi or autonomous systems to develop more effectively their specific applications.
follow upcoming standards like JAUS and CORBA.
follow the Distributed and Object Oriented paradigm, but be more than an object or an expert system by being reactive, autonomous and proactive.
transparently take care of inter agent communication and other basic tasks a control agent needs.
- a control agent is performing some control task while communicating with other agents or humans as needed.
We, at University of Haifa, Israel, are suggesting to collaboratively developing the needed infrastructure for Human Supervised Autonomous Control Agents.
A relevant application of semi-autonomous bulldozer.
Outline
September 7, 2005 3
Illustration from NASA and ORNL publication
Robot colony on a site preparation task for a PV tent on Mars.
The Need of Unmanned Systems
• DDD
– Dull
– Dirty
– Dangerous
• Distant – at different scale
– Macro: space,
– Micro: telesurgery, micro and nano devices
Is well recognized to perform tasks that are:
September 7, 2005 4
Remote Controlled vehicles in combat environment
RC is still preferred by designers
o Simple, but not practical for combat or other very demanding environment because the human operator:
is very much dependent upon the controlled process needs long readjustment time to switch between the controlled and the local environment.
The state of the art of the current technology has not yet solved the problem of controlling complex tasks autonomously in unexpected contingent environments.
o dealing with unexpected contingent events remains to be a major problem of robotics.
Consequence: A human operator should be able to interfere: remains at least in the supervisory loop.
September 7, 2005 5
All these applications require an effective interface between the machine and a human in charge of operating/ commanding the machine.
We are suggesting to perform Human Supervised Autonomous Control, which is known as tele-robotics.
That be done using the software agent technology
The needed control metaphor:
Human Supervised Autonomous
September 7, 2005 6
The Tele-robotics paradigm
Telerobotics is a form of Supervised Autonomous Control.
A machine can be distantly operated by:
• continuous control: the HO is responsible to continuously supply the robot all the needed control commands.
• a coherent cooperation between man and machine, which is known to be a hard task.
Supervision and intervention by a human would provide the advantages of on-line fault correction and debugging, and would relax the amount of structure needed in the environment, since a human supervisor could anticipate and account for many unexpected situations.
September 7, 2005 7
The spectrum of control modes.
Solid line= major loops are closed through computer, minor loops through human.
• traded control: control is or at operator or at the autonomous sub-system.
• shared control: the instructions given by HO and by the robot are combined.
• strict supervisory control: the HO instructs the robot, then observes its autonomous actions.
A telerobot can use:
September 7, 2005 8
Human Robot Interaction
• In supervised autonomously controlled equipment, a human operator generates tasks, and a computer
autonomously closes some of the controlled loops. • Control bandwidth
– Robot SW: high
– Human response: slow
• Human Operator is expected to Control several machines/ equipment/ systems be capable to deal with other duties (like a combat environment requests) in somehow relaxed mode of operation.
Make the machine an agent in human operator’s service.
September 7, 2005 9
Software Infrastructure
• Architecture– Distributed– Objects or some more sophisticated entities?– Communication
• Development Environment– Language– Tools– Reusable units
September 7, 2005 10
What is a Robot Architecture?
• There are many different ways in which a robot control program can be put together.
In order to program a robot in a structured and principled fashion, we use an appropriate robot control architecture.
September 7, 2005 11
Robot Architecture
• A control architecture provides a set of principles for organizing a control system. – It provides structure and constraints which aid the
designer in producing a well-behaved controller.
• To be successful a system designer has to decide how (in what order? with what priority?) does he put together multiple feedback controllers in a principled fashion and how to scale up control to more complex robots, which generally have to deal with many behaviors at once.
– How would you put multiple feedback controllers together?
– How would you decide which one to use when and for how long and in what priority relative to the others?
September 7, 2005 12
Misconceptions.1. Programming languages are implementation tools and not
architectures.
2. The issue of fundamental power or expressiveness of a robot
control architecture: claims have been made about one control architecture being able to compute fundamentally more than another. This cannot be true if we understand that all are grounded in Turing-complete programming languages.
However, the above is not to say that all architectures are the same. On the contrary, architectures impose strong constraints on how robot programs are structured, and the resulting control software ends up looking very different.
September 7, 2005 13
Robot Architecture Major Classes/Categories
Intuitively, this means that there are infinitely many ways to structure a robot program, but they all fall into one of major classes/categories of control: • Deliberative Control : Think hard, act later.
─ SPA, serial, complete each step first – then proceed
• Reactive Control : Don’t think, (re)act.
─ Direct connection between perception to action, no memory, no planning.• Hybrid Control : Think and act independently, in parallel.
─ Deliberative and Reactive modules run independently at different time scales
• Behavior-Based Control : Think the way you act. – Distributed by behavioral task decomposition– Each behavior has its restricted planning and execution capabilities
September 7, 2005 14
The Choice of the Control Architecture
• When it comes to more complex robots, i.e., robots that have to deal with complex environments and complex tasks, the control architecture becomes very important.
• The different properties of an environment that will impact the robot's controller (and therefore the choice of control architecture): – noisy,
– speed/response time of sensors and effectors
– total/partial hidden state/ observable
– discrete v. continuous state ; static v. dynamic ...
• Similarly, the properties of the robot's task impact the choice of the control architecture. The task requirements can constrain the architecture choice.
September 7, 2005 15
Parallel Processing Paradigm.
• As robot control is engaged to deal with more complex problems, centralized supervisory architectures encounter barriers to real time performance caused by computational complexity coupled with insufficient computing power and sensor resources. – Despite startling advances in hardware and software
technology and similarly surprising cost reductions, these fundamental barriers remain unchanged.
• The parallel-processing paradigm may be the only technology to challenge this fact.
September 7, 2005 16
Asynchronous and Synchronous processes
• The other leading architectural trend is typified by a mixture of asynchronous and synchronous control and data flow. – Asynchronous processes are characterized as loosely
coupled and event-driven without strict execution deadlines.
– Synchronous processes, in contrast, are tightly coupled, utilize a common clock and demand hard real-time execution.
September 7, 2005 19
Time Scale.
• Reactive systems respond to the real-time requirements of the environment,
• while deliberative system look ahead (plan) and thus work on a longer time-scale.
• Hybrid systems must combine the two time-scales in an effective way, usually requiring a middle layer; consequently they are often called three-layer architectures.
• Finally, behavior-based systems attempt to bring the different time-scales closer together by distributing slower computation over concurrent behavior modules.
Time-scale is an important way of distinguishing control architectures.
September 7, 2005 20
Representation
• Another key distinguishing feature between architectures is representation of the world/environment, also called world modeling.
• Some tasks and architectures involve storing information about the environment internally, in the form of an internal representation of the environment.
• For example, while exploring a maze, a robot may want to remember a sequence of moves it has made (e.g., "left, left, right, straight, right, left"), so it can back-track and find its way.
Thus, the robot is constructing a representation of its path through the maze.
The robot can also build a map of the maze, by drawing it using exact lengths of corridors and distances between walls, etc. .
This is also a representation of its environment, a model of the world.
• If two robots are working together, and one is much slower than the other, if the fast robot remembers/learns that the other is always slower, that is also a type of a model of the world, in this case, a model of the other robot.
September 7, 2005 21
Different World Models.
• There are numerous aspects of the world that a robot can represent/model, and numerous ways in which it can do it, including:
* spatial metric or topological: maps, navigable spaces, structures
* objects instances of detectable things in the world
* actions outcomes of specific actions on the self and environment
* self/ego stored proprioception: sensing internal state, self- limitations, etc.
* intentional goals, intended actions, plans
* symbolic abstract encoding of state/information
September 7, 2005 24
• Regarding the architecture of robotic systems, we discussed so far two key issues distinguishing architectures, as had to do with – time-scale (reactive) and
– looking ahead (deliberative).
• A third key issue we need to consider is modularity, i.e., the way in which the architecture decomposes into components.
September 7, 2005 25
What is a behavior?
• An individual behavior is a stimulus/ response pair for a given environmental setting that is modulated by attention and determined by intention.
• Behaviors serve as the basic building blocks for robotic actions.
Attention: prioritizes tasks and focuses sensory resources and is determined by the current environmental context.
Intention: determines which set of behaviors should be active based on the robotic agent’s internal goals and objectives.
Apparent or emergent behavior: the global behavior of the robot as a consequence of the interaction of the active individual behaviors.
September 7, 2005 26
What are Behaviors?
• typically has the following properties:
– are feedback controllers (closed-loop, extended in time )
– achieve specific tasks/goals
– are typically executed in parallel/concurrently
– can store state and be used to construct world models/representation
– can directly connect sensors and effectors
– are typically higher-level than actions
– can also take inputs from other behaviors and send outputs to other behaviors
• when assembled into distributed representations, behaviors can be used to look
ahead but at a time-scale comparable with the rest of the behavior-based system.
September 7, 2005 27
Behaviors and Modularity
Behavior-based systems are not limited in the ways that reactive systems are. As a result, behavior-based systems have the following key properties:
1) the ability to react in real-time.
2) the ability to use representations to generate efficient (not only reactive) behavior.
3) the ability to use a uniform structure and representation throughout the system (so no intermediate layer).
September 7, 2005 28
Assembling Behaviors.
• Systems are constructed from multiple behaviors.
• Emergent behavior implies a holistic (attention to the “whole”) capability where the sum is considerably greater than its parts.
• Emergence is “the appearance of novel properties in whole systems”.
• Intelligence emerges from the interaction of the components of the system.
• Coordination functions are algorithms used to assemble behaviors.– Conflict can result when two or more behaviors are active, each
with its own independent response.
September 7, 2005 29
The Agent
• An agent is a computer system capable of autonomous action in some environments.
• A general way in which the term agent is used is to denote a hardware or software-based computer system that enjoys the following properties: – autonomy: agents operate without the direct intervention of
humans or others, and have some kind of control over their actions and internal state;
– social ability: agents interact with other agents (and possibly humans) via some kind of agent-communication language;
– reactivity: agents perceive their environment, (which may be the physical world, a user via a graphical user interface, or a collection of other agents), and respond in a timely fashion to changes that occur in it;
– pro-activeness: agents do not simply act in response to their environment; they are able to exhibit goal-directed behavior by taking the initiative.
September 7, 2005 30
Agents and Behaviors
• Behavior is defined as the way how we/people observe the system/robot acts/behaves.
• The robot system is NOT aware of what we know about it.– What makes the system act as we observe is its
software.
• Behaviors are implemented by agents.
September 7, 2005 31
Interface Agent
• A software entity, which is capable to represent the human in the computer SW environment.
• It acts on behalf of the human• Follows rules and has a well defined expected
attitude/ action.• May be instructed on the fly and may receive
during mission updated commands from the human operator.
We need to build agents in order to carry out the tasks, without the need to tell the agents how to perform these tasks.
September 7, 2005 32
Agents are not Objects
• Differ from Objects– autonomous, reactive and pro-active
– encapsulate some state,
– are more than expert systems
– are situated in their environment and take action instead of just advising to do so.
• Agents may act inside the robot software to implement behaviors:
Feedback controllers Control subassemblies Perform Local Goals/ tasks
September 7, 2005 33
Agent control loop
• agent starts in some initial internal state i0 .
• observes its environment state e, and generates a percept see(e).
• internal state of the agent is then updated via next function, becoming next_(i0, see(e)).
• the action selected by agent is action (next(i0, see(e))))
This action is then performed.
• Goto (2).
September 7, 2005 36
Human Operator• Monitors the activities and the performance of the assembly of
agents.• Responsible for the completion of the major task (global goal)
– may interfere by sending change orders. • emergent (executed immediately, without considering any
possibility to return to achieve the goal/in the shortest possible way)
• “as is ordered” or• normal
– checked by the interface agent – which negotiates execution with other agents in order to
optimize execution performance
– Conflict resolution algorithm• defined as default, or• defined by the human operator in its change order or • suggested to the operator by a simplified decision support
algorithm.
September 7, 2005 38
D9 Bulldozer
• The operator has very limited information about his surroundings or machine performance.
• A good starting project: – earthmoving tasks are loosely coupled with
locomotion tasks.– earthmoving tasks are not really simple and – locomotion tasks are not really complicated.
September 7, 2005 39
RCSEmbeds a hierarchy of agents within a hierarchy of organizational units: Intelligent Nodes or RCS_Nodes.
SquadCommander
SquadCommander
Squad Commander
PlatoonCommander
VehicleCommander
VehicleCommander
VehicleCommander
VehicleCommander
SquadCommander
JAUS
From M. W. Torrie
A hierarchy of Commanders
different resolution in space and time
September 7, 2005 41
RCS_Node
Value Judgment
Sensory Processing
World Modeling
Behavior Generation
Knowledge Database
Update Plan
StatePredicted Input
Observed Input
Perceived Objects &
Events
Commanded Actions
(Subgoals)
Commanded Task (Goal)Plan
EvaluationPla
n R
esul
tsS
ituation E
valuation
September 7, 2005 43
Agents in Behavior Generation hierarchy
• Tasks are decomposed and assigned in a command chain.
• Actions are coordinated
• Resources are allocated as plan approved.
• Tasks achievements are monitored (VJ)
• Execution in parallel
September 7, 2005 45
Small Size Scaled Model
• The implementation differs by mechanical, perceptual and control elements from the full scale application.
• It still may help to identify unusual situations which the software agent must be capable to deal with.
• Full scale machines may be tested only at field ranges, which are time consuming and very expensive.
• A small scale model may be tested in office environment, enabling the software developers to shorten test cycles by orders of magnitude.
September 7, 2005 46
Expected situations
• The bulldozer moves forward placing the blade too low – The human decides: the blade should be placed higher
Command issued: “lift the blade”.
• experiencing too much power to enable earth moving forward – the human operator would prefer to withdraw and
attack the soil from a new position behind– the human operator is distant– the bulldozer is “close” to the ditch;
> a better practice would be to first complete the maneuver.
Bulldozer using Fuzzy Control decides to perform the better practice and withdraws only after the maneuver is completed.
September 7, 2005 47
The Model
September 7, 2005 48
autonomous-bulldozer\robot.WMV
autonomous-bulldozer\robot.mpg
4 min
3 min
September 7, 2005 49
September 7, 2005 50