Human-Robot Interaction

Elective in Artificial Intelligence

Lecture 2 – Interaction Management

Luca Iocchi

DIAG, Sapienza University of Rome, Italy

M. Cirillo, L. Karlsson, A. Saffiotti. Human-aware task planning: An application to mobile robots. In ACM Transactions on Intelligent Systems and Technology (TIST), 1 (2), 2010.

de Silva, L. and Lallement, R. and Alami, R. The HATP Hierarchical Planner: Formalisation and an Initial Study of its Usability and Practicality. In Proc. of the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), 2015.

Valerio Sanelli, Michael Cashmore, Daniele Magazzeni, and Luca Iocchi.Short-Term Human Robot Interaction through Conditional Planning and Execution.In Proc. of International Conference on Automated Planning and Scheduling (ICAPS) 2017 (to appear).

Eugenio Sebastiani, Raphaël Lallement, Rachid Alami, and Luca Iocchi. Dealing with On-line Human-Robot Negotiations in Hierarchical Agent-based Task Planner.In Proc. of International Conference on Automated Planning and Scheduling (ICAPS) 2017 (to appear).

L. Iocchi - Human-Robot Interaction 2

Readings

Interaction design/management

• Efficiency and effectiveness of “interaction work”

Human-Robot Interaction design/management

• Efficiency and effectiveness of HRI

L. Iocchi - Human-Robot Interaction 3

HR Interaction Design

Interaction = execution of actions by humans and robots according to the current situation

Interaction Design = Action planningInteraction Management = Plan execution

Formalisms & Tools

• Low-level (encoding interaction in the program/behavior)

• High-level (explicit representation of the interaction)

L. Iocchi - Human-Robot Interaction 4

HRI Design as Planning

High-level frameworks for HRI Design and Management

• HA-PTLplan – Human-aware Task Planner [Cirillo et a., 2010]

• HATP – Hierarchical Agent-based Task Planner [Lallement et al. 2014, de Silva et al. 2015]

• HATP + Sensing [Sebastiani et al. ICAPS 2017 (to appear)]

• ROSPlan conditional planning[Sanelli et al. ICAPS 2017 (to appear)

• MDP + Sensing [Iocchi et al. ICAPS 2016]

L. Iocchi - Human-Robot Interaction 5

HR Interaction Design

[Cirillo et al., 2010]• Robots take into account human actions both at planning time and at

execution time. Does not plan for human actions.• Respecting interaction constraints.• Multiple hypotheses about the world (Partial Observability/POMDP)

L. Iocchi - Human-Robot Interaction 6

Human-aware Task Planning

Human actions predicted (env. sensors / HMM / plan recognition)

Monitor / Failure detection / Replanning

L. Iocchi - Human-Robot Interaction 7

Human-aware Task Planning

[Lallement et al. 2014, de Silvae et al. 2015]• Based on Hierarchical Task Networks (HTN)

• Distinguish different agents

• Generates different streams of actions (linear plans) for each agent

• Generate plans also for humans.

L. Iocchi - Human-Robot Interaction 8

Hierarchical Agent-based Task Planner

Limitations

• Roles of humans and robots are decided at planning time

• Humans must be aware of the plan and execute it according to the given constraints

• No flexibility in on-line modification and adaptation

L. Iocchi - Human-Robot Interaction 9

Hierarchical Agent-based Task Planner

kcl-planning.github.io/ROSPlan

Integrate planners in ROS

Modular and extendible

L. Iocchi - Human-Robot Interaction 10

ROSPlan

Plan representation• Sequence of actions • Temporal model• Petri Nets

Execution mechanism• Continuous state estimation • Re-planning

L. Iocchi - Human-Robot Interaction 11

ROSPlan

Classical/HTN Planning (e.g., STRIPS, HATP)• Complete knowledge about the state

• Output is a linear plan (sequence of actions)

• No representation of uncertainty

MDP/POMDP Planning• No explicit representation of sensing actions

• Assumes perfect or very reliable perception

L. Iocchi - Human-Robot Interaction 12

Discussion

• Execution mechanism• State estimation error-prone (perception involving humans

more difficult)

• Monitor + replanning after failures not adequate for HRI tasks

• Non-expert users highly unpredictable

L. Iocchi - Human-Robot Interaction 13

Discussion

Service robot has to make sure user needs are satisfied. User needs are not known in advance.

L. Iocchi - Human-Robot Interaction 14

Example

Classical planning

• Guess a user need

• Plan with this guess

• Execute the plan

• If guess is wrong, adjust conditions and replan

When guess is wrong, behavior is not socially acceptable.

• The robot does not move, but the user needs something.

• The robot prepares food or drink that is not requested by the user.

L. Iocchi - Human-Robot Interaction 15

Example

Plan with explicit sensing action

• Go to person

• Ask if s/he needs something // Sensing action

• if (need_food AND need_drink)• Go to the kitchen

• Prepare food and drink

• Serve food and drink to person

• if (need_food)• …

• else• Do nothing

L. Iocchi - Human-Robot Interaction 16

Example