Enchaînement de tâches robotiques

Tasks sequencing for sensor-based control

Nicolas Mansard

Supervised by Francois Chaumette

Équipe Lagadic

IRISA / INRIA Rennes

2

ContextSensor-based

controlMotion planning Control

architecture

initial

desired

Execution controller

Path deform.

Traj. planning

?

?

Path deformation

Dynamic planning

Motors Sensors

Motion planning

Execution level

Symbolic high-level controller

Data extraction

Motion prediction

Complete solutionGlobal convergence

Required knowledgeLack of reactivity

AccuracyReactivityRobustness

Local convergence

Realistic solution

Complex softwareInherent problems due to

the path planning

Improve the expressivity of the sensor-based control methods

Less planning – More freedom

4

Task sequencing for sensor-based control

GLOBAL TASK

CONSTRAINTS Centering

Zoom

Perspective

Z-rotation

Stack of Tasks

TASK-LEVEL CONTROLLER

add

removeswap

5

Task sequencing for sensor-based control

Example

Centering

Zoom

Perspective

Z-rotation

Stack of Tasks

CONSTRAINTS

6

Task sequencing for sensor-based control

CONSTRAINTS

Centering

Zoom

Perspective

Z-rotation

Stack of Tasks

TASK-LEVEL CONTROLLER

add

removeswap

7

Outline

Credo

Low Level Stack of tasks Improvements

High Level Task-level controller Applications

8

Sensor

4

12

3

4

1 2

3

What is a task?

Robot position: Robot control input:

An error between current and desired sensor values

A reference behavior of the error

The associated Jacobian matrix

Classical control law

[Samson91],[Espiau91]

Sensor

- +

9

Stack of task

Principles Compute the control law from several (sub-) tasks

Ensure a hierarchy For avoiding any conflicts is perfectly realized is realized under the condition is realized is realized under the condition , … and are realized

Take into account additional constraints As a lowest-priority task Using the potential field formalism

Ensure the continuity at task changes

10

z can be used to vary the trajectory (obstacle avoidance)

Stack of task

Redundancy formalism

P

We will use z to realize at best the second elementary task

[Rosen60],[Liegeois77]

11

Two recursive equations to stack several tasks

Continuity at stack change

e1

ei-1

ei

STACK

en

Stack of task

Priority and continuity

[Chiaverini97]

[Robea-Egocentre04]

12

Gradient Projection Method Potential function : 0 far from the obstacle

Maximal when the robot reaches the obstacle Gradient as a repulsive force Projection onto the remaining DOFs

Application to joint limits avoidance

Stack of task

Considering the constraints[Liegeois77],[Marchand98]

[Khatib87]

13

Outline

Credo

Low Level Stack of tasks Improvements

Directional redundancy Varying-feature-set tasks

High Level Task-level controller Applications

14

Directional redundancy

15

Directional redundancy

16

Directional redundancy

Comparison

Classical redundancy Directional redundancy

Additional DOF

17

Varying-feature-set task

Final goal: Introduce the constraint IN the stack

Joint limits

JLmaxJLmin

18

Varying-feature-set task

+

=

Positioning

Joint limits

Control law

CONTINU

Positioning

Joint limits

STACK

19

Varying-feature-set task

+

=

Joint limits

Positioning

Control law

DISCONTINUOUSPositioning

Joint limits

STACK

20

Varying-feature-set task

+

=

Joint limits

Positioning

Control law

Positioning

Joint limits

STACK

OSCILLATIONS

21

Varying-feature-set task

Definition of a task: Error vector Activation matrix

Definition of a new inverse operator

Inverse of J activated by H: Associated projector:

Secondary task alone

Joint limits + secondary task

Joint limits alone

22

Low level

Stack of tasks Control from several tasks + constraints Continuity at stack change Define some functionality for high-level control

Directional redundancy Enlargement of the main-task free space

Varying-feature-set task Enlargement of the expressivity

LOCAL CONVERGENCE

23

Task sequencing for sensor-based control

CONSTRAINTS

Centering

Zoom

Perspective

Z-rotation

Stack of Tasks

TASK-LEVEL CONTROLLER

add

removeswap

24

Outline

Credo

Low Level Stack of tasks Improvements

High Level Task-level controller Applications

25

Higher Level Controllers

Stack controller

Joint-limit controller

Obstacle controller

Occlusion controller

…

CO

LLIS

ION

PR

ED

ICT

ED

?

Remove a task

e1

ei-1

ei

STACK

en

constraints

26

Higher Level Controllers

Examples

REMOVE

JOINT-LIMIT COLLISION?

27

Higher Level Controllers

Push-back controller

Joint-limit controller

Obstacle controller

Occlusion controller

…

CO

LLIS

ION

PR

ED

ICT

ED

?

Push-back controller

OB

ST

AC

LE

AV

OID

ED

?

Add a removed taskRemove a task

e1

ei-1

ei

STACK

en

constraints

28

Higher Level Controllers

Examples

PUSH-BACK

JOINT-LIMIT AVOIDED?

initial

final

29

Higher Level Controllers

Look-ahead controller

Joint-limit controller

Obstacle controller

Occlusion controller

…

CO

LLIS

ION

PR

ED

ICT

ED

?

Push-back controller

OB

ST

AC

LE

AV

OID

ED

?

Look-ahead controller

LO

CA

L M

INM

A?

D

EA

D L

OC

K?

Add a specific task

Add a removed taskRemove a task

e1

ei-1

ei

STACK

en

constraints

30

Higher Level Controllers

Examples

Desired position

REMOVE

PUSHBACKRECONFIGURE

PUSHBACK

31

Global task Positioning (6 DOF)

Define a set of tasks Tasks to be realized

Centering Zoom Rotation Perspective

Constraints to be respected Joint limits Occlusion Obstacles

High level controller Constraint controller Push-back controller Reconfiguration controller

Applications Afma6 manipulator robot

32

Centering

Zoom

Perspective

Z-rotation

Stack of Tasks

CONSTRAINTS

33

Centering

Zoom

Perspective

Z-rotation

Stack of Tasks

CONSTRAINTS

34

Applications Non-holonomic robot

35

Applications Non-holonomic robot

Depending from controller state+1,-1

-1,+1

+1,+1

-1,-1

[Promete94]

36

Global task Positioning (no obstacles)

Define a set of tasks Tasks to be realized

Positioning: second order approximation Positioning toward a virtual position

High level controller Virtual goal controller

Applications Non holonomic robot

37

Applications Non-holonomic robot

38

Applications Humanoid robot

Application of the previous work Catching a ball while walking

Define a set of tasks

39

Applications Humanoid robot

Global task Catching a ball while walking

Define a set of tasks Tasks to be realized

Centering Grasping Walking

Constraints to be respected Joint limits Arm manipulability Chest (immobile)

40

Applications Humanoid robot

Task centering:

Input dim=2

Output dim = 10

Rank = 2

Application of the previous work Catching a ball while walking

Define a set of tasks Tasks to be realized

Centering Grasping Walking

Constraints to be respected Joint limits Arm manipulability Chest (immobile)

41

Applications Humanoid robot

Task grasping:

Input dim=3

Output dim = 14

Rank = 3

Application of the previous work Catching a ball while walking

Define a set of tasks Tasks to be realized

Centering Grasping Walking

Constraints to be respected Joint limits Arm manipulability Chest (immobile)

42

Applications Humanoid robot

Subtask distance:

Input dim=1

Output dim = 14

Rank = 1

Subtask orientation:

Input dim=2

Output dim = 14

Rank = 2

divideTask grasping:

Input dim=3

Output dim = 14

Rank = 3

Application of the previous work Catching a ball while walking

Define a set of tasks Tasks to be realized

Centering Grasping Walking

Constraints to be respected Joint limits Arm manipulability Chest (immobile)

43

Applications Humanoid robot

Task walking:

Input dim=12

Output dim = 12

Rank = 12

Application of the previous work Catching a ball while walking

Define a set of tasks Tasks to be realized

Centering Grasping Walking

Constraints to be respected Joint limits Arm manipulability Chest (immobile)

44

Applications Humanoid robot

Constraint Joint Limits:

Input dim=28 → 14

Output dim = 14

Rank = 0 to 14

Application of the previous work Catching a ball while walking

Define a set of tasks Tasks to be realized

Centering Grasping Walking

Constraints to be respected Joint limits Arm manipulability Chest (immobile)

45

Applications Humanoid robot

Constraint Manipulability:

Input dim = 6

Output dim = 6

Rank = 1

Application of the previous work Catching a ball while walking

Define a set of tasks Tasks to be realized

Centering Grasping Walking

Constraints to be respected Joint limits Arm manipulability Chest (immobile)

46

Applications Humanoid robot

Constraint Chest:

Input dim = 2

Output dim = 2

Rank = 2

Application of the previous work Catching a ball while walking

Define a set of tasks Tasks to be realized

Centering Grasping Walking

Constraints to be respected Joint limits Arm manipulability Chest (immobile)

47

Applications Humanoid robot

Application of the previous work Catching a ball while walking

Define a set of tasks Tasks to be realized

Centering Grasping Walking

Constraints to be respected Joint limits Arm manipulability Chest (immobile)

High level controller Joint-limits control Reachability control

48

49

Conclusion

Low level Stack of task: a new tool for task-level control Directional redundancy Varying-feature-set task

High level Task-level controllers to ensure multiple constraints during the displacement Application for humanoid robotic An alternative to path planning?

50

Perspectives

Multiple short-term perspective Application for multi sensors Integration of the varying-feature-set control laws

Non-holonomic robots control

Generalization of the method for humanoid robot

Integration of the path planning solutions

Learning by imitation

Merci à …

Many thanks to …

Appendix 1

State of the Art

53

Appendix 1

State of art Sensor-based control + constraints

Singularity [Nelson95], joint limits [Chan95] Redundancy formalism [Liegeois77],[Hanafusa84], [Siciliano91], [Samson91]

Switching control law Visual servoing 2D / 3D [Gans03] Positioning / Visibility [Chesi03] Positioning / Obstacle avoidance [Soueres03, Folio06]

Ad-hoc sequencing [Peterson01] [Chiaverini05]

High-level control Behavior control [Brooks87],[Mataharic01] Humanoid task-level control

Appendix 2

Calibration-robust Stack of Tasks

55

1. Centering

2. Rotation

Appendix 2

Calibration-robust Stack of Tasks

56robust not robust

Centering not respected

1. Centering

2. Rotation

Appendix 2

Calibration-robust Stack of Tasks

57

The redundancy formalism is not robust to jacobian misestimation

We have an initial estimation of J … Analytical solution

… and then we estimate J on-line Learning Correction of the perturbation

[Hosoda94],[Jagersand96]

[Piepmeier99]

[Lapreste04]

Appendix 2

Calibration-robust Stack of Tasks

58

Two cameras on mobile Pan-Tilt Fixed head for the experiments

Eye-To-Hand servoing Markers on the hand

Flexible robot No real zero-position Difficult to calibrate Approximation of the Jacobian

Appendix 2

Calibration-robust Stack of Tasks

59

Analytical matrices On-line estimated matrices

Appendix 2

Calibration-robust Stack of Tasks

Appendix 3

Pseudo-linear Control for Non-holonomic Robots

61

+

Appendix 3 Pseudo-linear Control for Non-holonomic Robots

BUT

?=

62

where

In our case

+ =

[Hettlich98]

Appendix 3 Pseudo-linear Control for Non-holonomic Robots