SEOUL NATIONAL UNIVERSITY Bio-inspired Robot Motor Learning F.C. Park Seoul National University.

SEOUL NATIONAL UNIVERSITY

Bio-inspired Robot Motor Learning

F.C. ParkSeoul National University


The State-of-the-Art

Sony Q-Rio Demonstration


Some Questions

Is there a “physical intelligence” behind these movements, e.g., are they based on certain “primitives” that can be reused and generalized to new contexts (learning)? Or are they just manually created movements?

How much “feedforward” versus “feedback” is used in the motor program? What, if any, distinctions can be made between “planning” and “control”?


Lessons from Human Motor Control

Bernstein’s degrees of freedom problem and motor synergies

Movement learning as a transition process from closed-loop to open-loop control.

Alain Berthoz’s theories about perception as simulated action—the brain as a “simulator” that anticipates (Alain Berthoz,The Brain’s Sense of Movement)


Topics of this (Very Brief) Talk

Describe my attempts to build a general framework for movement generation

Creating feedforward plans: dynamically optimal movements.

Movement primitives using principal components and HMMs

Primitives for balancing and posture


State-of-the-Art in Robotics

Operational space control and elastic roadmaps (Khatib, Brock): Potential functions in operational space are used to define goal “attractors”, avoid collisions, etc., all the while exploiting null space dynamics to, e.g., maintain posture.

Rapidly-exploring random trees (RRT) (Kuffner, LaValle): Incrementally growing a randomized search tree by applying control inputs over short intervals.


Elastic Roadmap Implementation


Is There Room for Improvement?

Potential functions only capture feedback; little if any feedforward or memory-based aspects of movement.

Potential functions for, e.g., dynamic balance, are not easily formulated (ZMP depends on COM acceleration coordinates)

RRT methods are sensitive to the choice of metric, and nearest neighbor computations are still a burden

The resulting motions are not necessarily optimal or reusable.


Creating Optimal Movements

Minimize an integral cost functional of the form

subject to the dynamic equations

and various boundary conditions and

constraints on the state and control


Example: Minimum Torque Motions

Numerical algorithms like steepest descent, Newton-type methods require gradient (and sometimes Hessian) information:

Dynamic equations must be differentiated.

Without analytic gradients (and sometimes Hessians), convergence is unreliable.


Robot Dynamics Equations Dynamics for a standard 6-axis industrial robot:


Robot Dynamics Equations (p2)


Robot Dynamics Equations (p3)


Robot Weightlifting Demo (w/J. Bobrow)

PUMA Robot Weightlifting Demo


Why Can’t Robots Jump like Humans?

The human biarticular muscle is important in the generation of fast explosive human movements.

Spring-like behavior (energy restoring and release) is achieved by the elastic property of muscle, and efficient energy transfer by extension over two joints improves, e.g., jumping performance.

Biarticular structures for robots are intriguing.

< monoarticular muscle > < biarticular muscle >


A Biarticular Robot Leg (J. Babic)

< Robot model >

(1) Initially acts as passive prismatic joint

(2) When biarticular mechanism is activated, robot becomes redundantly actuated by biarticular force

(3) Immediately after robot pushes off the ground, the biarticular actuator is deactivated

0

0

0

-

-

-

( )biarticular

x b a

x b a

b a

k

f k x x

Length of

Length of at the moment of activation

Vector representing the direction of force

Biarticular link stiffness (N/m)

Activation moment angle


Optimal Jumps for Biarticular Legs Dynamics model involves not only a closed chain with redundant

actuation and spring element, but also contact models between the feet and ground.

Lemke’s algorithm for the LCP formulation to solve contact problem with friction

Time stepping schemes to reduce integration error

Optimization involves diverse parameters in addition to the actuator inputs: Spring stiffness, Biarticular actuator activation angle

,

,

( ) ( , ) a

p

c b

c b

a aT Tc c b b

p p

q

q

F F

J J

qM q b q q J F J F

q

active joints

passive joints

Generalized contact forces and biarticular forces

J acobians for each position and orientation


Maximum Height Jumps Maximum height vertical jump

Nearly 23% higher (comparing the conventional robot)

Dynamics constraints considered. Joint torque limits, joint velocity limit

Conventional robot

< Initial motion >

Biarticular legged robot (k=10000N/m, )

< Opimized motion >0

90


Maximum Distance Broad Jumps Maximum distance broad jump

Nearly 20% longer (comparing the conventional robot)

Dynamics constraints considered. Joint torque limits, joint velocity limit

Conventional robot

< Initial motion >

Biarticular legged robot (k=5000N/m, )

< Optimized motion > 080


Optimization: Some Shortcomings

Computationally expensive, not real-time.

Results are not readily re-usable. Not clear how to, e.g., perform complex

movement sequences, or optimally perform multiple tasks.


Reusability: Movement Primitives

Movement primitives for reaching motions are extracted by a principal component analysis of training data (e.g., dynamically optimal movements, motion capture data)

The primitives can serve as basis functions for linear interpolation.

The primitives can also serve as basis functions for dynamic optimization.


PCA of Arm-Reaching Motions

Repeated trials of an arm motion (e.g., lifting) are performed

PCA

First 4 principal components for each jointJoint angle trajectories of sample motions


Sample Primitive Database

TaskTrial

s

raise hand 156

put down hand 156

reach out hand 88

spread arms 89

revolve arms forward 55

revolve arms backward 60

raise hand with dumbbell

100

put down hand with dumbbell

100

hold out hand for handshaking

100

handshaking 100

restore hand after handshaking

100

small hand-waving 92

big hand-waving 88

bow 54


Example Reaching Movements

Optimal Movements Sub-Optimal Movements

Optimal movements actually appear less natural than the PCA-based (suboptimal) movements


Primitives for Balancing & Posture Given:

A legged robot standing on some subset of its legs

An input reference joint trajectory, possibly dynamically unstable

Objective: Adjust the reference joint trajectory in real-time to maintain posture while tracking the input motion withstanding external

disturbances


An Optimization Formulation

General balancing can be cast as a second-order cone programming (SOCP) problem:

nfzZMPfzZMP

MzZMPMzZMP

fxyZMPfxyZMP

COMCOM

ViconstV

MxyZMPMxyZMP

ref

ttCxM

tCxM

tCxM

bxA

bVxA

CxM

txx

t

ii

21,,

2,,

1,,

,

,,

0

0

0

subject to

min


A Hierarchy of Optimization Problems

A partial hierarchy of convex optimization problems:

SOCP is a slightly generalized version of QCQP

General formulation of SOCP problem (c = 0 leads to QCQP):

General features of convex programming problems: Any local optimum is also a global optimum. There exist many algorithms and software, e.g., MOSEK, free software

by Boyd and Vandenberghe, etc.

gFx

idxcbxA

xf

iTiii

T

subject to

min

LCLP LCQP QCQP SOCP NLCP⊂ ⊂ ⊂⊂


Maintaining Posture in Real-Time

The original motion is a standing posture. The results closely resemble a human’s reaction.

Acceleration profile

Time/step

(along x-axis)

1.4 0.423052

-0.9 0.390567

(rotation about y-axis)

6 0.385343

-6 0.398187

sina ta

2 sina t

2 / ft The units of are and a 2/m s 2deg/ s


Kicking while Maintaining Balance

Acceleration profile

Time/step

(along x-axis)0.7 0.38905

2

-0.8 0.392851

(along y-axis)0.45 0.38260

4

-0.65 0.395679

(rotation about y-axis)4.5 0.39582

8

-2.5 0.385179

(rotation about x-axis)3 0.38397

0

-3 0.391993

sina t

a

2 sina t

sina t

2 sina t

Original (unstable) kick

Stabilized kick

ZMP trajectories COG trajectories


Towards a Unified Motor Control Theory

Further inspiration from human motor control: Minimum Variance Principle (Wolpert),

Minimum Intervention Principle (Todorov and Jordan)

The Minimum Attention Paradigm (Brockett): Often imprecise control is as good as precise control—

reducing control implementation costs is more important.

Defining control implementation costs: the easiest control is a constant control.

One possible optimal control formulation:

),(

),,(

subject to

min

uxfdt

dx

dxdtdx

du

dt

duxL

SEOUL NATIONAL UNIVERSITY Bio-inspired Robot Motor Learning F.C. Park Seoul National University.

Documents

Transcript of SEOUL NATIONAL UNIVERSITY Bio-inspired Robot Motor Learning F.C. Park Seoul National University.