Optimality in

Motor Control

By : Shahab VahdatSeminar of Human Motor

ControlSpring 2007

Agenda Optimal Estimation

Optimal Control

Proposed Model

Optimality Wolpert, D. M., Ghahramani, Z. & Jordan, M. I. An

internal model for sensorimotor integration. Science (1995).

Van Beers RJ, Baraduc P & Wolpert DM. Role of uncertainty in sensorimotor control. Transactions of the Royal Society (2002)

Emanuel Todorov. Optimality principles in sensorimotor control. NatureNeuroscience (2004)

Emanuel Todorov Optimal Control Theory. Bayesian Brain, Doya, K. (ed), MIT Press (2006)

Kalman Filter

kkkkkk wuBcFc 1

kkkk vcHy

State-space model is described with these equations:

The prediction step consists of two calculations:

State estimate propagation:

Kalman Filter

1)1|( ˆ.ˆ kkkk cFc

Error covariance propagation:

kTkkk QFPF

1kP

The updating step consists of three calculations

Kalman Filter

Kalman gain matrix:

State estimate update:

Error covariance update:

1kk )P(P k

Tkk

Tkk RHHHG

kkkkk IGcc )1|(ˆˆ)1|(ˆ. kkkkk cHyI

kkkkk PHGPP

For controlling a goal-directed arm movement, there are three sources of noise :

(i) noise in the sensory signals that limits perception, (ii) noise in the motor commands, leading to inaccurate

movements (iii) sensorimotor noise, which origins from inaccuracies in the

forward model and causes noisy predicted location of the body during movement.

Therefore, the time-varying Kalman gain is used for minimizing the effect of these noises and uncertainty in the overall estimate.

Sources of Noise

Kalman Filter: Sensorimotor Integration

Sensorimotor Integration

When we move our arm in darkness, we may estimate the position of our hand based on three sources of information:

• proprioceptive feedback.

• a forward model of how the motor commands have moved our arm.

• by combining our prediction from the forward model with actual proprioceptive feedback.

Experimental procedures:

Subject holds a robotic arm in total darkness. The hand is briefly illuminated. An arrow is displayed to left or right, showing which way to move the hand. In some cases, the robot produces a constant force that assists or resists the movement. The subject slowly moves the hand until a tone is sounded. They use the other hand to move a mouse cursor to show where they think their hand is located.

Experiment: Sensorimotor Integration

Optimal Control

Bellman equations:

Continuous control:

Hamilton-Jacobi-Bellman equations:

Deterministic control: Pontryagin’s maximum principle

Hamiltonian:

Linear-quadratic-Gaussian control:

Riccati equation:

Optimal control as a theory of biological movement

state equations:

Optimal control as a theory of biological movement

Open-Loop versus Close-Loop Optimal Controller

Feed forward optimality models explain some of the classical motor properties (bell shaped profiles, etc) Harris & Wolpert, 1998- Min. Variance Flash and Hogan, 1985- Min. Jerk

Task constraints and motor noise combine to determine optimal motor plans

Humans use continuous visual feedback Noise in the sensory system very accurately predicts how

people use feedback Task constraints may also impact feedback control

strategies

Schematic illustration of open- and closed-loop optimization. (a) The optimization phase, which corresponds to planning or learning, starts with a specification of the task goal and the initial state. Both approaches yield a feedback control law, but in the case of open-loop optimization, the feedback portion of the control law is predefined and not adapted to the task.

(b) Either feedback controller can be used online to execute movements, although controller 2 will generally yield better performance. The estimator needs an efference copy of recent motor commands in order to compensate for sensory delays. Note that the estimator and controller are in a loop; thus they can continue to generate time-varying commands even if sensory feedback becomes unavailable. Noise is typically modeled as a property of the sensorimotor periphery, although a significant portion of it may originate in the nervous system.

Proposed Model:

Optimal Primitive State Prediction

k

iii tqqct

1

),,()(

),,()( qqqDt

k

iii tqqcqqqD

1

),,(),,(

Force fields as primitives for internal models:

Proposed Model:

Optimal Primitive State Prediction

Estimation and Control Equations:

)...( )1|()1|()|( kkkkkkkkk CHYGCC

..)|()|1( kkkkkk UBCC

Proposed Model:

Optimal Primitive State Prediction

Proposed Model:

Optimal Primitive State Prediction

Primitive modular representation of the cerebellum