Crash course in control theory for neuroscientists and biologists

1

Control-theoretic approach to the analysis and synthesis of sensorimotor loops

A few main principles and connections to neuroscience

Neurotheory and Engineering seminar - 05/28/2013

Matteo Mischiati

2

• Control theory framework - linear time-invariant (LTI) case

• Properties of feedback - internal model principle

• Common control schemes - forward and inverse models, Smith predictor - state feedback, observers, optimal control

• A model of human response in manual tracking tasks (Kleinman et al. 1970, Gawthrop et al. 2011)

Matteo Mischiati Control theory primer

3

Control theory framework

Assume you have a system (PLANT) for which you can control certain variables (inputs) and sense others (outputs). There may be disturbances on your inputs and your sensed outputs may be noisy.

𝑢 𝑦PLANT

𝑢𝑐++

𝑛++

𝑑𝑦 𝑛


4


SYNTHESIS problem: design a controller that applies the right inputs to the plant, based on the noisy outputs, to achieve a desired goal while satisfying one or more performance criteria

𝑢 𝑦PLANT

𝑢𝑐++

𝑛++

𝑑𝑦 𝑛

~𝑦CONTROLLER


5


SYNTHESIS problem: design a controller that applies the right inputs to the plant, based on the noisy outputs, to achieve a desired goal while satisfying one or more performance criteria

Possible Goals: • Output regulation (disturbance rejection, homeostasis) : keep constant despite disturbance• Trajectory tracking : keep

Performance criteria:• Static performance (at steady state): e.g. • Dynamic performance: transient time, etc…• Stability: not blowing up!• Robustness: amount of disturbance that can be tolerated• Limited control effort

𝑢 𝑦PLANT

𝑢𝑐++

𝑛++

𝑑𝑦 𝑛

~𝑦CONTROLLER


6


ANALYSIS problem: infer the functional structure of the controller, given the observed performance of the overall system in multiple tasks

Goals: • Output regulation (disturbance rejection, homeostasis) : keep constant despite disturbance • Trajectory tracking : keep

Performance criteria:• Static performance (at steady state): e.g. • Dynamic performance: transient time, etc…• Stability: not blowing up!• Robustness: amount of disturbance that can be tolerated• Limited control effort

++𝑢

𝑛𝑦

PLANT

++

𝑑𝑢𝑐 𝑦 𝑛

~𝑦CONTROLLER

?


7

Example of analysis problem: uncovering the dragonfly control system

𝒗 𝑫𝑭

𝒉𝒆𝒂𝒅

𝒓

• We want to precisely characterize the foraging behavior of the dragonfly (what it does) to gain insight on its neural circuitry (how it does it).

𝒗 𝑫𝑭

𝒉𝒆𝒂𝒅𝒇𝒍𝒚

?

, relative to dragonfly, in ref. frame

dragonfly accel.

head rotation

dragonfly head, body & wing dynamics

dragonfly visual system

𝐑𝐄𝐓𝐈𝐍𝐀? 𝐓𝐒𝐃𝐍 s?𝐖𝐈𝐍𝐆𝐌𝐔𝐒𝐂𝐋𝐄𝐒𝐍𝐄𝐂𝐊𝐌𝐔𝐒𝐂𝐋𝐄𝐒

𝒃𝒐𝒅𝒚

𝒃𝒐𝒅𝒚


8

Linear time-invariant systems

,

• Stability (of a system) have negative real part• Performance (of a system): depends on location of poles and zeros • Related to transfer function in frequency domain:

++𝑢

𝑛𝑦

PLANT

++

𝑑𝑢𝑐 𝑦 𝑛

~𝑦CONTROLLER

𝑃 (𝑠) ++𝑈 (𝑠)

𝑁 (𝑠)𝑌 (𝑠)

PLANT

++

𝐷(𝑠)𝑈𝐶 (𝑠) 𝑌 𝑛(𝑠)~𝑌 (𝑠 )𝐶 (𝑠)

CONTROLLER

Laplace transform

Y (𝑠 )=𝑃 ( 𝑠 ) ∙𝑈 (𝑠)


9

Linear time-invariant systems

Laplace transform for signals:• Final value theorem : (if limit exists)• If then

Typical reference/disturbance signals:- Step

- Ramp

- Sinusoid

𝑃 (𝑠) ++𝑈 (𝑠)


PLANT

++

𝐷(𝑠)𝑈𝐶 (𝑠) 𝑌 𝑛(𝑠)~𝑌 (𝑠 )𝐶 (𝑠)

CONTROLLER

Y (𝑠 )=𝑃 ( 𝑠 ) ∙𝑈 (𝑠)

𝑡

~𝑦 (𝑡) 𝑎

𝑡

~𝑦 (𝑡) 𝑎𝑡

𝑡

~𝑦 (𝑡) sin (𝜔 𝑡 )


10






11

Feedforward (inverse model)

StabilityDepends on poles (and zeros!) of

Performance (static and dynamic)Arbitrarily good if and its inverse exists and is stable:

Robustness to disturbance (disturbance rejection)None :

𝑃 (𝑠)𝑈 (𝑠) 𝑌 (𝑠)PLANT

++

𝐷(𝑠)𝑈𝐶 (𝑠)~𝑌 (𝑠 ) �̂�−1(𝑠)

CONTROLLER

Y (𝑠 )=𝑃 ( 𝑠 ) ∙𝑈 (𝑠)U c (𝑠 )= �̂�−1 ( 𝑠 ) ∙~𝑌 (𝑠)


12

Properties of Feedback

StabilityDepends on . Can potentially stabilize unstable plants.

Disturbance rejectionDepends on . Can potentially attenuate/cancel effect of


++

𝐷(𝑠)𝑈𝐶 (𝑠)𝐸 (𝑠)𝐶 (𝑠)

CONTROLLER~𝑌 (𝑠 )

Y (𝑠 )=𝑃 ( 𝑠 ) ∙𝑈 (𝑠)U c (𝑠 )=𝐶 ( 𝑠 ) ∙𝐸(𝑠)+-


13


Static performance

Let’s see how different controllers perform:


++



+-𝑒 .𝑔 . 1

1+𝜏 𝑠𝑘


14


Static performance

Proportional controller: errors in tracking a step

(but small if is large)

cannot track a ramp at all


++



+-

𝑎𝑠

𝑎𝑠2

𝑒 .𝑔 . 11+𝜏 𝑠

𝑘


15


Static performance

Proportional+Integral (PI) controller: perfect in tracking a step


++



+-

𝑎𝑠

𝑎𝑠2

11+𝜏 𝑠𝑘𝑝+

𝑘𝑖

𝑠


( but small if 𝑘𝑖 is large )

16

Properties of FeedbackStatic performance

To perfectly track: We need: Step

Ramp

How about sinusoid? We need:

𝑎𝑠

𝑎𝑠2

𝜔𝑠2+𝜔2

𝑃 (𝑠)𝐶 ( 𝑠 )=1𝑠 ⋅(𝑃 ( 𝑠) 𝐶 (𝑠 )) ′

𝑃 (𝑠)𝐶 ( 𝑠 )= 1𝑠2

⋅(𝑃 ( 𝑠 ) 𝐶 (𝑠 )) ′

𝐺 ( 𝑠)


17

Internal model principleTo perfectly track: We need: Step

Ramp

Sinusoid

Internal model principle: To achieve asymptotical tracking of a reference signal (rejection of a disturbance signal) via feedback, the controller (or the plant) must contain an “internal model” of the signal.

It is a necessary condition, not a sufficient condition (need also stability)

𝑎𝑠

𝑎𝑠2

𝜔𝑠2+𝜔2

𝑃 (𝑠)𝐶 ( 𝑠 )=1𝑠 ⋅(𝑃 ( 𝑠) 𝐶 (𝑠 )) ′

𝑃 (𝑠)𝐶 ( 𝑠 )= 1𝑠2

⋅(𝑃 ( 𝑠 ) 𝐶 (𝑠 )) ′

𝑃 (𝑠)𝐶 ( 𝑠 )= 1𝑠2+𝜔2 ⋅(𝑃 ( 𝑠 ) 𝐶 (𝑠 )) ′


18

Feedback vs. Feedforward

Feedback• is needed if plant is unstable or for disturbance rejection• does not require full knowledge of the plant• incorporating the knowledge of possible reference and disturbance

signals is very useful (internal model principle)

Feedforward• if plant is known, and no disturbance, its performance can’t be beat• no sensory delays

𝑌 (𝑠 )= 𝑃 ( 𝑠) 𝐶 (𝑠 )1+𝑃 (𝑠 )𝐶 (𝑆 )

~𝑌 ( 𝑠 )+ 𝑃 ( 𝑠 )1+𝑃 (𝑠 ) 𝐶 ( 𝑆 )

𝐷 (𝑠 )𝑌 (𝑠 )=𝑃 (𝑠 ) ∙ (�̂� ¿¿−1 ( 𝑠) ⋅~𝑌 ( 𝑠)+𝐷 (𝑠))≈~𝑌 (𝑠)+𝑃 (𝑠)⋅ 𝐷(𝑠)¿


19






20

Feedback + Feedforward

The feedback controller kicks in only if inverse model is incorrect.


++

𝑈 𝐹𝐵(𝑠)

𝐸 (𝑠)𝐶 (𝑠)~𝑌 (𝑠 )

+-

�̂�−1(𝑠)INVERSE MODEL

FEEDBACK

𝑈 𝐹𝐹(𝑠)


21



The corrective command from the feedback path can be also used as a learning/adaptation signal by the inverse model.


++

𝑈 𝐹𝐵(𝑠)

𝐸 (𝑠)𝐶 (𝑠)~𝑌 (𝑠 )

+-


FEEDBACK

𝑈 𝐹𝐹(𝑠)


22



The corrective command from the feedback path can be also used as a learning/adaptation signal by the inverse model.

Significant sensory delays are still a problem.


++

𝑈 𝐹𝐵(𝑠)

𝐸 (𝑠)𝐶 (𝑠)~𝑌 (𝑠 )

+-


FEEDBACK

𝑈 𝐹𝐹(𝑠)

𝑒− 𝑠𝜏


23

Forward model

The control signal is sent through a model of the plant (“forward model”) to predict the sensory output.

𝑃 (𝑠)𝑌 (𝑠)PLANT

𝑈𝐶 (𝑠)𝐸 (𝑠)𝐶 (𝑠)~𝑌 (𝑠 )

+-

�̂� (𝑠)FORWARD MODEL

CONTROLLER

predicted sensory output


24

Forward model

The control signal is sent through a model of the plant (“forward model”) to predict the sensory output.

The (delayed) sensory output can be used as a learning/adaptation signal for the forward model.

Direct use of the delayed sensory output in the control is problematic because of time mismatch.

𝑃 (𝑠)𝑌 (𝑠)PLANT

𝑈𝐶 (𝑠)𝐸 (𝑠)𝐶 (𝑠)~𝑌 (𝑠 )

+-

�̂� (𝑠)FORWARD MODEL

𝑒− 𝑠𝜏

CONTROLLER



25

Smith predictor

Assuming and :

Delay has been moved outside the control loop.

PLANT

𝑃 (𝑠)𝑌 (𝑠)𝑈𝐶 (𝑠)𝐸 (𝑠)𝐶 (𝑠)~𝑌 (𝑠 )

+- -

�̂� (𝑠)𝑒− 𝑠𝜏

𝑒− 𝑠�̂�

+ -

delay model

plant model


error in sensory output prediction

CONTROLLER


26

Models of the cerebellum

1. Cerebellum as an inverse model in a feedback+feedforward motor control scheme

Wolpert, Miall & Kawato, 1998 “Internal models in the cerebellum” Not in the sense of my presentation !


27

Models of the cerebellum

2. Cerebellum as aforward model in a Smith predictor control scheme

Wolpert, Miall & Kawato, 1998 “Internal models in the cerebellum”


28

State feedback

𝑦PLANT

𝑢~𝑦CONTROLLER

𝒙

𝑦PLANT

𝑢~𝑦CONTROLLER

𝒙

Linear time-invariant case:


29

State feedback

If the plant is reachable, it is possible to achieve any arbitrary choice of closed-loop poles with an appropriate linear and memoryless controller:

𝑦PLANT

𝑢~𝑦CONTROLLER

𝒙

𝑦PLANT

𝑢~𝑦CONTROLLER

𝒙

Linear time-invariant case:

𝐾

𝐾 𝑟 +-


30

Observers

Observer: dynamical system designed to estimate the full state (when not fully available)

If the plant is observable, it is possible to achieve (with right )

𝑦PLANT

𝑢

𝒙 )

OBSERVER


31

Observers

Observer: dynamical system designed to estimate the full state (when not fully available)

If the plant is observable, it is possible to achieve (with right )

Separation principle: if the plant is reachable & observable, can replace with and design independently of (use observed state just as real one)

𝑦PLANT

𝑢

𝒙 )

~𝑦

𝐾

𝐾 𝑟 +-

OBSERVER


32

Optimal control

Linear Quadratic Gaussian (LQG) optimal output regulation: linear plant, additive Gaussian white noise on state (with covariance ) and output (); minimize quadratic cost :

Solution is linear observer (Kalman filter) with linear memoryless controller:

𝑦PLANT

𝑢

𝒙 )

~𝑦=0

𝐾

+-

OBSERVER (KALMAN FILTER)

𝒅++

𝑛𝑦 𝑛


33

Internal model principle

Internal model principle (state space): to achieve asymptotical tracking of a reference signal (rejection of a disturbance signal) produced by an exosystem, the controller must contain an “internal model” of the exosystem.(Francis & Wonham, Automatica, 1970)

It is a necessary condition, not a sufficient condition (need also stability).

General principle with extensions to nonlinear systems.

𝑦PLANT

𝒖~𝑦 �̇�=𝑆𝜼+𝐺𝑒+-𝑒

CONTROLLER

𝜼INT.MODELEXOSYSTEM


34






35

Human response modelD. L. Kleinman, S. Baron and W. H. Levison, “An Optimal Control Model of Human Response. Part I: Theory and Validation”, Automatica, 1970 (revisited, more recently, by Gawthrop et al. 2011)

Task: by controlling a joystick (position, velocity or acceleration control), subject is asked to keep a cursor on the screen as close as possible to a target location, while unknown disturbances are applied by the computer.

Plant:𝑚𝑜𝑛𝑖𝑡𝑜𝑟 𝑌 (𝑠)

++

(computer)

𝑈 (𝑠) 𝑗𝑜𝑦𝑠𝑡𝑖𝑐𝑘 𝑃 𝐽𝑀 ( 𝑠 ) ∈{𝑘 , 𝑘𝑠 , 𝑘𝑠2 }

“Human controller”:

dynamics

𝑈 (𝑠)++

𝑚𝑜𝑡𝑜𝑟 𝑛𝑜𝑖𝑠𝑒

computation

𝑃 𝑁 (𝑠 )𝐶 (𝑠 )𝑒− 𝑠𝜏


36

Human response modelTask: Output regulation/disturbance rejection with linear time-invariant plant and significant delay on any potential feedback line

𝑒− 𝑠𝜏

𝑃 (𝑠) ++𝑈 (𝑠)


++

𝐷(𝑠)𝑈𝐶 (𝑠) 𝑌 𝑛(𝑠)𝐶 (𝑠)

CONTROLLER

𝑃 𝐽𝑀 ( 𝑠) ∙𝑃𝑁 (𝑠 )

ANALYSIS problem: infer a model of the neural controller from the observed performance of the subjects tested.


37

Human response modelTask: Output regulation/disturbance rejection with linear time-invariant plant and significant delay on any potential feedback line

𝑒− 𝑠𝜏

𝑃 (𝑠) ++𝑈 (𝑠)


++

𝐷(𝑠)𝑈𝐶 (𝑠) 𝑌 𝑛(𝑠)𝐶 (𝑠)

CONTROLLER

𝑃 𝐽𝑀 ( 𝑠) ∙𝑃𝑁 (𝑠 )

ANALYSIS problem: infer a model of the neural controller from the observed performance of the subjects tested.

So what are the performances?• Very good and robust to disturbances up to 2Hz (sum of sinusoids), for all

three types of joystick dynamics• Apparently delay-free

Must be some kind of FEEDBACK + FORWARD model !


38

Human response modelHypothesis: optimal control to minimize average error & control effort

Theoretical solution * (with assumptions similar to LQG problem): - Optimal observer (Kalman filter) to estimate delayed state (as in LQG)- Optimal least mean-squared predictor to predict current state- Optimal linear memoryless controller (as in LQG)

𝑦PLANT

𝑢

�̂� (𝑡−𝜏 )KALMAN FILTER

~𝑦=0

𝐾

+-

𝒅++

𝑛𝑦 𝑛

* D. Kleinman, “Optimal control of linear systems with time-delay and observation noise”, IEEE Trans. Autom. Control, 1969

𝑒− 𝑠𝜏PREDICTOR

�̂� (𝑡) 𝑦 𝑛(𝑡−𝜏 )


39

Controller freq. response with plant Controller freq. response with plant


40

Human response modelGawthrop et al. * (2011): - Introduced, in both estimator and predictor, a copy of the exosystem

generating sinusoidal disturbances (internal model principle!)- Show that intermittent control is also compatible with results

* P. Gawthrop et al., “Intermittent control: a computational theory of human control”, Biol. Cybern., 2011

Actual response to sinusoid Response without int.model


41

• Crash course in control theory (for LTI systems) - many concepts can be extended to more general settings

• An example of control-theoretic approach to modeling sensorimotor loops

- need to iterate between modeling/experiments to discern among alternatives and improve understanding of the system

Conclusions

THANK YOU FOR YOUR ATTENTION !


Crash course in control theory for neuroscientists and biologists

Education

Transcript of Crash course in control theory for neuroscientists and biologists