Manish Kurse PhD research slides

Inference of computational models of tendon networks via

sparse experimentation

Manish Umesh KurseApr 11, 2012

1

Brain-Body Dynamics LaboratoryPh.D. committee: Dr. Francisco J. Valero-Cuevas, Dr. Hod Lipson,

Dr. Gerald E. Loeb, Dr. Eva Kanso

2

MSMS: Davoodi et al., 2007

Measurement of internal states Injury, deformity, surgery

h4p://www.ispub.com/

Ergonomics, prosthetic design, etc.

h4p://www.anybodytech.com

Inputs Outputs

tension in each cord, which was fed back to the motor so that a desired amount oftension could be maintained on each tendon. The fingertip was rigidly attached to6 DOF load cell (JR3, Woodland, CA).

We examined 5 different postures in 3 specimens, and 3 different postures inthe final specimen. Each posture was neutral in add-abduction. The examinedpostures were chosen to cover the workspace and simulate those found ineveryday tasks. After positioning the finger in a specific posture, we determinedthe action matrix for the finger: we applied 128 combinations of tendon tensionsrepresenting all possible combinations of 0 and 10 N across the seven tendons, andheld each combination for 3 s. The fingertip forces resulting from each coordina-tion pattern was determined by averaging the fingertip load cell readings acrossthe hold period. Linear regression was performed on each fingertip forcecomponent using the tendon tensions as factors. In this way, the fingertip forcevector generated by 1 N of tendon tension was determined for all muscles. Theforce vector generated by each muscle was scaled by an estimate for maximummuscle force (Valero-Cuevas et al., 2000) to generate the columns of the actionmatrix for each specimen and posture examined.

2.2. Action matrix for human leg model

We also studied the necessity of muscles for mechanical output for asimplified, but plausible, sagittal plane model of the human leg (hip, knee, andankle joints). The model contained 14 muscles/muscle groups (Kuo and Zajac,1993) (muscle/muscle group abbreviation in parentheses): medial and lateralgastrocnemius (gastroc), soleus (soleus), tibialis posterior (tibpost), peroneusbrevis (perbrev), tibialis anterior (tibant), semimembranoseus/semitendenosis/biceps femoris long head (hamstring), biceps femoris short head (bfsh), rectusfemoris (rectfem), gluteus medialis/glueteus minimus (glmed/min), adductor

longus (addlong), iliacus (iliacus), tensor facia lata (tensfl), gluteus maximus(glmax). Moment arms for hip flexion, knee flexion and ankle dorsiflexion for allof these muscles were obtained from a computer model of the lower limb (Arnoldet al., 2010). When necessary, multiple muscles in were combined into the singlemuscle groups. We derived a 3!3 square Jacobian mapping changes in the hip,knee, and ankle angle to the foot position in the plane (2 components) and theorientation of the foot in space. This Jacobian matrix, inverted and transposed, wascombined with the moment arms and maximal muscle forces to form the actionmatrix mapping muscle activation to forces and torques at the foot (Valero-Cuevas, 2005b), although our analysis of muscle redundancy was only performedwith respect to the endpoint forces.

2.3. Analyzing the action matrix to determine muscle necessity

We used the action matrix to determine whether muscles are necessaryfor a given desired output force using standard tools in computational geometry.The muscle redundancy problem can be expressed as a set of linear inequalities(Chao and An, 1978; Spoor, 1983). These inequality constraints enforce that theactivation for each muscle lie between 0 and 1, and that the actual output force isequal to the desired force. The inequality constraints define a region in muscleactivation space called the task-specific activation set: any point inside that set willproduce the desired output force (Kuo and Zajac, 1993; Valero-Cuevas, 2005b;Valero-Cuevas et al., 2000, 1998). We computed the vertices defining the task-specific activation set using a vertex enumeration algorithm (Avis and Fukuda,1992). We then found the task-specific activation ranges to achieve the desiredoutput force for each muscle by projecting all vertices onto the seven musclecoordinate axes to determine the minimum and maximum task-specific activations.While previous studies have used similar experimental (Valero-Cuevas et al., 2000)

Muscles cooperateto exert force

Feasible force set,one target force vector

Solutions in muscle activation space,task-specific activation set

Fy

Fx

Target x-force

Target y-force

Feasible force set

Target force vector

2

12

3

LIMB

31

Fy

Fx

Task-specificactivation ranges

0

1

Muscle

1 2 3

Neces

sary

Neces

sary

Redun

dant

Muscle 3

Muscle 2

Muscle 1

Target x-force sol’nsTarget y-force sol’ns

Task-specific activation set

Fig. 1. Three muscle ‘‘schematic model’’ conceptually illustrates the necessity of muscles. (a) Muscles can be functionally visualized as force vectors at the endpoint. (b) Aregion of force space, the feasible force set, is achievable given this musculature. A particular target force vector can be decomposed into a target x-force and a targety-force. (c) The valid coordination patterns for the x and y targets can also be viewed in muscle activation space as planes; the portion of the intersection of these twoplanes that is inside the unit cube is the task-specific activation set. Any point on the task-specific activation set will generate the same target force vector. (d) The task-specific activation set can be projected onto the muscle coordinate axes, revealing the minimum and maximum activation in each muscle for the given applied force vector.The task-specific activation ranges can be constructed for each muscle, and reveal which muscles are necessary and which are redundant for a given target force vector.

J.J. Kutch, F.J. Valero-Cuevas / Journal of Biomechanics 44 (2011) 1264–1270 1265

Kutch and Valero-Cuevas, 2011

Computational modeling of musculoskeletal systems

3

Modeling : Structure based on observation + experimental measurement of some parameters.

Drawbacks: • Not possible to measure all parameters. • Not validated with experimental input-‐output data. • Structure assumed need not be funcEonally accurate representaEon.

R

System

✓s

R

Structure assumed,

parameters fit.

1 R2

Infer structure and parameters from input-

output data

(✓)

4

Develop computational methods to simultaneously infer structure and parameter values of functionally accurate

models of musculoskeletal systems directly from experimental input-output data.

Objective

Inputs Outputs











Fy

Fx

Target x-force

Target y-force

Feasible force set

Target force vector

2

12

3

LIMB

31

Fy

Fx


0

1

Muscle

1 2 3

Neces

sary

Neces

sary

Redun

dant

Muscle 3

Muscle 2

Muscle 1






Tendon networks of the fingers

5

Lateral bands

Central slip

Terminal slip

Retinacular ligament

Sagittal band

Transverse fibers

Clavero et al. (2003). “Extensor Mechanism of the Fingers: MR Imaging-Anatomic Correlation”, Radiographics

Netter, F. Atlas of Human Anatomy, 3rd edition, pp 447-453

Computational models

6

Analytical models Anatomy-based models

s2 = ✓21 + 9e✓2

Significance of research

7

http://www.myefficientassistant.com/

http://www.punchstock.com/

‘Plant’

‘Controller’

Motor control

http://www.ispub.com/

Clinical

Wilkinson et al. 2006

Robotics and prosthetics

?


Experimentation Mathematical modeling

Inference algorithms

8

Dissertation outline

Experimental actuation of a cadaveric hand.

2

3

4

5

6

New inference approach to

learn functions of tendon

routing.

Application to the human

index finger.

Experimental validation of an existing

model.

Tendon network simulator and

sensitivity analysis.

Inference of anatomy-based models

from experimental data.

1


ASME SBC ’10 & IEEE TBME ’12

ASB ’11

CSB ’12

ASME SBC ’09

9

Cadaver finger control

10

Load cells

Strings to the tendons

Motion capture markers

6 DOF Load cell

DC motorsPositon encoders

1 2 3 4 5 6

1

Finger tapping

11

1 2 3 4 5 6

Slow finger movements

12

All intactRadial nerve palsyMedian nerve palsyUlnar nerve palsy

fmIteratively,

1 2 3 4 5 6

Finger equilibrium

13

⌧ = RFm

0

Neutral equilibrium

Stable equilibrium

1 2 3 4 5 6

14

NeutralStableFDPFDSEI

EDCLUMFDIFPI

1 20

1

2

3

4

Dis

tanc

e fro

m n

ull s

pace

(N)

1 20

2

4

6

8

Coordination pattern

Tend

on te

nsio

ns

iDistance from null space

1 2 3 4 5 6

Conclusions

• Spring-based (muscle-like) control effective to control movement.

• Simple tap requires a coordinated set of tendon excursions.

• Neutral equilibrium in specific postures and tendon tensions.

15

1 2 3 4 5 6


16


s2 = ✓21 + 9e✓2

1 2 3 4 5 6

Inference of analytical functions

17

Co-authors: Dr. Hod Lipson, Dr. Francisco Valero-Cuevas

1 2 3 4 5 6

2

• Analytical functions for tendon excursionss = f(✓)

Deshpande et al. 2009

• State of the art : Polynomial regression

s s

‘Controller’

‘Plant’• Why? R(✓) =@s

@✓⌧ (✓) = R(✓)Fm

R

✓

R 1R

2(✓)

• Can we simultaneously learn form and parameter values from data?

• Compare accuracy with polynomial regression.

Specific Aims

18

1 2 3 4 5 6

s = f(✓)

19

Schmidt and Lipson, 2009

Koza 1992

Symbolic regression

1 2 3 4 5 6

Robotic tendon driven system

20

s1

s2

s3

2

1

3

Position encoders

Motors keeping tendons taut

Load cells

Motion capture markers

Motion capture camera

1 2 3 4 5 6

Landsmeer model I

Landsmeer model II

Landsmeer model III

s = 3.6sin(0.5θ)

s = 0.6θ + 3.2(1 − θ/2tan(θ/2)

)

s = 1.8θ

s = f(✓1, ✓2, ✓3)

21


Polynomial regressionKoza 1992

Symbolic regression vs.

LinearQuadratic

CubicQuartic

1 2 3 4 5 6

Comparing symbolic and polynomial regressions

22

2

5

10

20

2

5

10

20

2

5

10

20

Tendon 1

2

5

10

20

Tendon 2

Tendon 3

2

5

10

20

n/256

n/16

n/64

n n

2n

4n

8n

16n

32n

64n

128n

256

X

X

X

XX

n/256

n/16

n/64

2

5

10

20

n n

2n

4n

8n

16n

32n

64n

128n

256

n/256

n/16

n/64

n/256

n/16

n/64

n/256

n/16

n/64

n/256

n/16

n/64

Symbolic

Quartic

LinearQuadraticCubic

Dataset size (n =1688) Dataset size (n =1688)

X

X

X

Cross-validation Extrapolation

RM

S er

ror (

%)

RM

S er

ror (

%)

X Error for all sizes > 5%Min training set size < n/256

2

5

10

20

Tendon 1

Tendon 2

2

5

10

20

25%

75%

125%

25%

75%

125%

25%

75%

125%

2

5

10

20Tendon 3

RM

S er

ror (

%)

Extrapolation by volume (%)

0 25 15075 10050 125 Symbolic

Quartic

LinearQuadraticCubic

X

X

X All extrapolation errors > 5%Achievable extrapolation > 150%

Fewer training data points required More extrapolatable

2

23

Extrapolation by volume (%)

0

25

50

75

100

125

150

>150

n n

2n

4n

8n

16n

32n

64n

128

Training set size (n =1688)

Extra

pola

tion

by v

olum

e (%

)

SymbolicQuartic

LinearQuadratic

Cubic

Comparing symbolic and polynomial regressions

Fewer training data points required

More extrapolatable

Kurse et al. 2012 (in press)

1 2 3 4 5 6

24

Landsmeer model I

Landsmeer model II

Landsmeer model III

s = 3.6sin(0.5θ)

s = 0.6θ + 3.2(1 − θ/2tan(θ/2)

)

s = 1.8θ

Simulated musculoskeletal systems

Landsmeer comb. Expressions

I, I, I

Target 1.8✓1 + 1.8✓2 + 1.8✓3

Evolved 1.8✓1 + 1.8✓2 + 1.8✓3

I, II, III

Target

1.8✓1 + 3.6sin(0.5✓2) + 0.6✓3 �(1.6✓3)/tan(0.5✓3) + 3.2

Evolved

1.8✓1 + 3.61sin(0.5✓2) + 1.54✓3 �0.778sin(✓3)

II, II, I

Target 3.6sin(0.5✓1)+3.6sin(0.5✓2)+1.8✓3

Evolved 3.6sin(0.5✓1)+3.6sin(0.5✓2)+1.8✓3

Table 1: Target and inferred expressions with training, cross-validation and extrapolation RMS errors(%)

for some combinations of Landsmeer’s models I, II, III

1

1 2 3 4 5 6

Error vs. number of parameters

25

RM

S er

ror (

%)

Cro

ss-v

alid

atio

nEx

trapo

latio

n

SymbolicQuartic

LinearQuadratic

Cubic

Experimental dataWith no noise

Number of parameters

Simulated dataWith noise added

1

2

3

0 20 401

2

3

5

.0001

.01

1

0 20 40

.0001

.01

1

0 20 401

2

5

10

1

2

5

10

1 2 3 4 5 6

Conclusions

• Symbolic regression outperforms polynomial regression

• Number of training data points

• Extrapolatability

• Robustness to noise

• Number of parameters

• Insight on physics

26

1 2 3 4 5 6

Kurse et al. 2012 (in press)

27

Novel method of inference of

analytical functions from data

Application to the human finger


s1 = f(✓1, ✓2, ✓3, ✓4)

1 2 3 4 5 6

Analytical functions: Index finger

28

Constant moment arm

(Linear)

Polynomial regressions

Landsmeer based models

Landsmeer model I

Landsmeer model II

Landsmeer model III

s = 3.6sin(0.5θ)

s = 0.6θ + 3.2(1 − θ/2tan(θ/2)

)

s = 1.8θ

Landsmeer, 1961, Brook 1995

3

Co-authors: Dr. Hod Lipson, Dr. Francisco Valero-Cuevas

1 2 3 4 5 6

Eg. An et al. 1983, Valero-Cuevas et al. 1998

Eg. Franko et al. 2011

Eg. Brook et al. 1995

Specific aims

• Infer analytical functions for the seven tendons of the index finger.

• Compare against polynomial regression and Landsmeer based models.

29

1 2 3 4 5 6

Cadaver experimental setup

30

MoEon capture

Load cells

Position encodersServo motors

Markers

1 2 3 4 5 6

s1 = f(✓1, ✓2, ✓3, ✓4)

s7 = f(✓1, ✓2, ✓3, ✓4)

...

31


Polynomial and LandsmeerKoza 1992

Symbolic regression vs.

LinearQuadratic

CubicQuartic

Landsmeer

1 2 3 4 5 6

Across trials

32

FDP FDS EIP EDC LUM FDI FPI2

5

10

20

SymbolicLandsmeer

Quartic

LinearQuadratic

Cubic

2 10 502

5

10

20

50FDP

2 10 502

5

10

20

50FDS

2 10 502

5

10

20

50EIP

2 10 502

5

10

20

50EDC

2 10 502

5

10

20

50LUM

2 10 502

5

10

20

50FDI

2 10 502

5

10

20

50FPI

Tendon Number of parameters

Nor

mal

ized

RM

S er

ror (

%)

1 2 3 4 5 6

Across hands

33


5

10

20

2

5

10

20

50

Tendon

Nor

mal

ized

RM

S er

ror (

%)

SymbolicLandsmeer

Quartic

LinearQuadratic

Cubic

1 2 3 4 5 6


5

10

20

SymbolicLandsmeer

Quartic

LinearQuadratic

Cubic

10

20

50

Tendon

Nor

mal

ized

RM

S er

ror (

%)

Conclusions

• For subject-specific models as well as generalizable models,

• Symbolic regression more accurate than other models.

• Error bounds on generalizability.

• Models insight on tendon routing.

34

1 2 3 4 5 6


35


s2 = ✓21 + 9e✓2

1 2 3 4 5 6

Anatomy-based modeling

36

Co-author: Dr. Francisco Valero-Cuevas

Netter, F. Atlas of Human Anatomy, 3rd edition, pp 447-453

1 2 3 4 5 6

4

Clavero et al. 2003

Boutonniere deformity


Swan-neck deformity

Mallet finger deformity

37

• Widely used representation: An-Chao normative model (1978, 79)

TE=RB+UBRB=0.133 RI+0.167 EDC+0.667 LUUB=0.313 UI+0.167 EDCES=0.133 RI+0.313 UI+0.167 EDC+0.333 LU

Chao et al. 1978,79

1 2 3 4 5 6

Validation of An-Chao model

38

6 DOF loadcell

Load cells measur-ing tendon tensions

Strings connecting tendons to motors

Fingertip force vector

1 2 3 4 5 6

Validation of An-Chao normative model

39

• Large magnitude and direction errors in fingertip force magnitude and direction.

1 2 3 4 5 6

(Sagittal plane)


20

40

60

Dir

erro

r (de

gree

s)


200

400

600

800

1000

Mag

erro

r %

Magnitude errors Direction errorsFlexTapExtend

40

• Let the physics and mechanics decide force distribution.

• Existing musculoskeletal modeling software do not model tendon networks.

• Environment to understand role of components in force transformation.

Valero-Cuevas and Lipson, 2004

1 2 3 4 5 6

Specific aims

• Develop a modeling environment to represent these tendon networks.

• Study sensitivity of fingertip force output to properties of the extensor mechanism.

41

1 2 3 4 5 6

Tendon network simulator and sensitivity analysis

5

Import MRI scan of bones.

Define tendon network.

Tendon network simulator

Solve the nonlinear finite element problem.

1 2 3 4 5 6

42

Iteratively,

• Node and element penetration testing.

• Apply input Forces in increments

• Solve by Newton-Raphson iteration method the displacements of nodes, U(i), for system equilibrium :

Finite Element Method

• Assemble the internal force vector and the tangent stiffness matrix in each element.

43

Tension (N)

44Differential loading as observed by Sarrafian, 1970, Micks 1981

Validation

46

Motors

Load cells

Hemisphere

Reflective markers

Fixed nodes

Hemisphere

1 2 3 4 5 6

0 1 2 3 4 50

1

2

3

4

5

RF Magnitude (Model) in N

RF

Mag

nitu

de (D

ata)

in N

0 1 2 3 4 50

1

2

3

4

5

RF Magnitude (Model) in N

RF

Mag

nitu

de (D

ata)

in N

RF Node 1RF Node 2x=y line

Node 1

Node 2

Node 1

Node 2

Sensitivity analysis of parameters and topology

47

Tessellated bones

i. Locations of nodes

ii. Cross-sectional areas

iii. Resting lengths

iv. Topology

1 2 3 4 5 6

48

Fully flexed

Tap

Fully extended

Fully flexed Tap Fully extended

Fixed nodes

Sensitivity of fingertip force output in three postures1 2 3 4 5 6

Results: Sensitivity analysis

49

0

5

10

15

20

25

30

35

40

Dire

ctio

n de

viat

ion

(deg

)

−100

−50

0

50

100

150

200

Mag

nitu

de d

evia

tion

(%)

1 2 3 4 1 2 3 4

Network parameters Topology Network parameters Topology

Fully flexedTapFully

extended

Fingertip force direction and magnitude

• Sensitive to:

- Posture

- Resting lengths

- Topology

• Less sensitive to

- Node positions.

- Cross-sectional areas.

1 2 3 4 5 6

Conclusions

50

• Developed a novel tendon network simulator to represent these tendon networks.

• Studied what properties the fingertip force output is most sensitive to.

51

1 2 3 4 5 6

Simultaneous inference of topology and parameter values

Valero-Cuevas et al. 2007 Saxena et al. (in review)

Inference of anatomy-based modelsCo-authors: Dr. Hod Lipson, Dr. Francisco Valero-Cuevas

6

R

✓

R 1R

2(✓)

52

Specific aims

•Simultaneous inference of 3D tendon networks from

input-output data in simulation.

•Inference of models of the finger’s extensor

mechanism directly from input-output data via sparse

experimentation.

Inference of anatomy-based modelsCo-authors: Dr. Hod Lipson, Dr. Francisco Valero-Cuevas

61 2 3 4 5 6

Data

530 5000 10000 150000.1

0.5

2

10

50

Fitness error vs iterations

Tota

l RF

erro

r as

%

Num evaluations

CPU 1

CPU 2

CPU N

...

?

Topology and parameter inference

of 3D models

1 2 3 4 5 6

Inference of tendon networks in simulation

54

6 DOF loadcell

Load cells measur-ing tendon tensions

Strings connecting tendons to motors

Fingertip force vector

3 Postures,3 sets of inputs

1 2 3 4 5 6

Inference parameters

55

Tessellated bones

i. Locations of 6 nodes

(4 variables)

ii. Resting lengths of 13 elements(7 variables)

iii. Topology : 8 elements

(4 variables)

1 2 3 4 5 6

Estimation-exploration algorithm

Models Tests

Bongard et al., Science, 2006

1 2 3 4 5 6

56

Inference using EEA

57

Test suite

Converged?

5N 5N3N

1N1.5N3N

Start3 Random testsMeasured data

Evolve models

No

End

Two best tests selected

Estimation Exploration

1N3N3N

Identify most `intelligent’

tests (posture +

tendon tensions)

1 2 3 4 5 6

Inference results

58

0

100

200

300

400

500

600

700

RM

S M

ag e

rror

(%)

An Chao 1979

Optimized (best 5)

Full flex Tap Full Ext0

5

10

15

20

25

30

35

40

45

50

RM

S D

ir e

rror

deg

Full flex Tap Full Ext

1 2 3 4 5 6

5

10

15

20

25 N

0

Conclusions

• Demonstrated for the first time the successful inference of model topology and parameters of a complex musculoskeletal system from experimental input-output data.

• Inferred models are more accurate than models in the literature.

59

1 2 3 4 5 6

60

?




61

?




http://www.myefficientassistant.com/

http://www.punchstock.com/

‘Plant’

‘Controller’

Motor control

Boutonniere deformity


Swan-neck deformity

Mallet finger deformity

Clinical



s2 = ✓21 + 9e✓2

Conclusions and future work

62

• Applies to other systems.

•Step towards subject-specific models inferred from

data.

R

System

✓s

R2

Infer structure and parameters from input-

output data

(✓)











Fy

Fx

Target x-force

Target y-force

Feasible force set

Target force vector

2

12

3

LIMB

31

Fy

Fx


0

1

Muscle

1 2 3

Neces

sary

Neces

sary

Redun

dant

Muscle 3

Muscle 2

Muscle 1






Acknowledgements

63

Dr. Francisco Valero-Cuevas

Dr. Hod Lipson

Dr. Gerald Loeb

Dr. Eva Kanso Dr. Jason

KutchJosh Inouye Sudarshan

Dayanidhi

Dr. Heiko Hoffmann

Dr. Anupam Saxena

Dr. Jae-Woong Yi

Kornelius Rácz

Brendan Holt

Alex Reyes Emily Lawrence

Dr. Srideep Musuvathy

John Rocamora

Dr. Marta Mora

Na-hyeon Ko

Alison HuDr. Evangelos Theodorou

Dr. Caroline LeClercq

Dr. Vincent Rod Hentz

Dr. Nina Lightdale

Dr. Isabella Fasolla

Kari Oki

Dr. Terrance Sanger

Acknowledgements

64

The NaEonal Science FoundaEon: CAREER award,

EFRI - COPN to FVC

The National Institutes of HealthNIAMS/NICHD R01-AR050520; R01-AR052345

Thank you!

65

Manish Kurse PhD research slides

Science

Transcript of Manish Kurse PhD research slides