Trajectory optimization for musculoskeletal state estimation with IMU sensors
Ton van den BogertMechanical Engineering Department
1 CSU Center for Human-Machine Systems 8/27/2019
Combination of two conference presentations
2
XVth International Symposium on3D Analysis of Human Motion
Manchester, UK, July 2018Herman J. Woltring memorial lecture
International Society of BiomechanicsAmerican Society of Biomechanics
Calgary, Canada, August 2019
"[...] A generalization of these approaches is to define a state-spacemodel of rigid body positions, attitudes, and their (first and second)derivatives, and to apply optimal prediction, filtering, and smoothingtechniques [...]. This has the additional advantages of simultaneousderivative estimation and of automatic interpolation in the event ofpartial or complete data loss [...]. Unfortunately, such more universalprocedures are numerically expensive, and they require familiaritywith contemporary developments in the realm of optimal control,system identification, and parameter estimation."
Woltring HJ (1980) Comment on the paper "A technique for obtaining spatial kinematic parameters of segments of biomechanical systems from cinematographic data."Journal of Biomechanics 14: 277.
Is important in human movement, because measurements are often: inaccurate and noisy incomplete overcomplete & multiple modalities
Models can help (but aren't perfect...)
Optimal estimation can make use of all available measurements optimally combines measurements and model, with knowledge of how much to trust
each of them
Optimal estimation
Bayesian inferenceE: evidence (measurements)H: hypothesis (estimates of variables of interest)
Bayes' theorem:
Given the measurements E, which estimate H is best?Answer: find the H for which P(H | E) is highest
( | ) ( )( | ) ( ) ( | ) ( ) ( | )
( )P E H P H
P H E P E P E H P H P H EP E
ˆ argmax ( | ) ( )
HH P E H P H
H
( | ) ( )P E H P H
posterior probability
prior probability
H
Two approaches
10
Recursive (real time) Kalman filters, particle filters uses past measurements only
Trajectory optimization (not real time) uses past and future measurements
Both use the same "ingredients" process model observation model assumptions about how much to trust these models
Kalman filters: Q and R covariance matrices Trajectory optimization: effort weighting coefficient
recursiveestimation
trajectoryoptimization
Dynamic motion estimation
Nobody believes that this is actual human motion we know about Newton's laws of motion we know there is measurement error how do we estimate the actual motion?
mar
ker X
(mm
)
marker coordinatefrom optical motion capture
Optimal dynamic motion estimation (1-dimensional)
Trajectory to be estimated: Measurements:
Model: Process model: High forces have lower probability:
Observation model: Gaussian measurement error:
x u
y x v
2
2( ) exp
2 v
vP v
( ), 0x t t T
1 2( ), ( ) ( )My y t y t y t
2
2( ) exp
2 u
uP u
σ 2σ‒2σ ‒σ 0 u
2
22
u
e
P(u)
Optimal dynamic motion estimation (2)
1 2 1 2( ) | | ( (, , ) )M MP ty y y y xy yx t P x P t
Probability that x(t) is the correct trajectory, given data y:
1 2 1 2
2
2
2 2 21 1 2 2 1 2
2 2 2
2
1
2
| ( ) | ( ) | ( ) ( ) ( ) ( )
( ) ( ) ( ) ( )~ exp exp exp exp
2 2 2 2
1exp
2
M M K
v v u u
v
P x t P x t P x t P x ty y y P x t P x t
y x t y x t x t x t
y
taking and factoring out constants:
with
2 2
21 1
22 2
21 0
1( ) ( )
2
~ exp ( ) ( ) ,
M K
i i ii iu
TMv
i ii u
x t x t
K
My x t p x t dt p
T
Optimal dynamic motion estimation (3)
The optimal estimate is the trajectory x(t) which minimizes:
2 2
1 0
( ) ( )TM
i ii
TRACKING EFFORT
y x t p x t dt
This is a trajectory optimization problemThe solution is a cubic smoothing spline
Which is (apart from boundary conditions) equivalentto a double (forward & backward) 2nd order low pass Butterworth filter
Woltring, Adv Eng Softw 1986
General, state space formulation
2 2
1 0
( ) ( )TM
i ii
TRACKING EFFORT
J y g x t p u t dt
Find state trajectory x(t) and control trajectory u(t) that minimize:
Subject to the process model: ( , , ) 0f x x u
Observation model: (sensor fusion) ( )y g x v
Cost function J: derived from Gaussian v and u & maximizing P(H|E)
Solution methods for trajectory optimization:Shooting (repeated simulation while iterating u(t) )Multiple Shooting (Mombaur)Direct Collocation (Ackermann & van den Bogert, J Biomech 2010)
trajectory smoothingand load sharing between muscles52 + 52 < 102 + 02
Control / effort cost term
Probabilistic interpretation
0 1 2 30
0.2
0.4
0.6
0.8
1k=2k=3k=5k=10
cost
( )
~
ku
k
P u e
u
u/σ
P(u)
cost
max
k
u
Traditionally based on physics or physiology electric motors: heat ~ sum u2
muscles: fatigue ~ sum u3
Gaussian prior: minimize sum u2
generalized Gaussian: minimize sum uk
uniform: minimize max(u)
uniform prior probabilitygeneralized Gaussians
Implicit musculoskeletal dynamics
van den Bogert et al., Procedia IUTAM 2011
, 0f(x, x u)
asvq
x
generalized coordinates
generalized velocities
projected fiber lengths
muscle activation states
max
( ) ( , ) ( )
( ) ( ) ( ) ( ) cos
1( )
TM
SEE M
SEE M FL CE FV CE PEE CE
act deact
F f f f
T T
q v
LM q v B q v F L q sq
0
F L q s a L L L
u ua u a
CE
refCErefCE
LsLsL
/cossin22
q1
q3
q2
Landing movement in downhill skiing
Data: low resolution video 2 panning cameras manual digitization 3D calibration in each frame
Nachbauer et al., J Appl Biomech 1996
Model and optimizationProcess model: planar 9-DOF skeleton 9-segment flexible skis 16 muscles ski boot stiffness ski-ground contact & friction x = (x1,x2...x82)
Observation model:g(x) = (q1,q2...q9) + v
Heinrich et al. Scand J Med Sci Sports 2014
2 2
1 0
( ) ( )TM
i ii
TRACKING EFFORT
J y g x t p u t dt
Cost function:
Solution:
, 0f(x, x u)
Kinematics of the landing movement
Heinrich et al. Scand J Med Sci Sports 2014
trajectory optimization
video data
LEFT RIGHT
trajectory optimization preserves high frequencies at impact
Other variables
left knee right knee
Ground reaction force
musclecontrols (u)
knee ligament forces
left
right
left right
Introduction
24
Inertial Measurement Unit (IMU) 3D accelerometer, 3D rate gyro, 3D magnetometer less expensive than optical motion capture not restricted to laboratory environment
Kinematic analysis sensor orientation from Kalman filter human inverse kinematics (Roetenberg et al., 2009)
Inverse dynamic analysis and muscle forces position drift no registration with force plate possible GRF prediction method (Karatsidis et al., 2019)
inverse kinematics, inverse dynamics, static optimization complete kinematic data needed
$36 - www.sparkfun.com
$12,000 - www.xsens.com
Trajectory optimization approach
25
Model of musculoskeletal dynamics: Find periodic trajectories x(t) and u(t) to minimize:
Potential advantages simultaneous estimation of kinematics, forces, muscle states solutions are dynamically consistent (obey physics and muscle physiology) can track raw unfiltered sensor data can work with incomplete data (and overcomplete data)
f(x,x,u) = 0
Model
26
Musculoskeletal dynamics model planar, 9 DOF 16 Hil-based muscles 2 ground contact points on each foot 66 state variables x, 16 control inputs u
Sensor models
gyro:
accelerometer:
f(x,x,u) = 0
:s sg
van den Bogert, Read, Nigg, J Biomech 1996
Evaluation study (J Biomech, in press)
27
10 subjects, 6 speeds (0.9 - 4.9 m/s) 10 trials at each speed 7 IMUs (sacrum, lateral thigh & shank, dorsal foot) Optical motion capture (OMC), force plates 2D joint angles and moments (Winter, 2009)
Ensemble averaging of data 100 points in average gait cycle, m ± σ
Trajectory optimization Comparison IMU vs. OMC results correlation coefficients RMS differences
Sensor signal tracking (running, one subject)
28
accel accel gyro
soft tissue artifactsare rejected
impact peaks aretracked
rightthigh sensor
rightshank sensor
rightfoot sensor
Joint angles and moments (walking, one subject)
29
effect of walking speed is correctly estimated
angl
e (d
eg)
mom
ent
(Nm
)G
RF
(BW
)
Joint angles and moments (running, one subject)
30
ankle moment rises too early
angl
e (d
eg)
mom
ent
(Nm
)G
RF
(BW
)
Discussion
32
Excellent correlation between IMU and OMC kinematics/kinetics (ρ > 0.9) Estimates of muscle forces and contraction/activation states consistent with typical EMG metabolic energy cost can be calculated from muscle state trajectories
Limitations computation speed: ~50 min for each of the 60 optimizations ankle moment overestimation, likely due to rigid foot model 2D analysis muscle load sharing is guided by optimization objective
Future work
33
Fewer sensors: minimal sensor set?
More sensors: EMG, foot pressure
Self-calibration Real time 3D analysis Validation for clinical and sports applications
Top Related