Optimization Framework of Humanoid Walking Pattern for HumanMotion Retargeting

Shimpei Masuda1,2, Ko Ayusawa1 Eiichi Yoshida1,2

Abstract— In this paper, we propose a novel method forretargeting human movements including the steps taken by ahumanoid robot. For applications, such as wearable device eval-uation by a humanoid, it is necessary to reproduce the originalhuman motions as closely as possible and to generate motionsthat can be realized on a real robot without the robot losingits balance. The proposed method features the optimizationframework that integrates the walking pattern generation basedon the linear inverted pendulum model whose parameters areoptimized to ensure both human likeness and feasibility of thegenerated motions. The effectiveness of the retargeting methodis validated by experiments that reproducing the measuredhuman working motion on the humanoid HRP-4.

I. INTRODUCTION

Along with an increase in workers, we are experiencingan increasing demand for wearable assistive devices, suchas power-assisted suits for amplifying human strength. Todevelop and spread wearable assistive devices, we need amethod for quantitatively evaluating the performance of eachdevice. As a framework for obtaining quantitative evaluation,a method using a humanoid robot has been proposed [1]. Inthis framework, humanoid robot performs human motionswith and without an assistive device, and the internal load ismeasured from the internal sensors. Also, this frameworkis expected to solve the problem of motion repeatabilityand ethics. Nabeshima et al. have established standards inJapan for assistive devices that are attached to the waist [2].However, this evaluation framework has not been realizedfor assistive devices that support walking or other motionsthat involve taking steps. One reason is that it is difficult togenerate robot motions that mimic dynamic human motionssuch as walking, which involve changes in the foot place-ments. Furthermore, it is necessary to realize the generatedmotion under various disturbances and modeling errors.

Some studies have already reported the reproduction ofhuman motions (measured by motion capture systems) forlife-sized humanoid robots. Nakaoka et al. defined the taskprimitives and parameters for lower body movements andreproduced a measured Japanese dance on HRP-2 [3]. ForHRP-2, Ramos et al. succeeded in reproducing the dancemotion including the leg movements by considering theconsistency of the dynamics [4]. Miura et al. generateda turning motion for the humanoid HRP-4 by using the

*This work was partly supported by JSPS Grant-in-Aid for ScientificResearch (A) Number 17H00768, and by JSPS Grant-in-Aid for ScientificResearch (B) Number 18H03315.

S. Masuda and E.Yoshida are with 1University of Tsukuba, Japan. S.Masuda, K. Ayusawa and E. Yoshida are with 2CNRS-AIST JRL (JointRobotics Laboratory), UMI3218/RL, Tsukuba, Japan. Corresponding author:S. Masuda s1720858@s.tsukuba.ac.jp

Fig. 1. Overview of motion reproduction method

measured human motions [5]. Ishiguro et al. realized real-time motion reproduction as a master–slave system [6].Although these approaches can generate feasible motionsefficiently, they are not aimed at precise preservation ofmotion characteristics such as the angle trajectory of thespecific joint. When converting the captured human motionsinto the feasible one for the humanoids, several studies havebeen reported such as Dynamics Filter proposed by Yamaneet al[7]. Though the method can generate the motion whichhas high reproducibility and dynamically consistent, the lackof locomotion control leads instability of biped motion of areal humanoid robot.

In this paper, we propose a novel method of reproducingthe walking motions of humans by a humanoid robot whileconsidering the stability issue of locomotion toward thethe applications in the real robot. We introduce a retar-geting framework that integrates optimization and patterngeneration. We constructed a walking pattern generator thatoutputs walking motions based on the linear inverted pen-dulum model (LIPM) [8], whose parameters are optimizedto achieve the target human motions while satisfying boththe reproducibility and feasibility. The pattern generatedaccording to LIPM is compatible with stabilization controlusing LIPM [9]. Therefore, the combination leads the highperformance against disturbances in the real world. Accord-ing to similar concept, Kondo et al. optimized small gaitparameters by genetic algorithm and reproduced measuredhuman walking motion by using WABIAN-2 [10]. In ourmethod, many parameters including the foot-print and waistpose can be optimized in order to enhance the reproducibilityof original human motion. On the other hand, more than60 parameters need to be optimized. To solve the nonlinearproblem, the gradient of the cost function need to be com-puted accurately. Since the function including pattern gen-

2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids)Beijing, China, November 6-9, 2018

978-1-5386-7282-2/18/$31.00 ©2018 IEEE 725

erator has complicated structure, we applied the automaticdifferentiation (AD) technique. We also built a generator thatcan generate the motion algorithmically from the parametersto apply AD. The motion retargeting method was validatedby an experiment, which reproduced the measured humanworking motion on the humanoid HRP-4 [11].

II. RETARGETING USING OPTIMIZATION AND LIPMBASE PATTERN GENERATION

A. Overview

Figure 1 provides an overview of our retargeting system.The system takes a measured human motion clip as an inputand outputs the retargeted robot motion pattern which isdynamically consistent and stable. The system is mainlycomposed of a motion morphing part and an optimizationpart. The motion morphing part converts the human motioninto a robot motion. The converted motion is used as areference in latter part. Then, the optimization part then opti-mizes the parameters of the pattern generator based on LIPMto minimize the difference between the generated motionand the reference. Finally, the overall system generates themotion pattern with the acquired optimal parameters.

B. Motion morphing for generating reference robot motion

The body structure of a human actor and a humanoid robotare quite different. We employed the method proposed byAyusawa et al. [12] for scaling the body structure; however,this method only considers the kinematic feasibility. In themotion optimization part, the robot motion that was morphedfrom human motion is set as the reference.

C. Motion optimization using LIPM based pattern genera-tion

The direct optimization of motions that involve severalfootsteps is difficult, especially for following: 1) The floorcontact state is discrete information that is difficult to use inmathematical formulations. 2) Complex constraints are nec-essary, for example, for tracking the targeted zero-momentpoint (ZMP) trajectory. We address these problems by usingoptimization including the walking pattern generation basedon LIPM [8].

When the humanoid robot’s CoM moves in the horizontalplane and the angular momentum around CoM is zero, itsdynamics can be represented as a linear inverted pendulumas follows:

x =g

zc(x− p) (1)

where x is position of CoM; zc is the vertical height ofCoM; p is position of ZMP; and g is the gravity acceleration.When a footprint (the floor contact positions and timings) isgiven as the input of the generator, it can determine a ZMPtrajectory based on some assumptions. For example, ZMPis set at the center of the sole during a single supportingphase, and so on. Then, a CoM trajectory that tracks theZMP trajectory is calculated from (1). The walking patternis calculated from the foot trajectory determined from thefootprint and the waist and the upper body posture trajectory

that tracks the CoM trajectory [13]. We have extended thisLIPM-based walking pattern generator that can generatevarious walking motions.

Let qtarg and qgen be the generalized coordinates of therobot as follows:

qtarg = [ ptargbase Rtargbase θtarg ] (2)

qgen = [ pgenbase Rgenbase θgen ] (3)

where ptargbase and pgenbase are the position of baselink, Rtargbase and

Rgenbase are the rotation of the baselink, and θtarg and θgen

are the joint angle of whole body. At each time instancet1, t2, · · · , tNT

, let qtargt and qgen

t (1 ≤ t ≤ NT ) be thereference and the generated robot postures, respectively. Letthe generated motion pattern for the humanoid be Qgen(

∆=

[qgen1 · · · qgen

NT]) = PG(ϕ,C), where ϕ is the set of variable

parameters for pattern generation (such as zc in (1)), and Cis the set of constant parameters. Assuming that the patterngenerator has sufficient DoF to reproduce the target motion,the motion reproduction can be expressed as an optimizationproblem of the parameters of PG as follows:

minϕ

f(PG(ϕ,C),Qtarg)(4)

where Qtarg ∆= [qtarg

1 · · · qtargNT

] is the reference motion, andf is the error function for calculating the difference betweenQgen and Qtarg . In this paper, we used the sum of squarederror of the angles of the hip and the knee joints as objectivefunction. The equality or inequality constraints are handledas the penalty terms in the objective function. The nonlinearoptimization problem can be solved by the Quasi-Newtonmethod or another such method, but a gradient is required.We employed AD [14] to compute the accurate gradientof the whole system from motion pattern generation tomotion evaluation. Though AD requires time and effort froma naive software implementation, it can quickly computethe derivatives with high accuracy in contrast to numericaldifferentiation.

The details of pattern generator improvements will bedescribed in Section III.

III. PATTERN GENERATOR FOR MOTION OPTIMIZATIONWITH AD

A. Overview

Figure 2 shows a diagram for the implementation of apattern generator. The steps in the pattern generator processare as follows:

1) Determine the reference ZMP trajectory and thefoot motion.

2) Calculate a baselink posture trajectory and generatethe whole-body motion once.

3) Compensate for the ZMP error by modifying thebaselink trajectory and output the motion pattern.

726

LIPM ZMP tracking

𝜙𝑹𝑏𝑎𝑠𝑒 , 𝜙𝒑𝑏𝑎𝑠𝑒𝑎𝑑𝑑

𝒑𝑧𝑚𝑝

Reference ZMP generation

𝜙𝒕𝑠, 𝜙𝒑𝑅𝑐 , 𝜙𝒑𝐿𝑐𝐶𝑺𝑠𝑢𝑝

𝒑𝑧𝑚𝑝𝑟𝑒𝑓

, 𝑍𝑐

Foot-motion generation

𝒑𝑅𝑓,𝒑𝐿𝑓

𝑹𝑅𝑓 , 𝑹𝐿𝑓

Baselink posture calculation

𝐶𝜽𝑢𝑝𝑝𝑒𝑟

𝜙𝑍𝑐

Output(Motion pattern)

𝒑𝑏𝑎𝑠𝑒 𝑹𝑏𝑎𝑠𝑒

𝑸

𝑸𝑔𝑒𝑛

Leg inverse kinematics

ZMP calculation

Whole body motion generation

Baselink traj. modification ( Eq.(14)-(16) )

ZMP compensation

𝜙𝑹𝑅𝑐, 𝜙𝑹𝐿𝑐𝜙𝑠𝑤𝑖𝑛𝑔

Whole-body motion generation

𝑃𝐺

(Section III-B)

(Section III-C)

(Section III-D)

(Section III-E)

(Section III-F)

(Section III-G)

Fig. 2. Implementation of the pattern generator PG

B. Reference ZMP generation

First, we define CSsupas a sequence of the support state in

the whole motion. It is a constant parameter and is measuredfrom the reference motion. Also ϕts

= [ts,1 · · · ts,Nts] is

defined as the change timings of the support-state sequence.ϕpRc

= [pRc,1 · · · pRc,NRc] and ϕpLc

= [pLc,1 · · · pLc,NLc]

are defined as the positions for each floor contact of eachfoot (R indicates right, and L indicates left). Nts , NRc

and NLc are the number of each parameter and is deter-mined from CSsup

. The reference ZMP trajectory prefzmp

∆=

[prefzmp,1 · · · prefzmp,NT

] is generated from these parameters asfollows:

prefzmp = fref

zmp(CSsup, ϕts

, ϕpRc, ϕpLc

) (5)

The trajectory is generated by the interpolation of the viapoint determined according to footprint parameters: ϕts

,ϕpRc

and ϕpLc. The interpolation of the two adjacent via

points can be formulated as follows:

a = (1− µ)a0 + µa1 (6)µ = (1− cos(πτ))/2 (7)

where τ is the normalized time between the time instancescorresponding to the two adjacent via points, and a is theinterpolated value from a0 and a1 at τ . This interpolation isemployed through all via points to avoid overshooting andto set the velocity to zero near the via point. Figure 3 showsthe implementation of (5) for the X-axis (the sagittal axis)and the Y-axis (the frontal axis);this figure also gives anexample of walking four steps by stepping out with the rightfoot. In Figure 3, the red points specify the via points, andthe green line is interpolated as the ZMP trajectory. “D” inCSsup

indicates the double support phase and “R” and “L”indicate the single support phase with the right foot or theleft foot. We used the constant transition time of ZMP in

the first swing and the last landing time to/from the adjacentZMP reference was experimentally set to tb = 0.1 s.

𝑡𝑠,1 𝑡𝑠,2 𝑡𝑠,3 𝑡𝑠,4 𝑡𝑠,5 𝑡𝑠,6 𝑡𝑠,7 𝑡𝑠,80 𝑡𝑒𝑛𝑑

𝑣1 𝑣2 𝑣3 𝑣4

𝑣5 𝑣6

𝑣7 𝑣8 𝑣9 𝑣10𝑣11 𝑣12

𝐷 𝐿 𝐷 𝑅 𝐷 𝐿 𝐷 𝑅 𝐷

𝑡𝑏 𝑡𝑏

𝜙𝒑𝑅𝑐 = [ 𝑝𝑅𝑐,1 𝑝𝑅𝑐,2 𝑝𝑅𝑐,3 ]

𝜙𝒕𝑠 = [ 𝑡𝑠,1 , ⋯ , 𝑡𝑠,8 ]

𝜙𝒑𝐿𝑐 = [ 𝑝𝐿𝑐,1 𝑝𝐿𝑐,2 𝑝𝐿𝑐,3 ]𝐶𝑺𝑠𝑢𝑝 = [ 𝐷, 𝐿, 𝐷, 𝑅, 𝐷, 𝐿, 𝐷, 𝑅, 𝐷 ]

( D : Double support, R/L : Right/Left foot )

Subjects :

𝑣1, 𝑣2 =𝑝𝑅𝑐,1 + 𝑝𝐿𝑐,1

2𝑣3, 𝑣4 = 𝑝𝐿𝑐,1

𝑣5, 𝑣6 = 𝑝𝑅𝑐,2

𝑣7, 𝑣8 = 𝑝𝐿𝑐,2

𝑣9, 𝑣10 = 𝑝𝑅𝑐,3

𝑣11, 𝑣12 =𝑝𝑅𝑐,3 + 𝑝𝐿𝑐,3

2

𝑣1 𝑣2

𝑣3 𝑣4

𝑣5 𝑣6

𝑣7 𝑣8

𝑣9 𝑣10

𝑣11 𝑣12

X

Y

𝒑𝑧𝑚𝑝,𝑥𝑟𝑒𝑓

𝒑𝑧𝑚𝑝,𝑦𝑟𝑒𝑓

Fig. 3. Procedure of reference ZMP trajectory generation.

C. Foot-motion generation

The foot motion is determined in a manner similarto the ZMP trajectory, as given in the previous sub-section. Let ϕRRc

= [RRc,1 · · ·RRc,NRc] and ϕRLc

=[RLc,1 · · ·RLc,NLc

] be the rotation of the right foot and theleft foot, respectively. The foot-motion generation definedby the function ffoot, to compute pRf ,pLf ,RRf ,RLf is asfollows:

pRf ,RRf ,pLf ,RLf =

ffoot(ϕSsup , ϕts , ϕpRc

, ϕRRc

, ϕpLc

, ϕRLc

, ϕswing)(8)

where ϕswing is the height of the swing foot during thewalking motion at half of the swing period. Figure 4 showsthe implementation of the foot-motion generation given in

727

(8) for the X-axis (sagittal axis) and the Z-axis (verticalaxis). Start timing of the swing and the landing timing areextracted from ϕts

with reference to CSsup so that the timingof each via point is determined. The trajectory is generatedby the interpolation formula given in (7). The procedureimplemented on the Y-axis (the frontal axis) and the rotationtrajectory were also same as that for the X-axis.

𝑣1 𝑣2

𝑣3 𝑣4

𝑣5 𝑣6

𝑝𝑅𝑐,1

𝑝𝑅𝑐,2

𝑝𝑅𝑐,3

𝜙𝒑𝑅𝑐 = [ 𝑝𝑅𝑐,1 𝑝𝑅𝑐,2 𝑝𝑅𝑐,3 ]

𝜙𝒕𝑠 = [ 𝑡𝑠,1 𝑡𝑠,2 𝑡𝑠,3 𝑡𝑠,4 𝑡𝑠,5 𝑡𝑠,6 𝑡𝑠,7 𝑡𝑠,8 ]

𝐶𝑺𝑠𝑢𝑝 = [ 𝐷 𝐿 𝐷 𝑅 𝐷 𝐿 𝐷 𝑅 𝐷 ]

𝑡𝑒𝑛𝑑𝑡𝑠,10

Right foot position X

X

Subjects :

𝑡𝑠,2 𝑡𝑠,5 𝑡𝑠,6

Right foot position Z

𝜙𝑠𝑤𝑖𝑛𝑔

Z

𝑡𝑠,5 + 𝑡𝑠,6 /2

0

𝑡𝑠,1 + 𝑡𝑠,2 /2

𝑣1 𝑣2

𝑣3

𝑣4 𝑣5

𝑣6

𝑣7 𝑣8

swing swing

𝒑𝑅𝑓,𝑥

𝒑𝑅𝑓,𝑧

𝜙𝑠𝑤𝑖𝑛𝑔

Fig. 4. Procedure for foot-motion generation.

D. LIPM ZMP tracking

In each sagittal and frontal plane, the CoM trajectorytracking the ZMP trajectory is obtained from the discretizedLIPM as follows [15]:

xcom = A−1xzmp (9)

A, the NT ×NT matrix, is defined as follows:

A =

a+ b c 0

a b c. . .. . .. . . a b c

0 a b+ c

a = −zc/(g∆t2)

b = −2zc/(g∆t2) + 1

c = −zc/(g∆t2)

zc = ϕzc

In this implementation, the CoM position is assumed to beclose to the position of the waist link (i.e. the baselink). Thetrajectory of the baselink is computed as follows:

pbase,x = A−1prefzmp,x (10)

pbase,y = A−1prefzmp,y (11)

pbase,z = ϕzc (12)

E. Baselink posture calculation

This component adds additional vertical movement topbase as follows:

pbase,z = ϕzc + fspline(ϕpaddbase

) (13)

where the function fspline calculates the trajectory by uni-form B-spline of order 3, and the argument is the controlpoint of the spline. Let Rbase, the rotational trajectory ofthe baselink, be given as follows:

Rbase = Rrpy(frpyspline(ϕRbase

)) (14)

where frpyspline calculates three spline trajectories in the same

as (13), and function Rrpy(·) calculates a rotation matrixwith given roll pitch yaw angles.

F. Whole-body motion generation

The joint angles of each leg are calculated from thebaselink posture trajectory and each foot trajectory by usinginverse kinematics (IK). Here, analytical IK is possiblebecause each leg of HRP-4 has only six DoFs and theaxis of three DoFs on the waist are orthogonal. Let θRleg

and θLleg be the calculated joint angles of each leg frompRf ,RRf ,pLf ,RLf , pbase and Rbase. When given a po-sitional relationship between waist and foot that can not berealized, we clip the norm of foot-baselink and the error isconsidered in the optimization process.

In this implementation, the joint angle trajectory of theupper body is set to the reference motion received as theparameter Cθupper

. Here, Q∆= [q1 · · · qNT

] is defined asthe motion pattern , where q = [ pbase Rbase θ ] andθ = [ θRleg θLleg θupper ].

After computing the trajectory generalized coordinate,ZMP can be computed by the whole-body dynamics, wherethe generalized velocity and acceleration are computed by thenumerical differentiation of Q. Let pzmp be the computedZMP trajectory.

G. ZMP compensation

Because of the assumptions made for LIPM and the useof CoM as the baselink, the result of pzmp can have an errorfrom the reference pref

zmp. To compensate for the error, thebaselink trajectory pbase is modified according to the ZMPerror as follows:

∆pzmp = pzmp − prefzmp (15)

∆pbase,x = A−1∆pzmp,x (16)

∆pbase,y = A−1∆pzmp,y (17)

where A is the same matrix in (9). The modified baselinkposition trajectory pgen

base is pgenbase = pbase +∆pbase.

Finally, the output motion pattern Qgen is calculated fromthe modified pgen

base by the whole-body motion generatordescribed in Section III-F.

IV. EXPERIMENTS

This section presents the results of the experiment to verifythe effectiveness of the proposed system.

728

A. Experiment setupThe left side of Fig.5 shows the human-sized humanoid

robot HRP-4. The geometric parameter designs, such aslink lengths, were based on the average Japanese female;the height and weight of HRP-4 is almost 155 cm and 40kg, respectively. The right side of Fig.5 shows the jointconfiguration of HRP-4. The total number of DoFs of therobot is 37. Joints were added to the toes and to the roll axisof the waist link from the original HRP-4. In this experiment,the toe joints and the hand joints were fixed and unused.

While executing the motion pattern, the joint positioncontroller at each joint tracks the joint angle trajectory.The robot is affected by various disturbances (such as adeformation of some rubbers for shock absorbing on theankle) and some modeling errors; therefore the stabilizationcontroller is needed. For the stabilization controller, we usedan implementation of the LIPM-based stabilization controlmethod that was proposed by Kajita et al. [9].

Fig. 5. Overview of HRP-4 (left) and joint configuration (right).

B. Experiment for working motion retargetingExperiments have been conducted by assuming the case

of tasks requiring devices wearable on the lower body. Weselected the inspection task in a half-sitting posture as thetarget motion and measured it by motion capture system(Motion Analysis). The motion is approximately for 6 s andinvolved four steps. Snapshots of the captured motion areshown at the top of Fig.6. The whole-body motion Qtarg

was generated as described in Section II-B.One of our goal is to evaluate the assistive device that

performs according to the joint angles of the lower body. Inthis paper, we used the error of the angles of the hip jointsand the knee joints as evaluation that to be minimized. Theoptimization problem is therefore formulated as follows:

minϕ

1

NT

NT∑t=1

( ||θtarghip,t − θgen

hip,t||2

+ ||θtargknee,t − θgenknee,t||2

+ ωcom||ϕzc − pcom,z,t||2

+ ωIK||pfoot,t − p∗foot,t||2)

+ ωfck

∑k

||ϕinitfc,k − ϕfc,k||2

subject to ϕmin ≤ ϕ ≤ ϕmax

(18)

where θgenhip,t are the three joint angles of the each hip joint

extracted from θgenRleg,t and θgenLleg,t, and θgenknee,t is the kneejoint angle. In the problem (18), the constraint conditions ex-cluding the boundary constraint are considered as the penaltyterm. Also, pcom = [pcom,1 · · · pcom,NT

] is the CoM trajec-tory of the generated motion, and p∗foot,t (foot = Rf,Lf)is the actual foot position calculated by forward kinematicsof the generated motion. The weights ωcom and ωIK in thecost function in (18) are for the vertical CoM motion andthe IK error, respectively, and are empirically set to 100 and2000, respectively. ωfc (fc = ts,pRc,pLc,RRc,RLc) isthe weight of the penalty term which constraints the changeof the parameter relating to the foot-print to the adjustment,and are set to ωfck = 0.4 for ϕts

and ωfck = 4.0 forother foot-print parameters. The parameters to be optimizedare ϕ = {ϕzc , ϕpadd

base, ϕRbase

, ϕts, ϕpRc

, ϕpLc, ϕRRc

, ϕRLc},

and ϕRbaseonly effects yaw axis rotation. The boundary

constraints are set as follows: 0.5 ≤ ϕzc ≤ 1.0, −0.2 ≤ϕpadd

base,i≤ 0.2 [m], and −π/2 ≤ ϕRbase,i

≤ π/2 [rad].The foot-print parameters ϕts , ϕpRc

, ϕpLc, ϕRRc

, ϕRLcare

initialized to the values measured from reference motionusing the method proposed by Miura et al. [5]; and ϕswing

is set to 0.06 m. We used the efficient “L-BFGS-B” method[16] for optimization.

C. Results and discussion

At first, we analyzed the relationship between the costfunction and the number of control points of the two B-splinetrajectories. Table.I shows the convergence cost values whenthe set of the target parameter of optimization is changed, andnumbers in parentheses are the number of control points ofthe B-spline trajectory. We confirmed that the error decreasedaccording to the number of control points. According to

TABLE ICONVERGENCE ERROR VALUES AND THE SET OF OPTIMIZED

PARAMETERS.

Optimized parameters ϕ Total dims Convergence value

zc,paddbase(5),Rbase(5) 11 7.501× 10�2

zc,paddbase(10),Rbase(10) 21 7.341× 10�2

zc,paddbase(20),Rbase(20) 41 7.190× 10�2

zc,paddbase(20),Rbase(20),

ts,pRc,pLc,RRc,RLc67 6.868× 10�2

the analysis, the number of control points of each splinetrajectory was finally set to 20, as a result of optimizing theentire 67 dimensional parameters.

To see whether the optimized solution is globally optimal,we tested the four initial values including ϕinit = ϕmin,ϕinit = ϕmax and ϕinit = (ϕmax + ϕmin)/2. The varianceof the final objective function among 4 cases is less than2× 10−10. This result can be regarded as converging to theneighborhood of the global minimum.

Snapshots of the motion that realized on HRP-4 are shownin Fig.6. As can be seen in the figure, the retargeted motionwas performed stably. The optimized CoM trajectory, thewaist trajectory, ϕzc , and CoM trajectory of the target motion

729

Fig. 6. Original human motion (top) and reproduced motion on HRP-4 (bottom). The motion includes four steps, and the timing of snapshots are thestart and end, and each leg swinging.

are shown in Fig.7. The waist vertical movement reproducedthe motion feature whereas the CoM trajectory tracks ϕzc ,which validated the effectiveness of the proposed method.It should be noted that our approach tends to modify CoMtrajectory according to the dynamics of LIPM. Therefore,the motion with a large perturbation of CoM is difficultto be retargeted by the proposed method, which will beinvestigated in our future work.

0 1 2 3 4 5 6time [sec]

0.65

0.70

0.75

z [m

]

optimized zcoptimized CoMoptimized waisttarget motion CoM

Fig. 7. CoM trajectory, baselink position trajectory and zc of the generatedmotion.

Figure 8 shows the joint angle trajectory of the hip yaw,hip roll, hip pitch (θhip), and knee joint (θknee) of thereference motion and generated motion. The hip yaw jointand the knee joint were reproduced with high accuracy (inthe hip yaw joint, a maximum error is 5◦ or less); however,there is room for improving the results about other joints.Assuming that the optimization of the ϕRbase

improved thehip yaw joint, other joints will also improve by optimizing

the other axis rotation of the baselink.

V. CONCLUSIONS

In this paper, we presented a novel retargeting methodfor human motions which included the steps taken by ahumanoid robot to cope with optimization under the con-straints of the complex dynamics and instability induced bythe changes in foot placements. We addressed this problemby optimizing the parameters of the LIPM-based walkingpattern generator, and minimizing the difference from thereference motion. The implemented pattern generator cangenerate various motions, and the gradient of all inputparameters for the output motion can be computed by the ADtechnique. Thanks to the efficient and accurate gradient cal-culation, we could optimize many parameters and reproducethe detailed motion features by determining 67 parametersincluding the parameter corresponding to the foot-print andthe baselink posture trajectory.

The effectiveness of our method was verified by a real-robot experiment supposing a practical inspection task. Theproposed method was applied to the motion of working in ahalf-siting posture, and the reproduced motion was realizedby the humanoid HRP-4.

We are also aware of several limitations to our method.The first limitation is related to the stabilizing controlleron a real robot. We have constructed a method assuminga stabilizing controller based on the LIPM, but this is arestriction when considering reproduction that uses a widerrange of motions. It is necessary to simultaneously improve

730

0

20

40Hi

p ya

w [d

eg] optimized

target

−20

0

20

Hip

roll

[deg

]

−60

−40

−20

Hip

pitc

h [d

eg]

0 1 2 3 4 5 6time [sec]

60

80

100

Knee

[deg

]

(a) Right leg

20

40

60

Hip

yaw

[deg

]

optimizedtarget

−20

0

20

Hip

roll

[deg

]

−40

−20

Hip

pitc

h [d

eg]

0 1 2 3 4 5 6time [sec]

60

80

100

Knee

[deg

]

(b) Left leg

Fig. 8. Comparison of the leg joint angle trajectory of the generated motionwith reference motion. In each figure shows the hip yaw joint, hip roll joint,hip pitch joint, and knee joint in order from the top.

both the stabilization controller and the retargeting method.The second issue is evaluating the reproducibility of theproposed retargeting method. In this method, it is possibleto discuss how much the error decreases with respect tothe reference robot motion, but it is still difficult to arguehow close it is to the original human motion as a whole;it will be needed when applying this method for evaluationof assistive devices in the future. To deal with this problem,it is necessary to extend our method for characterizing thequantitative features of human likeness in terms of differentaspects of kinematics and dynamics, as discussed in [12].

REFERENCES

[1] K. Ayusawa, E. Yoshida, Y. Imamura, and T. Tanaka, “New evaluationframework for human-assistive devices based on humanoid robotics,”Advanced Robotics, vol. 30, no. 8, pp. 519–534, 2016.

[2] C. Nabeshima, K. Ayusawa, C. Hochberg, and E. Yoshida, “Stan-dard performance test of wearable robots for lumbar support,” IEEERobotics and Automation Letters, vol. 3, no. 3, pp. 2182–2189, 2018.

[3] S. Nakaoka, A. Nakazawa, F. Kanehiro, K. Kaneko, M. Morisawa, andK. Ikeuchi, “Task model of lower body motion for a biped humanoidrobot to imitate human dances,” in 2005 IEEE/RSJ InternationalConference on Intelligent Robots and Systems, 2005, pp. 3157–3162.

[4] O. E. Ramos, N. Mansard, O. Stasse, C. Benazeth, S. Hak, and L. Saab,“Dancing humanoid robots: Systematic use of osid to compute dynam-ically consistent movements following a motion capture pattern,” IEEERobotics Automation Magazine, vol. 22, no. 4, pp. 16–26, 2015.

[5] K. Miura, M. Morisawa, S. Nakaoka, F. Kanehiro, K. Harada,K. Kaneko, and S. Kajita, “Robot motion remix based on motioncapture data towards human-like locomotion of humanoid robots,” in2009 9th IEEE-RAS International Conference on Humanoid Robots,2009, pp. 596–603.

[6] Y. Ishiguro, K. Kojima, F. Sugai, S. Nozawa, Y. Kakiuchi, K. Okada,and M. Inaba, “Bipedal oriented whole body master-slave systemfor dynamic secured locomotion with lip safety constraints,” in 2017IEEE/RSJ International Conference on Intelligent Robots and Systems,2017, pp. 376–382.

[7] K. Yamane and Y. Nakamura, “Dynamics filter-concept and im-plementation of online motion generator for human figures,” IEEETransactions on Robotics and Automation, vol. 19, no. 3, pp. 421–432, jun 2003.

[8] S. Kajita, F. Kanehiro, K. Kaneko, K. Yokoi, and H. Hirukawa,“The 3d linear inverted pendulum mode: a simple modeling for abiped walking pattern generation,” in Proceedings 2001 IEEE/RSJInternational Conference on Intelligent Robots and Systems, vol. 1,2001, pp. 239–246.

[9] S. Kajita, M. Morisawa, K. Miura, S. Nakaoka, K. Harada, K. Kaneko,F. Kanehiro, and K. Yokoi, “Biped walking stabilization based onlinear inverted pendulum tracking,” in 2010 IEEE/RSJ InternationalConference on Intelligent Robots and Systems, 2010, pp. 4489–4496.

[10] H. Kondo, A. Morishima, Y. Ogura, S. Momoki, J. Shimizu,H. ok Lim, and A. Takanishi, “Algorithm of pattern generation formimicking disabled person’s gait,” in 2008 2nd IEEE RAS EMBS In-ternational Conference on Biomedical Robotics and Biomechatronics,2008, pp. 724–729.

[11] K. Kaneko, F. Kanehiro, M. Morisawa, K. Akachi, G. Miyamori,A. Hayashi, and N. Kanehira, “Humanoid robot HRP-4 - humanoidrobotics platform with lightweight and slim body,” in 2011 IEEE/RSJInternational Conference on Intelligent Robots and Systems, 2011, pp.4400–4407.

[12] K. Ayusawa and E. Yoshida, “Motion retargeting for humanoid robotsbased on simultaneous morphing parameter identification and motionoptimization,” IEEE Transactions on Robotics, vol. 33, no. 6, pp.1343–1357, 2017.

[13] S. Kajita, F. Kanehiro, K. Kaneko, K. Fujiwara, K. Harada, K. Yokoi,and H. Hirukawa, “Biped walking pattern generation by using previewcontrol of zero-moment point,” in 2003 IEEE International Conferenceon Robotics and Automation, vol. 2, 2003, pp. 1620–1626 vol.2.

[14] R. Neidinger, “Introduction to automatic differentiation and matlabobject-oriented programming,” SIAM Review, vol. 52, no. 3, pp. 545–563, 2010.

[15] K. Nishiwaki, S. Kagami, Y. Kuniyoshi, M. Inaba, and H. Inoue,“Online generation of humanoid walking motion based on a fastgeneration method of motion pattern that follows desired zmp,” inIEEE/RSJ International Conference on Intelligent Robots and Systems,vol. 3, 2002, pp. 2684–2689.

[16] R. Byrd, P. Lu, J. Nocedal, and C. Zhu, “A limited memory algorithmfor bound constrained optimization,” SIAM Journal on ScientificComputing, vol. 16, no. 5, pp. 1190–1208, 1995.

731