Intel Serv RoboticsDOI 10.1007/s11370-013-0132-8

ORIGINAL RESEARCH

A multitasking architecture for humanoid robot programming

Jaeyoung Han · Frank Chongwoo Park

Received: 28 April 2013 / Accepted: 26 May 2013© Springer-Verlag Berlin Heidelberg 2013

Abstract We propose a hierarchical, three-tiered motionprogramming architecture for humanoid robots that allowsfor the prioritized coordination of multiple tasks while tak-ing into account the dynamics and other physics-based con-straints that underlie typical humanoid robot tasks. We firstintroduce a data structure for generic humanoid robots basedon a general description of what constitutes a humanoidthat is workable and practical from a programming perspec-tive, without overly restricting the diversity of humanoiddesigns. For the low-level language we develop an exten-sion of Brockett’s motion description language (MDL) thatallows for the prioritized coordination of multiple tasks tak-ing into account the dynamics-based requirements of typ-ical humanoid manipulation tasks. The extended multitask-ing motion description language (MDLm) inherits the advan-tages of the original MDL, while making use of change ofcoordinates and the null space control formalism of Sentisand Khatib (Int J Humanoid Robot 2(4):505–518, 2005). Wealso develop a high-level language consisting of pre-definedmotion primitives that constitute a vocabulary for generat-ing more complex free-space motions and object manipula-tion tasks. These occupational tasks and free-space motionsare derived based on methods and principles for measuringhuman task performance Drumwright (The Task Matrix: a

This research was supported in part by the Center for AdvancedIntelligent Manipulation, KIST-CIR, and SNU-IAMD.

J. HanConvergence R&D Lab, LG Electronics, 19 Yangjae-daero 11gil,Seocho-gu, Seoul 137-130, Koreae-mail: jaeyoung.han@lge.com

F. C. Park (B)School of Mechanical and Aerospace Engineering, Seoul NationalUniversity, 1 Gwanak-ro, Gwanak-gu, Seoul 151-744, Koreae-mail: fcp@snu.ac.kr

robot-independent framework for programming humanoids.Ph.D dissertation, University of Southern California, 2007),Karger and Bayha (Engineered work measurement. Indus-trial Press, New York, 1965). The structure of the high levellanguage is also inspired in part by well-established notationfor dance choreography Huang and Hudak (Dance: a declar-ative language for the control of humanoid robots, 2003)Hutchinson (Labanotation or Kinetography Laban: system ofanalyzing and recording movement. Oxford University Press,New York, 1972). In the resulting architecture, high-levelmotion primitives and MDLm are integrated in a balancedand consistent manner that allows for flexible and intuitiveprogramming in an efficient manner. A case study is offeredto illustrate our architecture.

Keywords Robot programming · Software architecture ·Motion description language · Humanoid · Multitasking

1 Introduction

As robots are asked to perform increasingly complex tasks inunstructured environments, programming the robot in a sim-ple and intuitive way without sacrificing efficiency becomesan even greater challenge. The choice of architecture—herewe use the word to refer to the modular, hierarchical wayin which the software to control and program the robot isdesigned—has a deep and long-lasting impact on program-ming ease and efficiency. Initial choices can persist for years,and making even small changes to the architecture can leadto major rewrites of existing code. The survey paper [17]describes the myriad issues involved in design of robot pro-gramming architectures.

Robot programming systems typically are designed ina hierarchical fashion such that at one end is a high-level

123

Intel Serv Robotics

motion description language that is expressive and device-independent, while at the other end is a set of robot-dependentcontrol laws. In [1,9] a high-level, three-tiered softwarearchitecture for general robots is proposed, consisting of ahigh-level planning layer, a low-level reactive control layer,and an intermediate “deliberation” layer that translates thehigh-level commands into low-level behaviors, and also mon-itors the execution of single behaviors. Although the frame-work is quite general and offers useful guiding principles inhigh-level architectural design, because robots by nature areso diverse in their range of designs and applications, thesehigh-level principles do not offer much practical guidancein, e.g., specifying the details of the layers and the interfacesbetween them.

Work that attempts to address these more practical issueshas instead focused on specifying details at the highest andlowest levels. Among high-level languages, the Task Matrix[7,8] framework offers hardware-independent task-level pro-gramming, particularly with humanoid platforms in mind.Task Matrix assumes that a common skill set directly con-trolling the robot hardware device has been implemented foreach robot and provides a planning framework in a robot-independent manner. While most task-level or behavior-based programming languages tend to use an ad hoc setof primitive motions, Task Matrix proposes a primitive taskvocabulary inspired by a well-established human work mea-surement system [14].

Among low-level languages, Brockett [5,6] has proposeda low-level motion description language (MDL) that pro-vides a formal basis for robot programming using behaviors,and at the same time permits incorporation of kinematic anddynamic models of robots given in the form of differentialequations. The basic atom in MDL consists of triples of theform (u, k, T ) that are applied to the governing robot stateequations: u is the feedforward control input, k is the feed-back control law, and T is the time duration over which the uand k is to be applied. Various extensions of MDL have beenproposed, e.g., MDLe [19], which allows for descriptions oftriggers generated by a sensor, and MDLn [20], devised tocoordinate multi-agent systems on networks. The implemen-tation details of MDL and its extensions, have mostly beenexamined for the case of simple wheeled mobile robots withno manipulation capabilities. Moreover, current versions ofMDL are limited in their ability to describe and execute mul-tiple tasks having priorities.

An issue of increasing prominence in robot software archi-tecture design, and one that has not been explicitly addressedin the existing literature, is the need to manage multiplebehaviors concurrently. Current architectures do account for,e.g., mobile robots navigating in unstructured environmentswhile simultaneously collecting various sensor informationabout the environment and avoiding obstacles; [1] proposes athree-layered architecture consisting of a decision, execution

control, and functional level, while [18] is similarly dividedinto a deliberate layer, a task execution layer, and a reactivelayer. Also, architectures like the Cognitive Robot AbstractMachine (CRAM) of Beetz et al. [3] offer a high-level sym-bolic language for coordinating multiple tasks (for example,the “in parallel do” construct of CRAM executes a set ofsubplans in parallel), but do not specify the structure of themiddle and lower layers, or how the interfaces between thelayers should be designed.

These vertical integration issues become considerablymore involved for humanoid robots. Consider, for example, ahumanoid robot traversing an obstacle-cluttered environmentwhile grasping a heavy object with both hands; not only mustthe robot maintain postural balance while avoiding obstacles,but in the event that the robot senses it is about to lose balance,it should a priori know which task to abort (“stop walking” or“drop the object”). While existing high-level languages mayoffer mechanisms for describing prioritized concurrent tasksat a task level, how these commands should be translated andexecuted at the middle and lower layers—in most humanoidrobot tasks the dynamics needs to be accounted for at alltimes—is a difficult task that has yet to be addressed in anexplicit manner.

In our view, a substantial improvement of this situation isnot possible without (i) first restricting the class of robots andtasks for which the architecture is intended and (ii) building anew vertically integrated environment. Attempting to build ageneral architecture that encompasses all types of robots, andperforming the most general of tasks, is a fruitless exercisethat more often than not will lead to an inefficient architectureof limited practical use.

With these constraints in mind, we propose a hierarchi-cal motion programming architecture, primarily intended forhumanoid robots, that allows for the prioritized coordina-tion of multiple tasks while taking into account the dynam-ics and other physics-based constraints that underlie typ-ical humanoid robot tasks. The architecture is verticallyintegrated, from a high-level task programming languageto a low-level reactive motion control layer, and designedwith rapid prototyping and reusable motion programs inmind. The architecture must be expressive enough to pro-duce robot behaviors similar to those occurring in everydayhuman life, beyond simple kinematic motions that can beexecuted independently, e.g., raise one arm while turning thehead.

Toward this end, we first introduce a data structure forgeneric humanoid robots based on a general description ofwhat constitutes a humanoid that is workable and practicalfrom a programming perspective, without overly restrictingthe diversity of humanoid designs. This will lead us to aminimum set of requirements on what constitutes a humanoidrobot. We call the constructed abstract humanoid model ahumanoid proxy,

123

Intel Serv Robotics

As a second contribution, for the low-level languagewe develop an extension of Brockett’s MDL that allowsfor the prioritized coordination of multiple tasks takinginto account the dynamics-based requirements of typicalhumanoid manipulation tasks. The extended multitaskingmotion description language (MDLm) inherits the advan-tages of the original MDL, while making use of change ofcoordinates and the null space control formalism of [27] toproperly account for kinematic constraints and the prioritiza-tion of tasks. It should be noted that MDLm is not restrictedonly to humanoids, but is applicable to a wider class of artic-ulated robots.

As a third contribution, based on the humanoid proxy,we develop a high-level language consisting of pre-definedmotion primitives that then constitute a vocabulary for gener-ating more complex free-space motions and object manipula-tion tasks. These occupational tasks and free-space motionsare derived based on well-established methods and principlesfor measuring human task performance [7,14]. The struc-ture of the high-level language is also inspired in part bywell-established notation for dance choreography [12,13]. Inthe resulting architecture, high-level motion primitives andMDLm are integrated in a balanced and consistent mannerthat allows for flexible and intuitive programming in an effi-cient manner.

The remainder of the paper is organized as follows. InSect. 2 we describe a data structure for our humanoid proxy,followed by our high-level task programming language inSect. 3 and the multitasking motion description language(MDLm) in Sect. 4. Section 5 describes a number of imple-mentation case studies using our architecture. We concludein Sect. 6 with directions for extensions of our frameworkto include, e.g., grasp planning, and the possibility of staticanalysis and global program verification.

2 Humanoid modeling and abstraction

The meaning of ‘humanoid’ is usually intuitively understoodin a way such that it does not seem essential to have certainguidelines or standards for design. There is little if any litera-ture that contains more detailed descriptions of the functions,equipment, and structure that a humanoid should possessother than having an appearance resembling that of a human.In this section, we develop a more concrete set of criteria onwhat constitutes a humanoid that is workable and practicalfrom a programming perspective without overly restrictingthe diversity of humanoid designs.

A distinguishing feature of humanoid robots is that theypossess enough degrees of freedom in the lower body to per-form bipedal locomotion. Humanoids designed to emulatehuman walking further possess extra DOFs in the foot as wellas waist. Existing humanoid arms typically have six or seven

Fig. 1 An animated humanoid character (left), highlighted extremitiesand trunk (center), schematic model (right)

DOFs, and the hands are capable of grasping and releasingan object with as few as one DOF. Humanoid heads typi-cally have two DOFs (pan and tilt) at a minimum to adjust itsgaze. Among the sensors possessed by a typical humanoid,a gyrometer, accelerometer, IMU, and/or inclinometer areessential for balancing and walking. The measurement ofthe force exerted on the sole requires FSR or force/torquesensors. Additionally, cameras, IR sensors, tactile sensors,and other sensors can be attached to humanoids.

In general, a humanoid robot is modeled as a tree struc-ture mechanism having five open chains attached to a root(typically a free-floating six-DOF rigid body). This modelingstructure resembles that of humans; a humanoid platform hasfour limbs, one head, and one trunk. In contrast to humans,each limb of a humanoid may consist of more than just twoarticulated links (the upper limb and lower limb). Figure 1shows an example of an animated humanoid character and itstopological structure. Though this robot has limbs comprisedof many articulated links, allowing for snake-like movement,it is generally agreed that this robot can be regarded as ahumanoid. Some humanoids may possess excess degrees offreedom for greater redundancy, or hybrid kinematic struc-tures, e.g., series-parallel articulated arms for enhanced stiff-ness.

2.1 Humanoid data structure

A typical multibody system consisting of rigid bodies con-nected by joint elements can be represented as a tree struc-ture mapping bodies to nodes, joints to edges, and the baseto the root node, respectively [16]. Figure 2 shows the treestructure representation of a generic humanoid robot. Thetree structure depth is determined from the degrees of free-dom of the waist (K ), neck (L), each arm (M), and each leg(N ), respectively. For the Asimo robot, for example, K = 1,L = 3, M = 7, and N = 6.

We now define a device-independent humanoid proxy datastructure as shown in Fig. 3. Users write their motion programon this proxy, and not on the individual humanoid hardware,using a generalized set of commands that is also device-independent.

123

Intel Serv Robotics

Pelvis

Torso

Leg1

Foot

Leg Joint1

Leg(N-1)

...

Leg Joint2

Leg Joint(N-1)

Leg Joint(N)

Arm Joint1

Arm1

Hand

Arm(M-1)

...

Arm Joint2

Arm Joint(M-1)

Arm Joint(M)

Waist1

Waist Joint1

...

Waist(K-1)

Waist Joint2

Waist Joint(K-1)

Waist Joint(K)

Neck Joint1

Head

Neck1

...

Neck(L-1)

Neck Joint2

Neck Joint(L-1)

Neck Joint(L)

Fig. 2 Tree representation of a humanoid robot

PELVIS (base)

TORSO(t)

HEAD(h)

LEFT FOOT(lf)

RIGHT FOOT(rf)

LEFT HAND(lh)

RIGHT HAND(rh)

RIGHT LEG (rl)LEFT LEG (ll)

WAIST (w)

NECK (n)

RIGHT ARM (ra)LEFT ARM (la)

Fig. 3 Data structure of a humanoid proxy: nodes represent body partswhile edges represent kinematic chains

2.2 Body part (node)

A body part is a basic element of a humanoid that also servesas the object of a given motion command; these commandsare described in detail in the next section. A robot whosebody parts do not match the number of nodes contained inFig. 3 is not regarded as a humanoid. The set of body partsis defined as follows:

Pelvis The pelvis node is regarded as the root note andcontains global position and orientation information for thehumanoid proxy. The body frame attached to the pelvis is

identified with that of the humanoid proxy. The pelvis alsoreturns information about the robot’s center of gravity (CoG)and zero moment point (ZMP).

Torso and head The torso node is the root of the upper bodyof the humanoid proxy. Motions of the hand and head aredetermined with respect to the torso body frame. The headnode can contain cameras, and commands issued to the headnode provide, e.g., information about the camera’s line ofsight, but do not generate motions of the head.

Hand The hand node contains information about whether anobject is being grasped and also provides information aboutthe contact forces and torques between the hand and the envi-ronment.

Foot The foot node returns the contact force between the soleand ground and also calculates the support polygon. The usercan obtain a Boolean value informing whether the foot is incontact with the ground.

2.3 Connector (edge)

A connector is an edge that represents a kinematic structureconnecting two body parts (the root and leaf nodes); the nodecloser to the base node is the root node. A configurationspace can be associated with each connector; a point in theconfiguration space determines the position and orientationof the leaf with respect to the root node. For a humanoid wedefine the following connectors:

Waist The waist edge determines the relative motion of thetorso node with respect to the pelvis node. Most humanoidshave a single degree of freedom waist.

Neck The neck connects the torso node and head node. Ingeneral, the neck edge has two degrees of freedom (pan andtilt). This edge generates head movements that control, e.g.,the line of sight.

Arm The arm determines the motion of the hand node fromthe torso node. The arm contains the shoulder joint and wristjoint. The arm reach arm, or the maximum distance betweenthe shoulder joint and hand node, is specified by the user.

Leg The leg connects the pelvis and the foot nodes, and alsoprovides Jacobian information not only for the foot, but alsofor the CoG and ZMP.

3 Humanoid programming vocabulary

Our humanoid programming vocabulary consists of twotypes of motions: free-space movements, and task-orientedmovements. In the former case we develop a motion construct

123

Intel Serv Robotics

inspired by Labanotation, a notation used to describe dancechoreography. In the latter case we construct a set of primi-tive motions based on the method-time measurement system(MTM) for categorizing and analyzing typical motions of ahuman in the workplace.

3.1 Free space movements

Our construct for describing free space movements is basedon Labanotation, a system for analyzing and recordinghuman movement invented by Rudolf von Laban in 1928; themodern form of Labanotation was established by Hutchin-son [13]. A typical Labanotation score is similar to that ofmusic, but instead of being read from left to right, it is readbottom to top, where each column represents the motion ofa part of the body. Figure 4a shows the basic arrangement ofcolumns. Labanotation offers a convenient means of express-ing free space motions in a device-independent manner.

The motion of each body part is expressed by a sequenceof geometrical symbols placed in the corresponding column.The innermost column is called the support column, describ-ing the movement of the feet, and also which body part sup-ports the weight at a given moment. Both the support andleg gesture columns correspond to the foot node in Fig. 3.The body column, the arm column, and the head columncorrespond to the torso, the hand, and the head, respectively.

Figure 4b shows direction and level symbols, which indi-cate in what direction a movement occurs. The basic directionsymbols represent the direction of motion in 45◦ increments.The shading of the direction symbol indicates the level ofmovement (high, middle, or low). The hand or the foot inthe middle level is on the same level as the shoulder or hip.The high level is above this, while the low level is below. Forexample, if the arm is forward and high, the hand is in frontof and above the shoulder, at a 45◦ angle from the shoulder.

Limbs are extended by contraction (Fig. 4c), which is fur-ther divided into seven states. A contraction level of 3 setsthe elbow angle at approximately 90◦.

Based on the above we now define two free space motionprimitives, Reach and Support (Table 1). Some data types arefirst defined that capture Labanotation’s notation for direc-tion, level, and contraction:

– Direction represents the directional labanotation symbolsfrom Fig. 4b: Center (C), Left (L), Right (R), Forward (F),Backward (B), Left forward (LF), Right forward (RF),Left backward (LB), Right backward (RB).

– Level represents the three possible levels as shown inFig. 4b: high, mid, low.

– Contraction represents the distance ratio from the base(shoulder or hip) to the hand or foot with respect to thefull extension of the arm or leg. Labanotation describes

Arm

Bod

y

Leg

Ges

ture

Sup

port

Sup

port

Leg

Ges

ture

Bod

y

Arm

Hea

d

Center

RightLeft

(a) Staff

RightLeft

Forward

Backward

Center(Place)

Right Forward

Right Backward

Left Forward

Left Backward

• Middle

Up, High

Down, Low

(b) Direction and Level

Contraction 0 Contraction 1 Contraction 2 Contraction 3

Contraction 4 Contraction 5 Contraction 6

(c) Contraction

Fig. 4 Labanotation

contraction in seven states, from 0 (full extension) to 6(full contraction).

3.1.1 Reach in free space

A Reach motion can be expressed in terms of a body part,direction, level, and contraction. For instance, the motion ofwaving a hand can be described as the repetition of “RHandReach Center High 2” and “RHand Reach Right High 2”. Ifthe motion is the head, Reach causes the head to face the lineof sight given by the direction and level, and contraction isignored.

123

Intel Serv Robotics

Table 1 Syntax for Reach and Support in free space

3.1.2 Support

The Support motion designates which body part supportsthe weight of a humanoid. This motion plays an importantrole in the whole-body balance control of robots involv-ing the zero moment point (ZMP). If a robot is issuedthe command “LFoot Support”, the motion controller thentries to locate the ZMP in the support polygon of the leftfoot.

3.2 Task-oriented primitive motions

In this section we construct a set of primitive motions forhumanoid robots based on the Method-Time MeasurementSystem (MTM-1) [14,23], a well-established means of char-acterizing and classifying human tasks. The fundamentalmotion primitives in MTM-1 that involve us of the handsinclude reach, move, turn, grasp, position, disengage, andrelease. Primitives involving the legs and feet, e.g., supportand transport of the trunk, are included under body motions,while eye movements generated by movement of the headand neck are addressed separately.

MTM-1 is useful for the ex-post analysis of a givensequence of tasks, but does specify the a priori sequenceof tasks for a humanoid robot. For example, Reach Type B inMTM-1 is defined as “reaching for a single object in a loca-tion that may vary slightly from cycle to cycle,” implyingthat the worker already knows where the object was located,e.g., in the toolbox, on the shelf, or on the conveyer belt. Forhumanoids, however, search and recognition of the given tar-get object should be performed prior to Reach. In this section,MTM-1 primitive motions are modified into motion primi-tives suitable for humanoid robots.

3.2.1 Reach for an object

Reach is the basic motion for moving a body part to a targetand is further classified into the following five cases (Table 2):

Table 2 Syntax for task-oriented Reach (Orientation defined via ZYXEuler angles)

Table 3 Syntax for Move

– Case A: Reach for an object in a fixed location or for anobject in the other hand.

– Case B: Reach for a single object in a location that mayvary slightly from cycle to cycle.

– Case C: Reach for an object in a group of objects whichare jumbled so that search and select occur.

– Case D: Reach for a very small object or where an accu-rate grasp is required.

– Case E: Reach to an indefinite location to get the hand inposition for body balancing or the next movement.

Subcategories from B to E require prior acquisition of the tar-get object location, whereas Case A is directly implementableon humanoids. To make the vocabulary more compact, footand head motions as well as hand motion are also admittedunder Reach.

3.2.2 Move/Position

Move uses hand motions to transport an object to a tar-get location and consists of the following three cases(Table 3):

– Case A: Move an object to the other hand or against astop.

– Case B: Move an object to an approximate or indefinitelocation.

– Case C: Move an object to an exact location.

In contrast to Reach, Move motion only admits hand motions,since we assume the feet and head do not have object grasp-ing capabilities. Cases A and B can be replaced by CaseC if a robot knows in advance the exact location of astop. Position is used to specify more refined movementsthan those generated by Move, and is typically used in the

123

Intel Serv Robotics

Table 4 Syntax for ApplyForce

Table 5 Syntax for Grasp/Release

context of mating and aligning parts. The syntax is iden-tical to Move, but the low-level controller is designed tooffer greater resolution and accuracy than that of Move.The implementation this command is optional and dependson the extent of the resolution of the available controllaws.

3.2.3 ApplyForce

ApplyForce is similar to Apply Pressure in MTM-1, in whichforces (pressures) are applied by a part of the body to anobject, e.g., pushing a tool cart with one’s back. For our pur-poses we restrict ApplyForce to hands and feet (Table 4).

3.2.4 Grasp/Release

Grasp and Release are the basic primitives for securing andrelinquishing control of an object. Tasks such as determiningthe optimal grasp configuration and internal forces are dele-gated to the gripper module that reports grasp configurationsand status; these functions should be provided by the user(Table 5).

3.2.5 Crank

In MTM-1 [14] the Crank motion is employed when thehand follows a circular path to rotate an object, with theforearm pivoting at the elbow and the upper arm fixed. Tomake a robot perform a circular motion, we eliminate theadditional conditions for the forearm and upper arm, i.e., theCrank motion is redefined as the motion employed when thehand follows an exact circular path to rotate an object. Cranktypically involves a hybrid position/force control scheme andrequires the center of the circular path, its radius, and numberof revolutions (Table 6).

3.2.6 Eye movement

In MTM-1 eye movement commands consider both the “eyetravel time” (a function of the distance between points to

Table 6 Syntax for Crank

Table 7 Motion primitives

Motions Objects Arguments Control

Reach Head Direction Position

Torso Level

Hand Contraction

Foot Position

Move/Position Hand Position Force/position

Force

ApplyForce Hand Force Force

Foot

Crank Hand Center Force/position

Radius

Revolution

Grasp/Release Hand × device dependent

Support Foot × Position

and from which the eye travels, and perpendicular distancefrom the eye to the line of travel) and the “eye focus time”(the time required to focus upon an object). In MTM-1 headmovements are not considered. For the humanoid robots thatwe consider, the line of sight is adjusted by movement ofthe neck, by controlling the pan and tilt degrees of freedom.This movement can be achieved through the Reach motiondescribed earlier.

3.3 Summary of primitives

Table 7 summarizes the primitive motions for free-space andtask-guided motions.

4 A motion description language for multi-tasking

In this section we propose a motion description language,MDLm, that can describe and execute multiple tasks with pri-orities specified. MDLm is extended from the original MDL[5], with redundant robots like humanoids in mind. We firstreview the main features of the original MDL and work outexplicit representations for the case of kinematically redun-dant chains taking dynamics into consideration; this is thecase most relevant to humanoid robots. We also point out

123

Intel Serv Robotics

some limitations when applied to humanoids in multitaskingcontexts. We then present the specifics of MDLm.

4.1 Brockett’s motion description language

The motion description language (MDL) proposed by [5]is a device-independent language for programming systemsthat are governed by differential equations. The system stateequations are expressed in the generic form

q(t) = f (q(t)) + G(q(t))ν(t)

y(t) = h(q(t)) (1)

where q, ν, and y, respectively, denote the state, control,and observation. The control ν(t) = u(t)+ K (y(t)) canbe further decomposed into the sum of a feedforward inputu(t) ∈ U and a feedback term K : Y → U , in which U isthe space of admissible controls, and Y the space of observa-tions. An atom in MDL consists of the triple σ = (u, K , τ ),where u and K , respectively, denote the feedforward andfeedback terms, and τ denotes a time interval. Given a stringof N MDL atoms

(u1, K1, τ1)(u2, K2, τ2) · · · (uN , KN , τN ), (2)

the system (1) evolves according to

x = f (x) + G(x)(u1 + K1), t ∈ [t0, τ1)

x = f (x) + G(x)(u2 + K2), t ∈ [τ1, τ2)

...

x = f (x) + G(x)(uN + KN ), t ∈ [τN−1, τN ], (3)

where t0 denotes the initial time.MDL allows for coordinate transformations in the fol-

lowing way: given a differentiable mapping x = φ(q), thevelocities of x relate to the velocities of q by

x = ∂φ

∂qq = J (q)q, (4)

with the previously stated equation for q still valid. An inputstring of the form σ1 φ1 σ2 σ3 φ2 σ4 . . ., in which the φi aredifferentiable mappings, allow for change of coordinatesbetween execution of atoms.

Finally, we note that previous works [19,11,21] haveextended MDL by allowing for interrupts by sensor feedback,with a view toward mobile robot applications in dynamicenvironments. The interrupt functions ξ : Y → {0, 1} aug-ment the MDL mode by creating a triple σ = (κ, ξ, T ),where κ = u(t) + K (y(t)) is an encapsulated control law.

4.2 Multitasking motion description language

One of the disadvantages of MDL in its original form isthat the programming of redundant robots is cumbersome.

The execution of multiple tasks, e.g., balancing while reach-ing, necessarily exploits redundancy, and planning redun-dant motions typically begins with an explicit decomposi-tion of the input control into its task and self-motion (or nullspace) subspaces. The task is typically specified in task coor-dinates (e.g., coordinates corresponding to the end-effector),and self-motions afforded by the redundancy is then exploitedto, e.g., perform additional tasks, or to minimize some per-formance criterion while performing the primary task. MDLin its current form does not offer explicit constructs to sup-port this type of multitask programming. In this section weshow MDL can be extended to make multitask programming,and more generally redundant motion planning, more directand intuitive. We first examine the state equations in bothjoint and task space for redundant robot systems, first takinginto consideration only the kinematics and then including thedynamics and allowing for hybrid force-position control. Wethen provide details of our multitasking MDL, or MDLm, andshow how the new constructs make multitask programmingmore amenable.

4.2.1 Redundant robot systems modeling

The kinematic modeling of redundant robot systems typi-cally begins by specifying a set of task coordinates x (e.g.,the end-effector coordinates) and specifying the relationshipbetween x and the configuration coordinates q (e.g., the for-ward kinematics x = φ(q)). The state equations can then bewritten in the general form

q = J †v + (I − J † J )η, (5)

where J † ∈ R3×2 is the pseudo-inverse of J [4], I ∈ R

3×3 isthe identity, and η ∈ R

3 is an arbitrary vector. Setting v = x ,where x(t) is some desired end-effector trajectory, the secondterm (I − J † J )η then generates null space motions; varyingη causes redundant internal motions while ensuring that theend-effector tracks x(t).

Redundant manipulator dynamics can also treated withinthe MDL framework. Given the operational space dynamicequations in the form

M(q)x + b = F,

where M(q) and b denote the operational space mass matrixand bias forces, and F is the command force, if we adopt thecomputed torque control scheme, for example, the commandforce for the desired trajectory, xd becomes [22]

F = M(q)(xd − Kν e − K pe) + b,

where e = x − xd , and Kν and K p are feedback gains. Then,the command torque can be written as [15,28]

τ = J T F + N T η, (6)

123

Intel Serv Robotics

Fig. 5 End-effector attached to an arbitrary kinematic chain

USER

Interpreter

Staff Manager

Compiler

Hardware Platform

Motion Vocabulary

Humanoid Proxy

Internal Dynamic

Model

Hardware Interface / Simulation Environment

Control Atom Dictionary

query

Motion Description

Motion Script

Report

ReportMotion Staff

Controller

Motion Programming

Layer

Physical Layer

MDLm Compilation

Layer

Fig. 6 Humanoid motion programming hierarchy

where N is the null space projection operator of J , and η

is an null space arbitrary force vector. The first term J T Fgenerates actuator torques that force the end-effector to trackthe desired trajectory xd , while the choice of η generatesredundant self-motions.

For motions like scrubbing a surface with a sponge, bothposition and force control are required while taking into con-sideration of the dynamics. Towards this end, consider anend-effectorE attached to an arbitrary kinematic chain whosebase is fixed in Euclidean space (Figs. 5, 6). Let o-xyz be theorthonormal reference frame {o} and B-xyz be the orthonor-mal frame {B} attached to the center of mass of the rigidbody. The dynamics of E with respect to the frame {B} canbe written as

M V + b = F − J Tc λ, (7)

where M is the operational space mass matrix, V = (ω, v)

is the spatial six-dimensional velocity in se(3), Jc is the con-

straint Jacobian, and b, F , and λ are, respectively, the biasforce, the actuation force, and a constraint force in se∗(3)

[24]. Let the state be x =(

TV

), where T ∈ SE(3) is the

position and orientation of E with respect to the frame {o},and V ∈ se(3) is a time derivative of T defined as above.Equation (7) can be rewritten as follows:

x =(

0M−1

)F +

(V

−M−1b

)+

(0

−M−1 J Tc

)λ

=(

V−M−1b

)+

(0 0

M−1 −M−1 J Tc

) (Fλ

)(8)

= f (x) + G(x)u (9)

where u = (F, λ) is the control input. We now treat E as akinematic machine governed by the differential equation (9).

Now we define the atoms of the control input u denotedby σ , as a quintuple of the form

σ = (Xd(t), λd(t), K , ξ, T d) (10)

where Xd(t) and λd(t) are the desired trajectory in SE(3) andthe constraint force in se∗(3), respectively, K is the feedbackgain (or stiffness matrix), ξ is an interrupt to the system whichis activated in the case of emergencies or changes, in theenvironment, and T d is the time duration that defines theepoch over which (Xd(t), λd(t), K ) is to be used.

For example, suppose that a computed torque controlscheme is adopted in order to track a given end-effector tra-jectory Xd(t), where t ∈ [0, T d ]. The control atom and con-trol input u become

σ = (Xd(t), 0, (K D, K P ), 0, T d), (11)

u =(

F0

)=

(M(Vd − K De − K P e) + b

0

), (12)

where (K D, K P ) are positive-definite control gains, Vd is thesecond-order derivative of Xd , and e is the operational spaceerror1 e = Xd − T .

4.2.2 MDLm

The basic idea behind MDLm is to allow multiple MDLatoms and coordinate mappings to be simultaneously input tothe MDL machine, in a manner consistent with the redundantkinematics and dynamics. A command in MDLm specifieswhich mapping between the actuator and task space shouldbe applied, what motion should be performed, and when theaction should occur for the given kinematic machine.

Formally, let X = {x : R+ → R

n} be the set of states ofthe robot, U = {u : R

p × R+ → R

m} the set of admissiblecontrol laws, Y = {y : R

n → Rp} the space of sensor

1 The operational space error may differ according to a definition ofthe orientation error.

123

Intel Serv Robotics

outputs, B = {ξ : Y × R+ → {0, 1}} the set of Boolean

functions on p variables, and C = {� : X → Rm} the set of

transformations from the state space to the input task space.The n, m, and p, respectively, stand for the dimensions of thespaces of the robot state, control input, and the sensor output.An atom in MDLm is then defined as follows:

Definition 1 A Multitasking MDL atom is a triple σ =(�, u, ξ), where u ∈ U is the control input, � : X → R

m

assigns each state to a corresponding control, and ξ ∈ Bis an interrupt signal triggered by sensor feedback or thetimeout measured from the time an atom is initiated. The setof multitasking MDL atoms is denoted .

An MDLm atom essentially defines a vector field overspace-time; here “space” refers to the state space or outputspace of a dynamical system. The life span of an atom ismaximal when ξ is turned off. To evaluate or execute the atomσ = (�, u, ξ) means to apply the input u to the kinematicmachine until the interrupt function ξ is 0 (or Boolean false),where u is specified in the coordinate system defined by themapping �.

The MDLm atom is similar in structure to that of MDLe[11,19]. However, it has an important additional element, themapping � between the state space and task space. � orig-inates in the general change of coordinates feature of MDL[5]. In MDLm � is a constitutive element of an MDLm atom,whereas in the original MDL � is a change of coordinatesoperator executed before the next atom.

As an example, consider a standard three-link planar openchain, and suppose the following atom is input:

σ+x1

(τ ) = (�, u+x1

, ξτ ), (13)

where

�(q)=[

l1 cos q1+l2 cos(q1+q2)+l3 cos(q1+q2 + q3)

l1 sin q1+l2 sin(q1+q2)+l3 sin(q1+q2+q3)

]

u+x1

(t) = (+1, 0)

ξτ (t) ={

1 if t ∈ [0, τ ]0 else

. (14)

σ−x1

, σ+x2

, and σ−x2

are defined similarly, with u−x1

= (−1, 0),

u+x2

= (0,+1), and u−x2

= (0,−1). σ+x1

causes the end-effector to move in the x1 direction with speed +1 for τ

seconds. For instance, the string

σ+x1

(0.5) σ+x2

(0.5) −x1

(0.5) σ−x2

(0.5)

draws a square of length 0.5 on each side.

Definition 2 The termination atom (I, 0, ξ)�, is a specialsubtask atom that denotes the end of every multitasking MDLword.

The termination atom always appears at the end of eachMDLm word. Its primary role is to prevent uncontrollable

internal motions that might occur in case the MDLm atomsare not given appropriately, or some of the atoms in the givenword terminate while the other atoms are still being executed.For this reason the priority of the terminal atom is always setto be the lowest.

This atom can be defined in other ways to suit the purposeof the user. The simplest termination atom is to maintainthe actuator positions at the instant that the previous atomis terminated by an interrupt. The termination atom for amachine controlled at the kinematic level would simply bethe zero vector. The machine maintains the final posture whenall given atoms are completed. In the case of dynamics-basedcontrol, the termination atom may be a simple controller thatcauses the joint positions to settle to the final position, e.g.,x = −(x − x0).

Definition 3 A multitasking MDLm word is a string com-posed of pairs (σ, z), where σ ∈ and z ∈ Z

+ denotes atask priority index. Each multitasking MDL word ends witha termination atom of the lowest priority.

In MDLm, a word w represents a whole-body behaviorthe machine should initiate at a given time t0, not a set ofmotions performed sequentially. To understand the functionof an MDLm word, consider the following example:

(�1, u1, ξ1), 1) (�2, u2, ξ2), 2) (In×n, u3, ξ3), 3),

which is input to the simple kinematic machine

x = v ∈ Rn . (15)

The kinematic machine must simultaneously complete thethree subtasks σi = �i , ui , ξi ), i), i = 1, 2, 3. If this isnot possible, the machine must carry out σ1 with the highestpriority, and then perform the others with designated prior-ities. For the kinematic null space decomposition describedearlier, we obtain the whole-body control law

v = J †1 u1 + P1(J †

2 u2 + P2(u3)) (16)

where Ji = ∂�i/∂x, i = 1, 2, 3, J † is the generalizedinverse of J , and Pi is the projection mapping onto the nullspace of Ji . Under this control law, the machine will com-plete subtask σ1, perform σ2 without disturbing σ1, and thentry to execute σ3 with redundant degrees of freedom whichare not contributing to σ1 and σ2.

Definition 4 A multitasking MDL word with task prior-ity switch is a string composed of triples (σ, z, μ), consist-ing of an atom σ ∈ , a task priority z ∈ Z

+, and μ ∈ B,and ending with a termination atom set to the lowest priority.

The multitasking MDL word with task priority adjustmentis inspired by [19], in which switching conditions includesensor feedback to allow the system to react to dynamicenvironment changes; the interrupt functions ξ : Y → {0, 1}

123

Intel Serv Robotics

augment the original MDL by creating a triple σ = (u, ξ, T ).From this idea, a type of flexible priority switching frame-work can be introduced, in which certain subtasks can betriggered on or off by sensor feedback. To illustrate, supposethat a person is walking holding a hot cup of coffee. Mostlikely the highest priority will be to hold the cup stably. Atthe moment the person slips on a slippery surface, the highestpriority will be to recover balance, even at the risk of spilling(in a safe manner) the contents of the cup. In our approachthe higher priority is turned off by a change of internal orexternal state. By bringing this idea into the MDLm word,the user can manipulate the priority of subtasks.

More specifically, suppose J1, u1 of Eq. (16) are initiallydefined so as to hold the cup stably, and J2, u2 are definedto maintain balance. By redefining J1 as

J1 = S(μ1)∂�1

∂x, (17)

where S(μ) = μI , if at any instant μ is turned off (e.g.,on a slippery surface), then the highest priority task will beswitched off (J †

1 u1 vanishes, and P1 = I ), and the robotwill abandon the cup in order to recover balance. (Note that‘abandon’ here does not imply dropping or releasing the cup;the termination atom causes the robot to settle into the finalposition.) After recovering balance, μ is turned on again,and the robot resumes the task of stably holding the cup.In this manner particular atoms can selectively contribute towhole-body control without reconstructing the entire controllaw.

5 Programming architecture

In this section we describe our three-tiered robot program-ming hierarchy, consisting of a motion programming layer atthe highest level, an MDLm compilation layer at the middlelevel, and a physical layer at the lowest level.

5.1 Motion programming layer

This layer aims to provide users a hardware-independent pro-gramming interface between different humanoid platforms.

5.1.1 Interpreter

This component enables the user to interact with and com-mand robots through, e.g., a keyboard or GUI. A syntax andvocabulary is defined as in Sect. 3. The purpose of this com-ponent is to provide a reusable motion program syntax acrossdiverse humanoid platforms. The interpreter parses and trans-lates the human inputs into commands, and constructs themotion staff.

5.1.2 Motion vocabulary

The motion vocabulary component we define here can bethought as a building block for humanoid motion descrip-tion. Restricting our focus to tasks that a humanoid typicallycarries out, it is possible to build such compound motionsfrom a small number of simpler motion primitives. For exam-ple, the motion of grasping an object far from the robotcan be achieved by the sequence of “Reach”, “Move”, and“Grasp”.

We define a motion vocabulary such that it has three ele-ments that are essential to describe the motion of a humanoid:the object of the motion, a verb, and the parameters describedin Sect. 3. The object represents what is performing themotion, and the verb is used to describe the motion. Thecontrol policy indicates how the motion is generated orcontrolled, e.g., position control, force control, or hybridposition-force control. The actual control law for each motionis changeable and can be defined by individual users via theMDLm Compilation Layer described below in Sect. 5.2. Theparameters represent the requirements that should be satis-fied at the end of the motion. Table 7 summarizes the motionprimitives considered in this paper.

5.1.3 Humanoid proxy

A reusable and portable system should provide an abstrac-tion of hardware platforms. The humanoid proxy acts as anabstract model that serves as the object of motion program-ming. Users write their motion program with a generalizedset of commands on this proxy, and not on the individualhumanoid hardware. These commands do not depend on anyparticular robot so that the resultant code can be reused onother humanoid robots that can be represented by a humanoidproxy.

5.1.4 Motion staff and staff manager

The motion staff is a type of a score recording the whole-bodymotion sequence of the humanoid proxy. Inspired by the staffin Labanotation (Fig. 4a), this data entity is simplified andadjusted to describe humanoid motions.

The staff manager is responsible for any communica-tion between the motion programming layer and the MDLmcompilation layer. This component contains the final motionprogramming code and applies the plan composer over themotion staff.

5.2 MDLm compilation layer

The MDLm compilation layer’s role is to connect the motionprogramming layer and the physical layer, by compiling ahigh-level motion program in the form of a script compre-

123

Intel Serv Robotics

query

Plan Manager

Word Compiler

Plan Composer

Controller

Internal Dynamic Model

Control Atom Dictionary

Motion Staff

Plan

Single Word

PlanTerminated

ControlLaw

Interrupt Condition

WordTerminated

InterruptCondition

Atom

ControlLaw

ControlLaw

MDLm Compilation Layer

Fig. 7 The MDLm compilation layer

Head

Torso

Pelvis

Left Hand

Right Hand

Left Foot

Right Foot

Reach

Support

Reach

Reach Support

SupportSupport

ApplyForceReach

Reach

Move

Fig. 8 Plan composer example: motion staff converted into MDLmplan p consisting of MDLm words w1, . . . , w5

hensible to humans, into a hierarchical control law that makessense for robots. The framework in Fig. 7 shows the proce-dure for compiling a multitasking motion description lan-guage and the associated data flow.

5.2.1 Plan composer

The plan composer receives the motion staff from the staffmanager in the motion programming layer and converts itinto an MDLm plan. This component acts as a simple inter-preter, which reconstructs the motion program from scriptform into a data structure for MDLm (Figs. 8, 9, 10, 11, 12).The converted MDLm plan is sent to the plan manager byuser command, and the plan composer waits for a feedbacksignal from the plan manager. If the plan manager sends asignal which indicates that the given plan has been termi-nated, then the plan composer informs the staff manager oftermination of the task, and waits for the next motion staff.

Fig. 9 Hubo simulation model

5.2.2 Plan manager

This component manages the MDLm words constituting theMDLm plan given by the plan composer and monitors thestate of a humanoid robot and its environment. Upon receiv-ing the MDLm plan, the plan manager inspects each MDLmword constructing the given plan. Suppose that a given plandescribes the motion “Go to the kitchen and fetch the cup onthe table”. If the robot is already at the kitchen table, thenit need not perform “Go to the kitchen”. The plan managerchecks whether the goal of each word has been accomplishedin the reverse order of the motion sequence, by querying thestate of the humanoid proxy to the internal dynamic modelso as to avoid redundant execution of the motion.

5.2.3 Word compiler

The word compiler queries and extracts the control laws fromthe control atom dictionary. After that, any interrupt for eachatom is examined, and atoms whose interrupts are activated

123

Intel Serv Robotics

Fig. 10 Wave sequence of Hubo

Fig. 11 Dishwashing sequence of Hubo

Fig. 12 Walking sequence for Hubo

are excluded from the given MDLm word. If the resultantword is empty, then all atoms are deactivated and the wordcompiler sends a ‘terminated’ signal to the plan manager.

The remaining atoms are sorted and rearranged accordingto their priorities, and the word compiler builds a whole-bodycontrol law. It can be implemented using the hierarchical con-trol framework proposed in previous work [28,2,27]. Algo-rithm 2 shows an example compilation of the hierarchicalcontrol at the force level.

5.2.4 Control atom dictionary and internal dynamic model

Each primitive motion introduced in Sect. 3 has its own con-troller, and these controllers are defined in the control atom

dictionary. The user may redefine the control laws corre-sponding to the motion primitives in any desired way.

The internal dynamic model possesses all informationabout the state of the humanoid robot that the humanoid proxycomponent represents. This information is updated from theAPI provided by the manufacturer and/or calculated by asimulation library.

5.3 Physical layer

This lowest layer in our architecture consists of a low-levelcontroller and the hardware interface. The low-level con-troller computes actuator commands needed to execute thegiven tasks in accordance with the control law received fromthe MDLm compilation layer described in Sect. 5.2. The con-

123

Intel Serv Robotics

troller connects with hardware devices via a hardware APIusually provided by the manufacturer, or middleware such asCORBA and Player/Stage [10].

A portable system should provide abstraction of hardware,and abstractions should be made at different levels. While thehumanoid proxy in Sect. 5.1.3 provides an abstraction for arobot platform, this layer abstracts hardware devices such asactuators and sensors. Another important role of the physicallayer is dynamic simulation.

6 Case study

In this section we demonstrate our programming architecturewith a case study of the Hubo humanoid robot. Hubo is 1.25 mand 55 kg. The robot has total 41 DOFs: six-DOF legs, six-DOF arms, five-DOF hands, one-DOF waist, two-DOF neck,and two-DOF eyes [25,26].

The MDLm atom class is defined as shown in Table 8.The command script from the user (such as Tables 10, 11)is parsed and converted into MDLm class instances. Thisprocess is executed by the plan composer and an illustrativeprocess is shown in Table 9.

As shown earlier, MDLm atom programming does notrequire direct access to the actual robot hardware. Instead,with an MDLm atom being built on the humanoid proxy

Table 8 MDLm atom class

Table 9 Code for ‘lhand reach (0.23, 0.15, 0.9) (−0.5,−2.0,−0.2)′

Table 10 Console commands for waving

Table 11 Console commands for dishwashing

layer, the user can reuse these MDLm atoms with little if anymodification.

We now present simulation results for the primitivemotions Reach, ApplyForce, and Support.Reach: Wave

Wave is composed of alternate repetitions of primitivemotions Reach parameterized by triples of Direction, Level,

123

Intel Serv Robotics

Contraction. Reach motions are applied to both hands simul-taneously so that the robot performs concurrent motions.Reach/ApplyForce: Dishwashing

Dishwashing is composed of the primitive motions Reachand ApplyForce. ApplyForce applies a force constraintbetween a dish and a sponge (a ball in the simulation) whileReach makes the hand scrub the dish.Reach/Support: Static Walking

We now consider an example of statically stable walk-ing. The control criterion is to maintain the projection of thecenter of gravity (COG) to be inside the foot support poly-gon. Support maintains the projection of the COG insidethe support polygon, while Reach moves each leg to thedesired step position. Thus, commanding Support and Reachalternately to one leg and the other causes a robot to walkin a statically stable manner. Other more advanced walk-ing algorithms can be implemented, although these are bynature dependent on the hardware and the programming canbecome quite involved depending on the choice of algorithm(Table 12).Motions with balance

We now consider motions with simultaneous balancerequirements. The above two motions, waving and dish-washing, are simulated with balance. Hubo stands on a platethat is rotating along its horizontal axes. The robot main-tains its balance without falling down via the motion primi-tive Support while performing two tasks. Simulation resultsgenerated on a dynamics simulation engine are shown inFig. 13.

Table 12 Console commands for static walking

7 Conclusion

In this paper, we have developed a multitasking motion pro-gramming framework for humanoid robots, from high-level,symbolic specifications to low-level control laws. This effortworks toward bridging the gap between a reusable descrip-tion of control programs and their execution on the targetsystems. The main contributions of this paper are as follows:

– Multitasking motion description language (MDLm) Wehave expanded the original MDL [5] to accommodatemultitask descriptions with task priority. MDLm inheritsthe advantages of the original MDL, and in addition, gen-erates multitasking motion control laws utilizing changeof coordinates.

– High-level motion primitives for abstract humanoid Wehave developed a concrete set of criteria on what consti-tutes a humanoid that is workable and practical from aprogramming perspective, without overly restricting thediversity of humanoid designs. Based on this criteria webuild an abstract humanoid model, called a humanoidproxy, and define motion primitives that constitute avocabulary for programming humanoid robots to performboth free-space motions and tasks involving interactionwith objects.

– Integrated programming architecture We have designedand implemented a new software architecture that facili-tates the development of motion programs for humanoidrobots. In this architecture, high-level motion primitivesand MDLm are balanced and cooperate in a consistentmanner. This architecture provides the necessary featuresfor performing dynamics simulation.

While in the current paper we assume perfect knowledgeabout the environment to focus on specifics of our archi-tecture, adaptation of our framework to unstructured envi-ronments can proceed in one of two ways. One approach

Fig. 13 Hubo motions with balance

123

Intel Serv Robotics

is to enhance the hardware controller via advanced sensorsand control schemes so that the robot reacts to unexpectedinterrupts by, e.g., unknown obstacles, through human-likereflexes. The other approach is to reprogram the whole-bodymotion every time information is updated, akin to higherlevel processing in the brain. A useful direction for furtherresearch is to combine our architecture with a high-level per-ception framework so that the robot is able to cope with highlydynamic environments.

References

1. Alami R, Chatila R, Fleury S, Ghallab M, Ingrand F (1998) Anarchitecture for autonomy. Int J Robot Res

2. Baerlocher P, Boulic R (1998) Task-priority formulations for thekinematic control of highly redundant articulated structures. In:Proceedings of the IEEE/RSJ international conference on intelli-gent robots and systems, vol 1, pp 323–329

3. Beetz M, Mösenlechner L, Tenorth M (2010) A cognitive robotabstract machine (CRAM) for everyday manipulation in humanenvironments. In: Proceedings of the IEEE/RSJ international con-ference on intelligent robots and systems

4. Ben-Israel A, Greville TNE (2003) Generalized inverses theoryand applications. Springer, Berlin

5. Brockett RW (1988) On the computer control of movement. RobotAutom

6. Brockett RW (1990) Formal languages for motion description andmap making. In: Proceedings of symposia in applied mathematics

7. Drumwright E (2007) The task matrix: a robot-independent frame-work for programming humanoids. Ph.D dissertation, Universityof Southern California

8. Drumwright E, Ng-Thow-Hing V, Mataric M (2006) The TaskMatrix framework for platform-independent humanoid program-ming. In: Proceedings of the IEEE/RAS international conferenceon humanoid robots

9. Firby RJ (1989) Adaptive execution in complex dynamic worlds.Ph.D dissertation, Yale University

10. Gerkey B, Vaughan R, Howard A (2003) The player/stage project:tools for multi-robot and distributed sensor systems. In: Proceed-ings of the 11th international conference on advanced robotics, pp317–323

11. Hristu-Varsakelis D, Egerstedt M, Krishnaprasad P (2003) On thestructural complexity of the motion description language MDLe.In: Proceedings of the 42nd IEEE conference on decision and con-trol

12. Huang L, Hudak P (2003) Dance: a declarative language for thecontrol of humanoid robots

13. Hutchinson A (1972) Labanotation or Kinetography Laban: systemof analyzing and recording movement. Oxford University Press,New York

14. Karger DW, Bayha FH (1965) Engineered work measurement.Industrial Press, New York

15. Khatib O (1987) A unified approach for motion and force control ofrobot manipulators: the operational space formulation. Int J RobotAutom 3(1):43–53

16. Kim J (2002) Geometric algorithms and data structures for physicalbased simulation. Ph.D dissertation, School of Mechanical Engi-neering, Seoul National University

17. Kortenkamp D, Simmons R (2008) Robotic systems architecturesand programming. In: Siciliano B, Khatib O (eds) Springer hand-book of robotics. Springer, Heidelberg

18. Lindström M, Oreback A, Christensen H (2000) BERRA: aresearch architecture for service robots. Robot Autom

19. Manikonda V, Krishnaprasad P, Hendler J (1998) Languages,behaviors, hybrid architectures and motion control. Math ControlTheory

20. Martin P, de La Croix J, Egersted M (2008) MDLn: a motiondescription language for networked systems. In: IEEE conferenceon decision and control (CDC)

21. Martin P (2010) Motion description languages: from specificationto execution. Ph.D dissertation, School of Electrical and ComputerEngineering, Georgia Institute of Technology

22. Murray RM, Li Z, Sastry SS (1994) A mathematical introductionto robotic manipulation. CRC Press, Boca Raton

23. Niebel B, Freivalds A (2003) Methods, standards, and work design.McGraw-Hill, New York

24. Park FC, Bobrow JE, Ploen SR (1995) A lie group formulation ofrobot dynamics. Int J Robot Res 14(6):609–618

25. Park I-W, Kim J-Y, Lee J, Oh J-H (2005) Mechanical design ofhumanoid robot platform KHR-3 (KAIST Humanoid Robot-3:HUBO). In: Proceedings of the IEEE/RAS international confer-ence on humanoid robots

26. Park I-W, Kim J-Y, Lee J, Oh J-H (2005) Mechanical design of thehumanoid robot platform, HUBO. Adv Robot 21(11):1305–1322

27. Sentis L, Khatib O (2005) Synthesis of whole-body behav-iors through hierarchical control of behavioral primitives. Int JHumanoid Robot 2(4):505–518

28. Siciliano B, Slotine JJE (1991) A general framework for managingmultiple tasks in highly redundant robotic systems. Adv Robot

123