SUPPORTING ONLINE MATERIALfor

Efficient bipedal robots based on passive-dynamic walkers

Steve Collins1, Andy Ruina2∗, Russ Tedrake3, Martijn Wisse4

1Mechanical Engineering, University of Michigan, Ann Arbor, MI 48104, USA2Theoretical & Applied Mechanics, Cornell University, Ithaca, NY 14853, USA

3Brain & Cognitive Sciences and Center for Bits and Atoms,Massachusetts Institute of Technology, Cambridge, MA 02139, USA

4Mechanical Engineering, Delft University of Technology, NL-2628 CD Delft, NETHERLANDS

∗Correspondence to: ruina@cornell.edu.

January 25, 2005

This supporting material includes

1. Materials and Methods. Details about therobots’ construction and control.

2. An analogy. A description of the parallels(in content, not in significance) between firstpowered flight and these robots.

In addition we hope readers will look at the videos:

S1 Cornell powered biped. This movie showsvideos of the robot walking on flat ground. Aslow-motion segment shows the ankle push-off actuation.

S2 Delft pneumatic biped. This movie shows therobot walking down a hall with views from thefront, side, and back.

S3 MIT learning biped. This movie begins withthe powered robot imitating passive walkingdown a 0.9 degree slope, from three cameraangles. Then it shows the robot learning towalk on flat terrain with foam protective pads.

The controller kicks the robot into random ini-tial conditions between learning trials. Aftera few minutes, the robot is walking well inplace, so we command it to walk in a circle.Finally, we show the robot walking down thehall, on tiles and outside; this footage is takenfrom a single trial where the robot adapted toeach change in the terrain as it walked.

More material and other videos are availablethroughhttp://tam.cornell.edu/˜ruina/powerwalk.html .

1 Materials and Methods

Details about the three robots are presented here.

1.1 Cornell powered biped.

This robot is autonomous; it has no power linesand no communication links to the outside. It con-sists of two 0.8 m long legs, each having knees, at-tached at a hip joint. The robot has curved-bottom

1

Figure 1: The Cornell powered biped

feet, arms, and a small torso which is kept uprightby connection to the legs with an angle-bisectingmechanism. Each arm carries a battery. The rightarm is rigidly attached to the left leg andvice versa,reducing yaw oscillations (Fallis, 1888, Collins,Wisse and Ruina 2001). The machine weighs 12.7kg and has 5 internal degrees of freedom (one hip,two knees, and two ankles). The thigh-to-shanklength and mass ratios are 0.91 to 1 and 3.3 to1, respectively, which mimics human architectureand seems important to the passive dynamics of thesystem. The hip joint is fully passive. A latch ateach knee passively locks the shank to be paral-lel with its proximal thigh throughout stance. This

latch is released by a solenoid at the completion ofankle push-off, at which point the knee is passiveuntil knee-strike. Ankle push-off restores energylost, mostly to heel-strike collisions. To minimizethe needed motor size, energy for ankle push-off isstored in a spring between steps.

The control circuitry is located in thehip/torso/head visible in the figure. A finite-state machine with eight binary inputs and outputsis implemented in 68 lines of code on an AtmelAT90S8515 chip running on an ATSTK500standard development board. A second boardwith relays and passive conditioning componentsconnects the board to the electromechanical andsensory parts. During the first state, Left LegSwing, all actuators are unpowered and the leftknee latch passively locks at knee strike. Whenground-detection contact switches below theleft foot detect impending heel strike, the statechanges to Right Ankle Push-Off. This beginsa timed activation of the solenoids that releasethe plantar-flexor spring of the right foot. Whenswitches detect full foot extension, the statechanges to Right Toe Return. During this state, a9.5 Watt, 6.4 oz gear-reduced MicroMo©R motor isactivated, slowly retracting the foot and restoringspring energy. Also, a short time after detection ofimpending left-foot heel-strike, a solenoid unlocksthe right knee. When a switch on the motorindicates full foot retraction, the state changes toRight Leg Swing, and the foot-retraction motoris deactivated. The state machine then swapsroles for the left and right legs and goes to theinitial state. Taking all sensing, including thesensing of internal degrees of freedom (whichcould in principle be made open loop), about 20bits of information per step flows to the processor.Environmental sensing, i.e., the instant of footcontact, is about seven bits per step.

This machine has only one capability, walkingforward. It is designed to walk with minimal en-ergy use. Its speed, path and joint motions are notshaped or controlled but follow from its mechanicaldesign and primitive ankle push-off actuation. An-

2

kle extension occurs mostly after the opposite leghas completed heel-strike collision, so in principlethe machine could be made to consume about fourtimes less energy by having ankle push-off before,rather than after, the opposing leg’s foot-to-groundcollision (Kuo, 2002). However, push-off beforeheel-strike seems to require more precise timingand also requires greater ankle torques. We sur-rendered this possible gain in energy effectiveness,in trade for greater simplicity of control. It is theo-retically possible for a robot of this size and speedto use less than one watt (3 watts divided by 4).

Because low energy use was a primary goal inthe design of the Cornell robot, we were careful inmeasuring its power consumption. Measurementswere taken during walking trials using an off-boarddigital oscilloscope connected with fine wires. At200 samples per second, the scope measured bat-tery voltage on one channel, and the voltage dropacross a 1 ohm power resistor in series with thebatteries on another. The product of the voltageand current was, on average, 11 watts (yieldingcet = 0.02). Mechanical energy use was measuredin experimentally simulated push-off trials. Theforce at each foot contact point was measured asthe ankle was slowly moved through its extensionrange, and this force was integrated to estimate me-chanical work per step, yielding an average over acycle of about 3 watts (cmt = 0.055). This is aslight over-estimate of the mechanical energy usedfor propulsion because some energy is lost at thecollision occurring when the ankle collides with theshank at full ankle extension.

The Cornell powered biped walked successfullyduring a period of a few weeks in August 2003.This robot is a proof-of-concept prototype, not aproduction run machine. It was developed as a one-shot attempt using a small ($10K) budget. As is notunusual for experimental robots, the device did notstand up well to long periods of testing; on averageabout one mechanical component would break perday of testing. For instance, the cables connectingthe motor to the primary ankle extension spring ranover a small radius pulley at the knee and broke fre-

quently. When the Cornell robot was best tuned itwould walk successfully at about 30% of attempts.Failed launches were due to inadequate matchingof proper initial conditions, most often ending withfoot scuff of the swing leg. The robot seems mildlyunstable in heading, so once it was launched, theprimary failure mode was walking off of the (nar-row) walking table or walking into a wall. Becauseit walked 10 or more steps many times, with the endonly coming from hitting a wall or cliff, the gait isclearly stable (although not very) for both lateraland sagital balance. However, the reader can makehis/her own judgments based on the videos whichare the basic documentation of success.

A key aspect of the success, and also the touch-iness, of this robot is with the shape and con-struction of the feet. The general issues related tofeet for this class of robots is discussed in Collins,Wisse and Ruina (2001). We tried various support-rail curve shapes and overall foot stiffnesses, andonly one of these led to successful walking.

The Cornell machine, which uses wide support-ing feet for lateral stability, is not being maintained.Rather, present efforts are aimed at developing amachine which uses active (simple) control for lat-eral balance using foot placement. Ths idea is re-lated to the kinematic lean-to-stear mechanism ofthe Delft Biped, or more intuitively, is somethinglike the steering used by a bicycle rider for balance.

1.2 Delft pneumatic biped

This robot is also autonomous; all power sourcesand computation are onboard. The robot weighs8 kg and has 5 internal degrees of freedom (onehip, two knees, two skateboard-truck-like ankles),and is 1.5 m tall. The robot consists of two legs,each with knees and ankles, and an upper body.Theswinging arms do not add degrees of freedom;the arms are mechanically linked to the oppos-ing thighs with belts. The knees have mechani-cal stops to avoid hyper-extension, and are lockedwith a controllable latch. Two antagonist pairs ofair-actuated artificial muscles (McKibben muscles)

3

A latch at the knee

joint is locked through

stance and unlocked in

the swing phase

The right arm is

slaved to the left

thigh via this

drive belt

The chains making up

the bisecting

mechanism that

keeps the

upper body upright

CO2 canister

The electronics,

provides on/off

signals for the

muscles

One foot

switch

underneath

each foot

One of the four

artificial muscles

Empty bucket

Skateboard-truck-

like ankle joints

add to lateral stability

Figure 2: Delft pneumatic biped.

provide a torque across the hip joint to power thewalking motion.

The muscles are fed with CO2 from a 58atm cannister, pressure-reduced in two steps to6 atm through locally developed miniature pneu-matics. Low-power, two-state valves from SMCPneumatics©R connect the artificial muscles eitherto the 6 atm supply pressure or to 0 atm. The calcu-lation ofcmt = 0.08 for the Delft biped, used in themain paper, is based on actuator work (measuringthe force-length relation of the muscles at the op-

erating pressure). It does not take into account thehuge (but inessential) losses from stepping downthe gas pressure. To find a value forcet, we calcu-lated the decrease ofavailable energy(or exergy)for a pressure drop from the 58 atm saturated liq-uid state to atmospheric pressure. Available energyrepresents the amount of work that could be donewith the pressurized gas if the both the gas expan-sion process and the simultaneous heat transfer pro-cess are reversible (i.e. lossless). In that hypothet-ical setting, one can use theenthalpyandentropyvalues for the gas at the beginning and the end ofthe expansion process. At a constant temperatureof 290 K, this amounts to a loss of available en-ergy of 664 kJ per kg CO2. A 0.45 kg canistercan power the 8 kg robot for 30 min of walkingat 0.4 m/s yieldingcet = 5.3. This value has lit-tle meaning, however. First, even the best real-world gas-expansion systems can only use about30% of the theoretically available energy, due toirreversibility issues. More importantly, most ofthe expansion loss would be eliminated if the CO2

had been stored at 6 atm. Unfortunately this wouldrequire an impractically large storage tank. Thusthe discrepancy betweencet = 5.3 andcmt = 0.08is due to practical problems associated with usingcompressed-gas energy storage.

McKibben muscles have a low stiffness whenunactuated, leaving the joints to behave almost pas-sively at zero pressure. At higher pressures, theMcKibben muscles behave as progressively stiffersprings. By activating opposing muscles in differ-ent proportions, the relaxed angle of a joint can becontrolled. This is applied at the hip where the ar-tificial muscles alternate in action. At the start ofeach step, determined by a foot switch, one mus-cle is set to 6 atm and the other to 0 atm. Theswing leg is thus accelerated forward until the re-laxed angle of the hip is reached, where it (approxi-mately) stays due to damping in the muscles and inthe joint. If sufficient hip joint stiffness is obtainedfrom the hip muscles, stable walking similar to thatof McGeer’s four-legged machine can be obtained.

4

The upper body is kept upright via a kinematic re-striction, a chain mechanism at the hip which con-fines the upper body to the bisection angle of thetwo legs (Wisse, Hobbelen and Schwab, 2005).

Lateral stability in two-legged robots can be ob-tained in a number of ways (Kuo, 1999), and onesolution was tested in the Delft robot. The feet areattached to the lower leg via special ankle joints(Wisse and Schwab, 2005) which have a joint axisthat runs from above the heel down through themiddle of the foot, quite unlike the human anklebut much like skateboard trucks. The mechanismcreates a nonholonomic constraint, which can en-able stability without dissipation, as found in skate-boards (Hubbard, 1979). If the robot starts to leansideways as a result of a disturbance, the ankle al-lows the foot to remain flat on the floor. Due to thetilted joint orientation, the leaning is accompaniedby steering. If the walker has sufficient forwardvelocity, this steering helps prevent its falling side-ways, much like the turning of a bike wheel into afall helps prevent a bike from falling down.

A Universal Processor Board from MultiMotions©R (based on the Microchip©R PIC16F877micro-controller) uses foot contact-switch signalsto open or close the pneumatic valves. The controlprogram is a state machine with two states: eitherthe left or the right leg is in swing phase. At thebeginning of the swing phase, the swing knee isbent. Four hundred milliseconds after the start ofthe swing phase, the knee latch is closed, waitingfor the lower leg to reach full extension through itspassive swing motion. Programmed in assembly,this amounts to about 30 lines of code. The onlysensing is the time of foot contact, used once perstep. Taking account of the implicit rounding fromthe processor loop time, we estimate the sensor in-formation flow rate is about six bits per second.

The Delft powered biped first walked success-fully in July 2004. When mechanically sound, mostof the manual launches (by an experienced person)result in a steady walk. Falls can often be attributedto disturbances from within the machine (a contactswitch that performs unreliably, or a cable that gets

stuck between parts), and occasionally to floor ir-regularities. Another problem is that the pneumaticand mechanical systems (which were developed atDelft for a proof-of-principle prototype rather thanan industrial-strength product) have frequent me-chanical failures that often need a day or more tofix. At present the machine is being kept workingso it can repeat the behavior shown in movie S2.

Figure 3: The MIT learning biped

1.3 MIT learning biped.

First we duplicated the Wilson design (Fig. 1a ofmain paper) using two rigid bodies connected by asimple hinge. The kneeless morphology was cho-

5

sen to reduce the number of joints and actuatorson the robot, minimizing the combinatorial explo-sion of states and control strategies that the learningalgorithm needed to consider. The gait was itera-tively improved in simulation by changing the footshape for a given leg length, hip width, and massdistribution. The resulting ramp-walker (Fig. 1bof main paper) walks smoothly down a variety ofslopes. The powered version uses tilt sensors, rategyros, and potentiometers at each joint to estimatethe robot’s state, and servo motors to actuate theankles. The completed robot weighs 2.75 kg, is43cm tall, and has 6 internal degrees of freedom(each leg has one at the hip and two at the ankle).Before adding power or control, we verified thatthis robot could walk stably downhill with the an-kle joints locked.

The robot’s control code runs at 200Hz on anembedded PC-104 Linux computer. The robot runsautonomously; the computer and motors are pow-ered by lithium-polymer battery packs, and com-munication is provided by wireless ethernet. Thiscommunication allows us to start and stop the robotremotely; all of the control algorithms are run onthe onboard computer.

The learning controller, represented using a lin-ear combination of local nonlinear basis functions,takes the body angle and angular velocity as inputsand generates target angles for the ankle servo mo-tors as outputs. The learning cost function quadrat-ically penalizes deviation from the dead-beat con-troller on the return map, evaluated at the pointwhere the robot transfers support from the leftfoot to the right foot. Eligibility was accumu-lated evenly over each step, and discounted heavily(γ ≤ 0.2) between steps. The learning algorithmalso constructs a coarse estimate of the value func-tion, using a function approximator with only an-gular velocity as input and the expected reward asoutput. This function was evaluated and updated ateach crossing of the return map.

Before learning, outputs of both the control pol-icy and the value estimate were zero everywhereregardless of the inputs, and the robot was able to

walk stably down a ramp; because it is simulatingpassive-dynamic walking, this controller runs outof energy when walking on a level surface. Therobot kicks itself into a random starting positionusing a hand-designed control script to initializethe learning trials. The learning algorithm quicklyand reliably finds a controller to stabilize the de-sired gait on level terrain. Without the value esti-mate, learning was extremely slow. After a learningtrial, if we reset the policy parameters and leave thevalue estimate parameters intact, then on the nexttrial the learning system obtains good performancein just a few steps, and converges in about two min-utes.

The resulting controller outputs ankle com-mands that are a simple, time-independent func-tion of the state of the robot, and does not requireany dynamic models, nor high-precision, high-frequency actuation. All learning trials were car-ried out on the physical biped with no offline sim-ulations. The learned controller is quantifiably (us-ing the eigenvalues of the return map) more stablethan any controller we were able to design by hand,and recovers from most perturbations in as little asone step. The robot continually learns and adaptsto the terrain as it walks.

The MIT biped, which was not optimized for en-ergy efficiency, hascet = 10.5, as calculated by theenergy put back into the batteries by the rechargerafter 30 minutes of walking. Thecet for this robotis especially high because the robot has a powerfulcomputer (700 MHz Pentium) on a light robot thatwalks slowly.

The version of the MIT powered biped shownhere first walked successfully in January 2004. Theearliest powered prototype of this type at MIT firstwalked successfully in June 2003.

The MIT biped is still working well, and is thesubject of active development and study. Newlearning algorithms and new design elements (suchas different curvatures in the feet) are being testedwith the same hardware. A new version with kneesis mostly developed. The robot has walked fora few one-hour on-the-treadmill energy-use trials

6

(the batteries would have lasted for about 90-100minutes).

2 Analogy with first poweredflight*

On December 17, 1903 the Wright brothers firstflew a heavier-than-air man-carrying powered ma-chine. There are various parallels between theirmachine and the simply-powered low-energy-usewalking robots described here.

Starting from before the work began, theWright’s were inspired by flying toys. The walk-ing machines here were also inspired by, and evenpartially based on, walking toys.

The Wright’s ideas about control of steer in air-craft were based on the relation between steer andlean in bicycles. Our research in the passive bal-ance of robots was inspired by the self-stability ofbicycles.

The Wrights worked for years developing glid-ers, planes powered by the release of gravitationalpotential energy as they flew down a glide slope.This was in contradiction to a common paradigmof the time, which was to try to get a powered planeto work, motor and all, all at once. Once they hadmastered gliding they were confident they couldmaster powered flight. On the second day they triedthe idea, adding a primitive engine to a glider de-sign, they made their famous flight. Our develop-ment of passive-dynamic walkers, robots that walkdown gentle slopes powered only by gravity, wasby far the bulk of our efforts. Once we had thoseworking well we were confident that the machinescould walk on the level with a small addition ofpower. That result, that adding power to a downhillmachine works, is the subject of the present paper.

The analogy above is not accidental. TadMcGeer, the pioneer of passive-dynamic robotics,was trained as an aeronautical engineer. McGeer’sforay into robotics was directly and explicitly animitation of the Wright Brothers paradigm. Itworked for the Wrights after others failed at mas-

tering power and flight all at once. Perhaps,McGeer thought, it could work for the more pedes-trian task of making an low-energy-use walkingrobot. McGeer put aside the project after makingsignificant progress with passive robots (walkingrobot gliders), returning to the world of airplane de-sign. Our research has been, more or less, to pickup where McGeer left off, improving the ‘gliders’and then adding simple power.

* The analogy has its limits. Heavier-than-airpowered flight was a well-defined major goal over along period of time with huge consequences. Thataccomplishment swamps anything that might hap-pen with robotics, including this research. TheWright analogy does not extend to the significanceof our work, which is hugely less.

References for supplementary mate-rial

S. H. Collins, M. Wisse, A. Ruina,Int. J. Robot.Res.20, 607 (2001).

G. T. Fallis, Walking toy, Patent, U.S.Patent Office (1888). Available athttp://www.tam.cornell.edu/.ruina/hplab.

M. Hubbard,J. Appl. Mech.46, 931 (1979).

A. D. Kuo, J. Biomech. Eng.124, 113 (2002).

A. D. Kuo, Int. J. Robot. Res.18, 917 (1999).

M. Wisse, A.L. Schwab,Int. J. Robot. Res., inpress (2005)

M. Wisse, D.G.E. Hobbelen, A.L. Schwab,IEEET. Robot., in press (2005)

7