Adaptive Foothold Selection for a Hexapod

Robot Walking on Rough Terrain

D. Belter ∗

∗ Institute of Control and Information Engineering,Poznan University of Technology, Poznan, Poland

(e-mail: Dominik.Belter@put.poznan.pl).

Abstract: This paper introduces an adaptive foothold selection algorithm for a legged robotwalking on a rough terrain. The proposed algorithm searches for the best footholds to preventslippages. It uses a known grid map of the surrounding terrain and a polynomial-basedapproximation method to create a decision surface. The robot learns from experiments, thereforeno a priori expert’s knowledge is required. The results of simulations show that the method isgeneral enough to work properly on various types of terrain. These results have been also verifiedon the real walking robot ’Ragno’.

Keywords: walking robot, motion planning, learning, approximation

1. INTRODUCTION

1.1 Problem statement

Walking robots are in the area of high interest because oftheir potential abilities to deal with a rough terrain. Theycan be successfully applied in search and rescue missions.However, walking on an unstructured terrain requires anappropriate control algorithm. In such an algorithm thereare three main problems to deal with: foothold selection,static and dynamic stability, and path planning. Thefoothold selection method is crucial, because whenever thefeet are placed improperly on the ground the probabilityof slippage and the risk of fall are much higher. On theother hand, a careful foothold selection simplifies stabilitymaintenance.

The problem of foothold selection has been consideredmany times. There are many walking robots with correctlyworking foothold selection systems. However, these sys-tems are usually based on a specific domain knowledge,with a large number of parameters that should be set bya human expert (Rebula et al. (2007)). Hence, they arehardly adaptable for another walking robots or differenttypes of gaits and terrains. In contrast, the algorithm pro-posed in this paper allows to create a ground scorer, whichis shaped automatically, by using results of experimentsconducted on the real robot or it’s simulated model. Ahuman expert is not necessary in the learning process, asthe robot learns unsupervised how to walk on a difficultterrain. This concept gives the robot a possibility to self-adapt it’s foothold selection algorithm to various terrainproperties.

1.2 Related work

To generate a path and select footholds for a walkingrobot it is necessary to build a model of the environment.⋆ This work has been funded by MNiSW grant no. N514 294635 inyears 2008-2010.

Fig. 1. Ragno the robot during tests on a rough terrainmockup.

The robot Lauron is an example of a walking machine,which is able to acquire a map of unstructured environ-ment (Gassmann et al. (2003), Roennau et al. (2009)). Ituses an occupancy and a credibility map to make decisionabout the appropriate footholds. The chosen position of afoot depends on the value of credibility and the distancefrom the center of the local map (which is close to the footposition for the “normal” walking behavior). Also Chenand Kumar (1996) use a special map characterizing theperception uncertainty to plan the motion of a roboton a rough terrain. Based on this information a gait isplanned by using the Ordinal Optimization method. Thequadruped LittleDog robot has been equipped with aterrain scorer/classifier (Rebula et al. (2007)). It evaluatespotential foot placement at a given point as acceptableor unacceptable, taking into account the elevation of theconsidered point and its neighboring points. The scorerrejects points which are located on too large slopes, aretoo close to the top edge or the base of a cliff, or are insideof a hole. The robot uses a previously acquired map of thesurroundings to compute a fine grid of scored terrain.

Fig. 2. Simulated robot.

Proper foothold selection may be also based on quite dif-ferent principles. For example, a controller that adapts tofoot slippages by using force control has been proposed insimulation by Prajoux and Martins (1996). The adaptivecontroller of a robot for walking on irregular terrain canbe also based on the Central Pattern Generators (CPG)and a system of reflexes (Fukuoka et al. (2003)). As aresult a dynamically stable gait on an irregular surfacehas been demonstrated. To improve the agility of therobot on challenging surfaces a mechanical feedback canbe used as well, instead of adding sensors or changing thecontrol system. A mechanical structure of the RHex robotwas inspired by the physiology of animals (Spagna et al.(2007)).

2. THE ROBOT AND THE SIMULATOR

The hexapod walking robot Ragno (cf. Fig. 1) is used inthis research. Each leg has three joints that are drivenby integrated servomotors. The robot weights 2.155 kgwithout batteries and can fit in a box of 33 × 30 cm.Its mechanical structure allows to walk with the trunkfrom 15 to about 70 mm over a ground. The robotis equipped with various types of sensors including theMEMS accelerometers that form an Inertial MeasurementsUnit (IMU) to measure the roll and pitch angles. Aninterested reader can find more information in (Walas et al.(2008)).

The simulator is based on the Open Dynamics En-gine (ODE). This library allows to simulate rigid bodydynamics. It implements methods for modeling variousjoints with friction and softness and for detecting colli-sions (Smith (2009)). For object visualization the OpenGLroutines are used. The robot is represented by simplerigid objects connected with rotary joints (cf. Fig. 2). Themodel of the robot is designed to be simple because ofthe simulation speed. The parameters of the simulatedmodel are optimized by using an evolutionary algorithm,which minimizes discrepancies between the real and thesimulated robot (Belter and Skrzypczynski (2009)). Thissimulator has been already successfully used to evolvefeasible gaits for the Ragno robot (Belter et al. (2008)).Servomotors are simulated as simple proportional con-trollers. The maximal torque for servomotors is limited to0.9 Nm, as in the servomotors in the real robot. Becausethis torque produces a high joint velocity in simulator,its absolute value is limited to vmax = 4.29rad/s. For highaccuracy the time step of the ODE solver was set to 1e-4 s.

z

z

z

z

z

z

i,ji+1,j+1

i-1,j-1

i-1,j

i-1,j+1

i,j-1

i,j+1

i+1,j

i+1,j-1

zz

z

x

yz

0

Fig. 3. Computation of K1 and K2 coefficients.

3. USING AN ADAPTIVE POLYNOMIAL FORDECISION MAKING

3.1 Control strategy

While walking on the rough terrain the robot uses atripod gait (Belter et al. (2008)). It is the fastest staticallystable gait for a hexapod walking robot. During the stancephase the robot maintains a constant average height of thetrunk above the ground. This average height is computedby using the last six footholds. To stabilize the robotorientation during walking the information about pitchand roll angles from the IMU are used.

At the end of the swing phase of the gait the robot searchesfor three new footholds. At first, it checks if the potentialfootholds are in the workspace of the considered leg. Ifthere are no appropriate footholds in the reach of one ofthe legs, the robot changes the height of it’s trunk to reacha proper support point. If this strategy fails the controlsystem informs the path planner module that the plannedmovement is not achievable. The problem is being solvedby the higher layers of the control system, what is outof the scope of this paper. However, in the experimentspresented here the reactions which change robot’s posturewere sufficient to deal with the problem.

3.2 Ground properties

The presented algorithm uses a known grid map of thesurrounding terrain and computes four coefficients whichdescribe the potential footholds. For the i and j grid co-ordinates the K1 coefficient is defined as follows:

K1(i, j) =

1∑

k=−1

1∑

l=−1

(zi,j − zi+k,j+l), (1)

where zi,j and zi+k,j+l are terrain elevations of the corre-sponding points of the grid map (Fig. 3). This coefficientallows to detect a top edge or a hole. For a top edge K1

is positive and for a hole K1 is negative. The value of K1

gives information about the local terrain extreme, howeverK1 is ambiguous when it’s value is 0. Flat terrains as wellas a terrain with constant slope give the same results.

The K2 coefficient for i and j grid references is defined asfollows:

K2(i, j) =

1∑

k=−1

1∑

l=−1

|zi,j − zi+k,j+l|. (2)

n

ss

s

s

s1

2

3

4

5f

α

x

y

Fig. 4. Computation of K3 coefficient.

The K2 coefficient provides an information about the slopeof a terrain. It returns different results for a constantslope and for a flat terrain. On the other hand, topedges and holes give the same results. Thus, K1 andK2 are ambiguous separately, but together they givea fundamental information about the structure of theterrain.

The K3 coefficient is defined by an angle α between avector f describing the foot movement and a vector nnormal to the surface. The normal vector n is computedonly for a part of the surface as it was shown in Fig. 4. Itdepends on the projection of the vector f on the surface xy.Tangent vectors s1, ..., s5 are computed for points whichneighbor to the quadrant where projection of the vector fis located. Next the vector n is computed as follows:

n =

4∑

i=1

si × si+1. (3)

The angle α between the vectors n and f is computed asfollows:

α = K3 = arccos

(

n · f

|n||f |

)

. (4)

When angle α is close to π the robot’s foot has a goodsupport for movement. When α is small there is a highprobability of slippage.

The K4 coefficient is computed as a distance between theconsidered point and the point being the center of thelocal map (foothold for the “normal” walking behavior(x0,y0,z0)):

K4(i, j) =√

(x0 − xi,j)2 + (y0 − yi,j)2. (5)

4. THE ALGORITHM

The aim of the algorithm is to evaluate potential footholdsand to choose the best one. The best point for a footstance should minimize a risk of slippage. Slippage ismeasured by the distance between the position of thefoot at the beginning and at the end of the stance phasedivided by |fi|. The fi vector defines a desired i-th footmovement during a stance phase. The control algorithmuses kinematics of the robot and generates the trunkmovement by computing fi – particular movements of therobot’s feet.

To evaluate a potential foot placement a grid map ofthe terrain with 5mm×5mm grid cells is used. In thepresented simulations and experiments the terrain map is

a) b)

d)

f)

c)

e) g)

Foothold

Fig. 5. The ground primitives.

given a priori, but this method can be used with an on-line terrain mapping system utilizing a laser scanner or astructured light sensor. For the considered points of themap the coefficients K1,K2,K3, and K4 are computed.The algorithm consists of three steps:

(1) collecting data,(2) learning – approximation,(3) regular operation.

4.1 Collecting data

At first, the robot collects the appropriate data for learn-ing. Two strategies of data acquisition were investigated.In the first one the robot walks on a rough terrain, ran-domly selects footholds, saves coefficients correspondingto these points, and as a result of this selection – a footslippage measured after the stance phase. In the secondapproach the robot tests seven types of previously pre-pared ”ground primitives” (shown in Fig. 5). Each of therobot’s foot is placed on the same primitive. The robotexecutes a movement, and after that a slippage of each footis registered. This experiment is repeated for all primitives.To test different ground types the height of the primitivesis changed during the acquisition phase in the range of4 cm, with a step of 1 cm, and the primitives are rotatedwith the step of 45◦. After this stage the robot has anappropriate data set to build the decision surface anddistinguish between the good and the poor footholds.

The point cloud shown in Fig. 6 was obtained in a situ-ation when only two coefficients were used (K1 and K2),and when the primitives were used for data acquisition.Whenever the input space has a higher dimensionality itis not possible to show the results graphically. It is difficultto deduce information about usefulness of footholds froma raw point cloud. The data are very often ambiguous.Selection of the same points (defined by the same co–ordinates K1 and K2) gives different slippage values. Thisis due to the closed kinematic chains made by robot’s legs.Footholds are evaluated locally, for individual legs, butthe slippage for selected points depends also on footholdsof the remaining legs. If one foot skids the forces are trans-mitted through the whole robot body and cause slippagein the contact points of other feet.

4.2 Learning

In the second phase the robot forms it’s decision surface.Learning is understood here in a broader sense, as devel-

-0.1

0.1

0.3

0

0.1

0.2

0.3

0

1

2

3

K1 [m]K2 [m]

Q

acquired data

output data pf

Fig. 6. Collected data and filtration effect.

opment of a decision surface on the basis of the terraincharacteristics. The learning phase provides an adaptationmechanism, which incorporates the knowledge gathered bythe robot into it’s control system. To deduce an usefulinformation from the point cloud a discretization in a fixedgrid is used. The input space K1, K2 and K3 is divided intoa regular grid (Fig. 6 for a situation when only K1 and K2

are used). For points whose projections are located in theconsidered grid cell a mean value is computed. The resultis placed in the center of the considered grid cell. After thisdiscretization a group of points pf = [pf1, ..., pfk]T whichrepresent a general trend is obtained. For further purposesonly these points are stored. The previous point cloud isremoved from the memory.

A foothold selection method should be general, but it isnot possible to test all types of terrains. However, thealgorithm should have the ability to judge the usefulnessof the ground points, which have not been tested explic-itly in the data acquisition stage. To this end a least-squares polynomial approximation is used (Dahlquist andBjorck (1974)). The points are no longer needed, becausethe knowledge is stored in the approximation polynomial.The approximation assignment requires a selection of anappropriate polynomial base:

P (K1,K2, ...,Kn) =

m∑

i=0

ci · φi(K1,K2, ...,Kn), (6)

where K1,K2, ...,Kn are coefficients which characterizethe terrain. The polynomial P allows to assess potentialfootholds represented by values of the K1, ...,Kn coef-ficients. To determine the ci values, the Least-SquaresFitting method is applied on the set of pf points obtainedin the data acquisition stage. The method uses the Grammatrix G:

G =VT · V, (7)

c =G−1 · VT · r, (8)

where

V =

[

φ1(pf1) φ2(pf1) ... φm(pf1)... ... ... ...

φ1(pfk) φ2(pfk) ... φm(pfk)

]

, (9)

is a Vandermonde matrix, and r is a vector of slippagescorresponding to the respective pf points.

Unfortunately, there are no general rules for polynomialbase selection. A knowledge about the approximated phe-nomenon is very useful, although in this case it is not avail-

-0.2

-0.1

0

0.1

0.2

0.3

0

0.1

0.2

0.3

0

1

2

3

K1K2

Q

aproximation surface

aproximation points

Fig. 7. Decision surface.

able. To solve this problem the population based Evolu-tionary Algorithm (EA) (Annunziato and Pizzuti (2000))has been applied to obtain the φi components of thepolynomial base, which are defined as:

φi(K1,K2, ...,Kn) = f1(K1) · f2(K2)·, ..., ·fn(Kn), (10)

where the functions f1,...,fn are chosen from the followingcandidates:

(Ki − ai)round(bi), (11)

sin(bi · Ki − ai), (12)

e−bi(Ki−ai), (13)

where the round() method rounds a real number to thenearest integer. The function (11) is a component of the n-th degree polynomial basis (Dahlquist and Bjorck (1974)).The functions (12) and (13) are added to increase thegenetic diversity, The experience gathered from a largenumber of EA runs suggests that such a diversity mini-mizes the risk that the Gram matrix G is ill-conditioned.

The values of ai, bi parameters, and the type of thefunction fi are determined by using EA. The EA is usedbecause it is able to self-adapt its own parameters duringthe search, what minimizes the number of parameters thathave to be set manually. The number of elements m inthe polynomial (6) is a design parameter. It might bedetermined automatically, e.g. by using the Akaike Infor-mation Criterion, however it is not used in this case. Fixedm makes an implementation of the EA much simpler,while the EA itself eliminates unnecessary elements of thepolynomial by setting their corresponding bi parametersto zero. The number of elements m in the equation (6)defines a number of genes in the chromosome. A singlegene contains information about one function (10). Whena single gene is mutated all values of the ai and bi pa-rameters are randomly changed and the type of functionfi is chosen from {(11),(12),(13)}. For reproduction thetwo–point crossover is used. When the EA optimizationstarts, only the boundaries for ai, bi and the number ofelements m have to be defined. The experiments showthat the parameter m should be small to ensure properquality of approximation and model complexity. The bi

parameters should be positive and rather small. When mand bi parameters are too big the approximation functionis not smooth between the approximation points pf . The

-0.04

0

0.04-0.04

-0.020

0.020.04

-0.01

0.01

0.03

0.05

y [m]

x [m]

z [m

]

surface

out of range

weak foothold

assessment

minima

Fig. 8. Terrain and assessment results.

search space boundaries for particular ai are set taking intoaccount the values of its corresponding Ki parameter. Thefitness function in the EA is defined as a sum of distancesbetween pf points and the corresponding points of thesurface (6).

The obtained approximation polynomial and approxima-tion points pf are shown in Fig. 7 (only K1 and K2 ininput). According to the obtained decision surface thebest potential footholds are holes (negative K1 and high,positive value of K2). Flat surfaces are also evaluatedas useful. The system learns to avoid local peaks (high,positive values of K1 and K2). Also deep depressions arenot good for foot support.

4.3 Regular operation

During the third phase the robot uses the approximatedpolynomial to assess the potential footholds. Then, theseresults are used to find the best place to put the foot onit. Such a system allows to cover successfully a demandingterrain and to avoid slippages.

The first three coefficients are used as an input to computethe polynomial (6). The fourth one (K4) is used to excludepoints on the ground which are too close to the boundariesof the robot’s leg workspace. The final Q(i, j) coefficientwhich describes the usefulness of the potential footholdgiven by K1,..,K4 features is given as:

Q(i, j) = P (K1(i, j),K2(i, j),K3(i, j))+k ·K4(i, j), (14)

The tuning constant k has been set to 8. When this valueis too big the robot chooses points which are in the centerof the local map. When it is too small, there is a risk thatthe robot will choose points which are very close to theboundaries of the leg’s workspace, what might render themovement impossible.

5. RESULTS

The algorithm has been tested on a specially preparedterrain map. It includes obstacles similar to scatteredflagstones, extensive hills with mild slopes, small andpointed hills, depressions and stones. The highest pointof the terrain is 8.7 cm high, and the deepest depressionis -0.35 cm.

During walking the robot uses the standard gait forwalking on the flat terrain. This gait gives an expected

0 50 100 150 200 250 300 3500

0.1

0.2

0.3

0.4

0.5

step

sum

of slip

page

on flat terrain

randomly selected footholds (collecting data)

test after collecting data(K1,K

2,K

3)

test - primitives(K1,K

2)

test - primitives(K1,K

2,K

3)

Fig. 9. Results - slippage.

foothold for each leg of the robot. Around the expectedfoothold a local grid map is defined. It’s size is set to 15×15cells (7.5×7.5 cm). The computed polynomial is then usedto evaluate all the points of this local map. An example ofthe local map with the assessment indicated is shown inFig. 8. There are cells which are excluded from the set ofthe potential footholds because they are out of the rangeof the robot’s legs. Cells are also excluded when the valueof Q is to big (such cells give a weak support to a leg’stip) or a cell is described by Ki values which are outsidethe boundaries defined in the learning phase (properties ofthis cell are unknown). The foothold selection algorithmsearches for the minimum among the rest of the cells. Theshorter is the bar shown in Fig. 8 the more useful is thegiven cell as a foothold. After a decision about footholdselection the control system of the robot modifies foot’strajectories of the feet and places them in the selectedpoints.

Slippages during the whole simulation run have beenrecorded to show results given by the proposed algorithm(Fig. 9). Each experiment consists of a series of three tests.Mean value of the accumulated slippage and its standarddeviation are shown. These results are compared to theresults of a simulation on a flat terrain, and a simulationin which the robot randomly selects footholds. Betterresults are obtained when a point cloud acquired usingthe prepared primitives is used. Data obtained duringexperiments on the real terrain are often ambiguous anddisturbed. The results when three coefficients are used fordecision making are slightly better. When only K1 andK2 coefficients are used the robot has a problem withextensive hills of mild slope.

Paths of the robot’s center of gravity during the simula-tions are shown in Fig. 10. The desired path is a 1 m×1 msquare. The robot starts the movement in the center of thesquare. The control system generates motion commandswithout any feedback information about the global posi-tion of the robot. It uses only the information about theorientation of the robot’s platform (pitch and roll).

The registered paths show that even on a flat terrainthe robot has difficulties to follow the nominal path.The same phenomenon is observed with the real robot.The path performed during collecting data on a rough

-1 -0.5 0 0.5 1-0.8

-0.4

0

0.4

0.8

x [m]

y [m

]

desired path

on flat terrain

test - primitives(K1,K

2)

test - primitives(K1,K

2,K

3)

START

FINISH

Fig. 10. Results – trajectories.

terrain is irregular and bent. Slippages of the feet changerobot’s position and orientation in the global coordinates.After the learning phase the robot’s trajectory is gettingsmooth. When the robot is walking straight it covers adesired distance. However it has difficulties while turning.The gait generation method assumes that a foot canchange orientation on the spot. But when the robot selectssmall holes as footholds (what prevents slippage) thisassumption does not hold in many cases, what causesorientation errors, particularly while turning. However,orientation errors of the whole robot can be corrected by aself-localization algorithm (Gassmann et al. (2005)), whilepoints with high adherence should be chosen to preventsudden fall-downs on an uneven ground. A close-up ofsuch a local disturbance of the path caused by extensiveslippage is shown in the enlarged part of Fig. 10.

6. CONCLUSIONS AND FUTURE WORK

This paper presents a method for creating a foothold selec-tion system for a multi-legged walking robot. The systemlearns automatically without any experts knowledge. Therobot acquires the ground surface characteristics by walk-ing on an example terrain or by testing ground primitives.Then it exploits this knowledge to build a decision surface,which is used for assessment of the candidate footholds.

The simulations show that after learning the robot is ableto select the appropriate footholds to prevent slippages.The robot has learned to avoid peaks of the terrain and toselect such points on the ground which give an appropriatesupport. This behavior is general. The same rules work ondifferent types of terrain. The obtained decision unit hasbeen tested on a map of the real terrain and verified onthe real robot Ragno 1 .

We are going to test the described method on our new,bigger hexapod robot ’Messor’, which is equipped with alaser scanner for on-line terrain mapping. We plan alsoto modify the selection algorithm by considering the Ki

coefficients for all robot’s legs at the same time.

1 A video is available on http://lrm.cie.put.poznan.pl/acd09.avi

REFERENCES

Annunziato, M., Pizzuti, S. (2000) Adaptive Parameter-ization of Evolutionary Algorithms Driven by Repro-duction and Competition. In Proc. ESIT 2000, vol. 1,pp. 31–35, Aachen.

Belter, D., Kasinski, A., Skrzypczynski, P. (2008) Evolvingfeasible gaits for a hexapod robot by reducing the spaceof possible solutions. In Proc. IEEE Int. Conf. onIntelligent Robots and Systems, pages 2673–2678, Nice,France.

Belter, Skrzypczynski, P. (2009) Efficient Gait Learning inSimulation: Crossing the Reality Gap by EvolutionaryModel Identification. In Mobile Robotics: Solutions andChallenges (O. Tosun et al., eds.), Singapore, WorldScientific 2009, 861–868.

Chen, C.H., Kumar, V. (1996) Motion planning of walkingrobots in environments with uncertainty. In Proc. IEEEInt. Conf. on Robotics and Automation, pages 3277–3282, Minneapolis, USA.

Dahlquist, G., Bjorck, A. (1974) Numerical Methods. Se-ries in Automatic Computing, Prentice Hall, EnglewoodCliffs, New Jersey.

Fukuoka, Y., Kimura, H., Cohen, A.H. (2003) Adaptivedynamic walking of a quadruped robot on irregularterrain based on biological concepts. Int. Journal ofRobotics Research, 22(3–4):187–202.

Gassmann, B., Frommberger, L., Dillmann, R., Berns,K. (2003) Real-time 3d map building for local navi-gation of a walking robot in unstructured terrain. InProc. IEEE Int. Conf. on Intelligent Robots and Sys-tems, pages 2185–2190, Las Vegas, USA.

Gassmann, B., Zacharias, F., Zoellner J., Dillmann R.(2005) Localization of walking robots. In Proc. IEEEInt. Conf. on Robotics and Automation, pages 1471–1476, Barcelona, Spain.

Prajoux, R., L. de S.F. Martins, F. (1996) A walk su-pervisor architecture for autonomous four-legged robotsembedding real-time decisionmaking. In Proc. IEEEInt. Conf. on Intelligent Robots and Systems, pages 200–207, Osaka, Japan.

Rebula, J. R., Neuhaus, P. D., Bonnlander, B. V., Johnson,M. J., Pratt, J. E. (2007) A controller for the littledogquadruped walking on rough terrain. In Proc. IEEEInt. Conf. on Robotics and Automation, pages 1467–1473, Roma, Italy.

Roennau, A., Kerscher, T., Ziegenmeyer, M., Zoellner,J. M., Dillmann R. (2009) Six-legged walking in roughterrain based on foot point planning. In Mobile Robotics:Solutions and Challenges (O. Tosun et al., eds.), Singa-pore, World Scientific 2009, 591–598.

Smith, R. (2009) Open dynamics engine, www.ode.org.Spagna, J. C. Goldman, D. I., Lin, P. C., Koditschek,

D. E., Full, R. J. (2007) Distributed mechanical feed-back in arthropods and robots simplifies control ofrapid running on challenging terrain. Bioinspiration andBiomimetics, 2(1):9–18.

Walas, K., Belter, D., Kasinski, A. (2008) Control and en-vironment sensing system for a six-legged robot. Journalof Automation, Mobile Robotics and Intelligent Systems,2(3):26–31.