Summer Semester Project 2004:

Aibo Simulation in Webots and

Controller Transfer to Aibo Robot

Lukas Hohl, semester 8

BIRG, Logic Systems Laboratory (LSL)School of Computer and Communication SciencesSwiss Federal Institute of Technology Lausanne

Supervision:Prof. Auke Jan Ijspeert & Dr. Olivier Michel

June 18, 2004

Contents

List of Figures 3

List of Tables 4

1 Introduction 6

2 System Overview 82.1 Webots by Cyberbotics . . . . . . . . . . . . . . . . . . . . . . . 82.2 Extension of Webots . . . . . . . . . . . . . . . . . . . . . . . . . 8

3 Simulation 103.1 Physical Model of Aibo ERS-210 . . . . . . . . . . . . . . . . . . 10

3.1.1 Robot Blocks . . . . . . . . . . . . . . . . . . . . . . . . . 103.1.2 Robot Primitives . . . . . . . . . . . . . . . . . . . . . . . 113.1.3 Webots Model . . . . . . . . . . . . . . . . . . . . . . . . 12

LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16Joints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19Plungers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.2 wxWindows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.3 Graphical User Interface . . . . . . . . . . . . . . . . . . . . . . . 23

3.3.1 Networking, Shutdown and Reboot . . . . . . . . . . . . . 233.3.2 Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.3.3 LEDs and Plungers . . . . . . . . . . . . . . . . . . . . . . 263.3.4 Joints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.3.5 MTN files . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.3.6 Logfile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.4 Controller Program . . . . . . . . . . . . . . . . . . . . . . . . . . 293.5 Possible Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4 Remote Control 314.1 Graphical User Interface . . . . . . . . . . . . . . . . . . . . . . . 31

4.1.1 Initial Position . . . . . . . . . . . . . . . . . . . . . . . . 324.1.2 MTN Playback . . . . . . . . . . . . . . . . . . . . . . . . 32

4.2 Controller Program . . . . . . . . . . . . . . . . . . . . . . . . . . 344.2.1 Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344.2.2 LEDs and Plungers . . . . . . . . . . . . . . . . . . . . . . 35

2

4.2.3 Joints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354.3 Possible Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . 35

5 Cross-Compilation 375.1 OPEN-R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

5.1.1 Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385.1.2 Inter-Object Communication . . . . . . . . . . . . . . . . 38

5.2 Objects on Aibo . . . . . . . . . . . . . . . . . . . . . . . . . . . 395.3 Controller Cross-Compilation . . . . . . . . . . . . . . . . . . . . 39

5.3.1 Webots Robot Controller Program . . . . . . . . . . . . . 395.3.2 Cross-Compilation . . . . . . . . . . . . . . . . . . . . . . 415.3.3 Aibo specific Programming Scheme . . . . . . . . . . . . 445.3.4 Timing, LEDs and Plungers . . . . . . . . . . . . . . . . 465.3.5 Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475.3.6 Servos . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475.3.7 MTN Playback . . . . . . . . . . . . . . . . . . . . . . . 48

5.4 Controller Transfer . . . . . . . . . . . . . . . . . . . . . . . . . . 495.5 Possible Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . 50

Bibliography 50

A MTN File Format 53

3

List of Figures

1.1 Aibo ERS-210 Overview . . . . . . . . . . . . . . . . . . . . . . . 61.2 ScreenShot Aibo Remote Control . . . . . . . . . . . . . . . . . . 7

2.1 System Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.1 Block Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.2 Scene Tree in Webots . . . . . . . . . . . . . . . . . . . . . . . . 153.3 Main Webots Model Nodes . . . . . . . . . . . . . . . . . . . . . 173.4 Webots Model Node Hierarchy . . . . . . . . . . . . . . . . . . . 183.5 Simulated Position Sensing Device . . . . . . . . . . . . . . . . . 183.6 Simulation of Aibo ERS-210 in Webots (with and without Dis-

tanceSensor ray) . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.7 ScreenShot Graphical User Interface (disconnected) . . . . . . . . 243.8 Simulation controlled by Graphical User Interface . . . . . . . . . 25

4.1 Simulation and Robot controlled by Graphical User Interface . . 314.2 JointMover State Diagram . . . . . . . . . . . . . . . . . . . . . . 32

5.1 Inter-Object Communication . . . . . . . . . . . . . . . . . . . . 395.2 Objects on Aibo . . . . . . . . . . . . . . . . . . . . . . . . . . . 405.3 Cross-Compilation . . . . . . . . . . . . . . . . . . . . . . . . . . 42

4

List of Tables

3.1 Block Weights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.2 LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.3 Plungers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.4 Joints (Servos) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.5 Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.6 Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.7 Webots Model Nodes with Bounding Box . . . . . . . . . . . . . 163.8 Transmission of multiple Sensor Values including MTN Key Frame 283.9 MTN Playback Logfile Format . . . . . . . . . . . . . . . . . . . 293.10 Physical Units in MTN Playback Logfile . . . . . . . . . . . . . . 29

4.1 Command Vector Construction by Sony’s MTN Algorithm . . . . 334.2 Command Vector Construction by new MTN Algorithm . . . . . 33

5.1 Implemented Functions for Cross-Compilation . . . . . . . . . . . 435.2 Messages of Type FileCommand . . . . . . . . . . . . . . . . . . 51

5

Chapter 1

Introduction

Aibo is a four-legged robotic dog produced by Sony. Despite the fact that itis principally sold as an entertainment robot system, it has powerful capabili-ties such as wireless network communication, a wide range of input and outputdevices such as speaker and microphone, color camera, distance sensor, acceler-ation sensors, various touch sensors, LEDs and of course controllable joints.

Figure 1.1: Aibo ERS-210 Overview

What makes the robot even more interesting, is the possibility to writecustom programs for Aibo. The binary files are put onto a Memory Stick whichis plugged into Aibo. For software development, Sony provides the OPEN-RSDK and a very complete documentation including sample programs.

One goal of the project was the integration of a standalone computer pro-gram (see figure 1.2) into the Webots mobile robotics simulation software. TheRemote Control System (client software with graphical user interface, softwarecomponents on Aibo, communication protocol) is the result of a previous project[RC] and enables controlling and monitoring of an Aibo ERS-210(A) over a wire-less network connection. It provides detailed control of all joints and sensorsof the Aibo robot and has the capability to load files containing predefinedmovements (MTN files) onto Aibo and to play them back while observing theevolution of the real joint angles. A similar functionality was desired for a

6

simulated Aibo in Webots. One should be able to control the real robot fromWebots, to control the simulation using the same interface and to give com-mands both to Aibo and its model in parallel. The commands would not onlybe given manually through the user interface, but also by Webots controllerprograms running on the client computer. An important aspect of the problemis the construction of a physical model of Aibo. The integration into Webotswould allow the user to analyze the behavior of a real robot in comparison to asimulated Aibo.

Figure 1.2: ScreenShot Aibo Remote Control

The second goal of the project was the cross-compilation of Webots controllerprograms for Aibo. This would allow the user to program the robot not only bythe means of a Webots controller running on the commanding computer, butalso to compile the controller into a binary format that is directly interpretableby Aibo. The robot could then operate independently by executing the programon its Memory Stick and without receiving commands over the wireless networkconnection.

7

Chapter 2

System Overview

2.1 Webots by Cyberbotics

The Webots mobile robotics simulation software provides the user with a rapidprototyping environment for modelling, programming and simulating mobilerobots. Webots includes proven technology that has been co-developed by theSwiss Federal Institute of Technology in Lausanne. With Webots it is possibleto define and modify a complete mobile robotics setup, even several differentrobots sharing the same environment. For each object, a number of propertiescan be defined, such as shape, color, texture, mass, friction, etc. Each robotcan be equipped with a large number of available sensors and actuators. Theuser can then program the robots using an arbitrary development environment,simulate them and optionally transfer the resulting programs onto a real robots.Key features of the Webots version taken as the starting point of the projectare:1

• Models and simulates any mobile robot (wheeled, legged, flying).

• Includes a complete library of sensors and actuators.

• Uses accelerated OpenGL and built-in 3D editor for robots and world with3D model import facilities.

• Uses the Open Dynamics Engine library for accurate physics simulation.

• Lets the user program the robots in C, C++ and Java or from third partysoftware through TCP/IP.

• Transfers controllers to real mobile robots, including Lego Mindstorms,Khepera, Koala and Hemisson.

2.2 Extension of Webots

In the modified Webots software, there are two things that can be controlledby Webots, namely a “real” Aibo ERS-210(A) equipped with an ERA-201-D1wireless LAN card and its physical simulation in Webots (the physical model is

1Visit the web page [Webots] to learn more.

8

stored in the file aibo_ers210.wbt). Commands to either the real or simulatedAibo can be given trough a graphical interface (red arrows on figure 2.1) or by acontroller program which uses Webot’s controller programming interface (purplearrows on figure 2.1). A third mode exist for real robot commanding: Thecross-compilation of Webots controller programs for direct execution on Aibo’sMIPS processor (green arrow on figure 2.1). This mode works independently ofWebots. A graphical interface for the uploading of binary files is provided, butthe files can also be transfered by other means, e.g. a Memory Stick Reader.

Figure 2.1: System Overview

The three main applictions resulting from the extension of Webots (simu-lation controlled by graphical interface and controller, real Aibo controlled bygraphical interface and controller, cross-compilation) are each described in aseparate chapter of this document. We start with the simulation, then we treatthe control of the Aibo robot and finally the cross-compilation will be explained.The three possibilities correspond to the black arrows on figure 2.1.

9

Chapter 3

Simulation

3.1 Physical Model of Aibo ERS-210

3.1.1 Robot Blocks

Aibo can be easily disassembled to a certain degree due to its modular archi-tecture. At the begin of the modeling process, the mass of each module wasmeasured in order to have a first idea of how to construct an Aibo model inWebots. Figure 3.1 shows the main blocks (except the battery) and table 3.1lists their weights.

Figure 3.1: Block Overview

10

Name Mass [g]OPEN-R bus connector 343(includes Memory Stick and ERA-201-D1 wireless LAN card)Battery (not on figure 3.1) 162Head block 265Left front leg block 196Left rear leg block 196Right front leg block 196Right rear leg block 196Tail block 58Total 1612

Table 3.1: Block Weights

3.1.2 Robot Primitives

The Remote Control System developed in the earlier project [RC] gives access tomost of Aibo’s devices. The components (called primitives1 in OPEN-R termi-nology) controllable by the system are given by their name and CPC PrimitiveLocator in the tables listing the primitives in each category (tables 3.2, 3.3, 3.4and 3.5). The CPC Primitive Locator is the unique address of a primitive inOPEN-R programs. There are a few entities which do not correspond to a phys-ical robot component and which do not have a CPC Primitive Locator (table3.6).

For each primitive, it is indicated in the tables, whether the component istaken into account in the current version of the Webots model. Additionalproperties and restrictions are provided for some primitive types. Section 3.1.3explains in more detail, how the model of Aibo was constructed in Webots inorder to achieve a similar level of control for the real robot and its simulation.

Name CPC Primitive Locator SimulationEye light (Lower left) PRM:/r1/c1/c2/c3/l1-LED2:l1

√

Eye light (Middle left) PRM:/r1/c1/c2/c3/l2-LED2:l2√

Eye light (Upper left) PRM:/r1/c1/c2/c3/l3-LED2:l3√

Eye light (Lower right) PRM:/r1/c1/c2/c3/l4-LED2:l4√

Eye light (Middle right) PRM:/r1/c1/c2/c3/l5-LED2:l5√

Eye light (Upper right) PRM:/r1/c1/c2/c3/l6-LED2:l6√

Mode indicator PRM:/r1/c1/c2/c3/l7-LED2:l7√

Tail light (Blue) PRM:/r6/l1-LED2:l1 ∅Tail light (Orange) PRM:/r6/l2-LED2:l2 ∅

LEDs can be either ON or OFF.

Table 3.2: LEDs

Table 3.4 lists various limitations (either specific to a joint or global) forthe position, speed and acceleration of a joint. No indication on speed andacceleration limits is given in [Docs]. There is a [FAQ] entry where at least

1See [Model] for a complete list of primitives and robot schemas.

11

Name CPC Primitive Locator SimulationLeft ear PRM:/r1/c1/c2/c3/e1-Joint3:j5

√

Right ear PRM:/r1/c1/c2/c3/e2-Joint3:j6√

Plungers (joints with two possible positions)can be either ON or OFF.

Table 3.3: Plungers

upper angular speed limits for Aibo’s joints are listed. The lower speed and bothacceleration limits were chosen to maximize consistency, i.e. the combinationof the minimum speed and the maximum acceleration still results in a non-zeroacceleration duration (with respect to the time unit on Aibo).

3.1.3 Webots Model

Simulating complex robotic devices including articulated mechanical parts re-quires precise physics simulation. Webots relies on ODE (Open DynamicsEngine) to perform accurate physics simulation wherever it is necessary. Foreach component of a robot, the user can specify a mass distribution matrix (oruse primitives for simple geometries), static and kinematic friction coefficients,bounciness, etc. Moreover, each component is associated with a bounding ob-ject used for collision detection. Servo devices can be controlled by a controllerprogram in torque, position or velocity.

The scene tree contains all information necessary to describe the graphic rep-resentation and simulation of the 3D world (see figure 3.2). A world in Webotsincludes one or more robots and their environment. The scene tree of Webots isstructured like a VRML97 file. It is composed of a list of nodes, each contain-ing fields. Fields can contain values (text string, numerical values) or nodes.Some nodes in Webots are VRML97 nodes, partially or totally implemented,while others are specific to Webots. Complex robots can be built by assemblingchains of servo nodes. That’s how for instance legged robots with several jointsper leg, robot arms and pan/tilt camera systems can be constructed.

A number of actuators can be plugged into a robot model and even be com-bined hierarchically: Differential wheel motor unit, independent wheel motors,servos (for legs, arms, etc.), LEDs, emitters, grippers. Similarly, a number ofsensors can be plugged into a robot model and tuned individually (range, noise,response, field of view, etc.): Distance sensors (IR and US), range finders, lightsensors, touch sensors, Global Positioning Sensor (GPS), cameras (1D, 2D, color,B&W), receivers, position sensors for servos, incremental encoders for wheels.

Figure 3.3 shows the most important parts of the physical Webots model ofAibo. In the top left corner, there is a 3D wireframe representation of the modelas it is displayed in Webots. The ’name’ field of every node representing a robotprimitive contains the CPC Primitive Locator of the primitive in question and’description’ contains the text in the name column in the tables 3.2, 3.3, 3.4 and3.5. The body is the only part which is not a robot primitive at the same time.The ’boundingObject’ field contains an object with the size of the robot partand the ’mass’ field of the Physics node in the ’physics’ field contains the massof the part. In this model, only objects with a bounding box have a mass (but

12

Iden

tific

atio

nPos

itio

n[d

egs]

Spee

d[d

egs/

s]Si

mul

atio

nN

ame

CP

CP

rim

itiv

eLoc

ator

min

max

init

max

Nec

kti

ltP

RM

:/r1

/c1-

Join

t2:j1

-82.

0043

.00

0.00

120.

625

√

Nec

kpa

nP

RM

:/r1

/c1/

c2-J

oint

2:j2

-89.

6089

.60

0.00

172.

50√

Nec

kro

llP

RM

:/r1

/c1/

c2/c

3-Jo

int2

:j3-2

9.00

29.0

00.

0017

2.50

√

Mou

thP

RM

:/r1

/c1/

c2/c

3/c4

-Joi

nt2:

j4-4

7.00

-3.0

0-3

.00

250.

50√

Lef

tfo

rele

gJ1

PR

M:/

r2/c

1-Jo

int2

:j1-1

17.0

011

7.00

-10.

0016

1.25

√

Lef

tfo

rele

gJ2

PR

M:/

r2/c

1/c2

-Joi

nt2:

j2-1

1.00

89.0

00.

0014

3.12

5√

Lef

tfo

rele

gJ3

PR

M:/

r2/c

1/c2

/c3-

Join

t2:j3

-27.

0014

7.00

30.0

016

2.50

√

Lef

thi

ndle

gJ1

PR

M:/

r3/c

1-Jo

int2

:j1-1

17.0

011

7.00

0.00

161.

25√

Lef

thi

ndle

gJ2

PR

M:/

r3/c

1/c2

-Joi

nt2:

j2-1

1.00

89.0

00.

0014

3.12

5√

Lef

thi

ndle

gJ3

PR

M:/

r3/c

1/c2

/c3-

Join

t2:j3

-27.

0014

7.00

25.0

016

2.50

√

Rig

htfo

rele

gJ1

PR

M:/

r4/c

1-Jo

int2

:j1-1

17.0

011

7.00

-10.

0016

1.25

√

Rig

htfo

rele

gJ2

PR

M:/

r4/c

1/c2

-Joi

nt2:

j2-1

1.00

89.0

00.

0014

3.12

5√

Rig

htfo

rele

gJ3

PR

M:/

r4/c

1/c2

/c3-

Join

t2:j3

-27.

0014

7.00

30.0

016

2.50

√

Rig

hthi

ndle

gJ1

PR

M:/

r5/c

1-Jo

int2

:j1-1

17.0

011

7.00

0.00

161.

25√

Rig

hthi

ndle

gJ2

PR

M:/

r5/c

1/c2

-Joi

nt2:

j2-1

1.00

89.0

00.

0014

3.12

5√

Rig

hthi

ndle

gJ3

PR

M:/

r5/c

1/c2

/c3-

Join

t2:j3

-27.

0014

7.00

25.0

016

2.50

√

Tai

lpa

nP

RM

:/r6

/c1-

Join

t2:j1

-22.

0022

.00

0.00

256.

25√

Tai

lti

ltP

RM

:/r6

/c2-

Join

t2:j2

-22.

0022

.00

0.00

256.

25√

Min

imum

spee

dfo

ral

ljo

ints

:12

.50

degs

/s.

Init

ialsp

eed

for

alljo

ints

:62

.50

degs

/sM

inim

umac

cele

rati

onfo

ral

ljo

ints

:15

.63

m/s

2.

Max

imum

acce

lera

tion

for

alljo

ints

:31

2.50

m/s

2.

Init

ialac

cele

rati

onfo

ral

ljo

ints

:48

.83

m/s

2.

Not

e:Sp

eed

and

acce

lera

tion

limit

sar

eab

solu

teva

lues

,th

eyar

eva

lidfo

rbo

thdi

rect

ions

ofm

ovem

ent.

Tab

le3.

4:Jo

ints

(Ser

vos)

13

Iden

tific

atio

nV

alue

Sim

ulat

ion

Nam

eC

PC

Pri

mit

ive

Loc

ator

min

max

Uni

tH

ead

sens

or(b

ack)

PR

M:/

r1/c

1/c2

/c3/

f1-S

enso

r:f1

00.

9806

65N

∅H

ead

sens

or(f

ront

)P

RM

:/r1

/c1/

c2/c

3/f2

-Sen

sor:

f20

0.98

0665

N∅

Chi

nsw

itch

PR

M:/

r1/c

1/c2

/c3/

c4/s

5-Se

nsor

:s5

01

bina

ry∅

PSD

(Pos

itio

nSe

nsin

gD

evic

e)P

RM

:/r1

/c1/

c2/c

3/p1

-Sen

sor:

p10

0.90

m√

Lef

tfo

rele

gpa

wP

RM

:/r2

/c1/

c2/c

3/c4

-Sen

sor:

s40

1bi

nary

√

Lef

thi

ndle

gpa

wP

RM

:/r3

/c1/

c2/c

3/c4

-Sen

sor:

s40

1bi

nary

√

Rig

htfo

rele

gpa

wP

RM

:/r4

/c1/

c2/c

3/c4

-Sen

sor:

s40

1bi

nary

√

Rig

hthi

ndle

gpa

wP

RM

:/r5

/c1/

c2/c

3/c4

-Sen

sor:

s40

1bi

nary

√

The

rmo

sens

orP

RM

:/r6

/t1-

Sens

or:t

1n/

an/

a◦ C

∅B

ack

sens

orP

RM

:/r6

/s1-

Sens

or:s

10

1bi

nary

∅A

ccel

erat

ion

y-ax

isP

RM

:/a1

-Sen

sor:

a1-1

9.61

3319

.613

3m

/s2

∅A

ccel

erat

ion

x-ax

isP

RM

:/a2

-Sen

sor:

a2-1

9.61

3319

.613

3m

/s2

∅A

ccel

erat

ion

z-ax

isP

RM

:/a3

-Sen

sor:

a3-1

9.61

3319

.613

3m

/s2

∅

Tab

le3.

5:Se

nsor

s

14

Identification Value SimulationName min max UnitRemaining battery capacity 0 100 % ∅Battery temperature 0 227.00 ◦C ∅MTN key frame -1 32767 integer

√

These sensors don’t have a corresponding robot primitive.

Table 3.6: Miscellaneous

Figure 3.2: Scene Tree in Webots

15

not inversely). They have a uniform mass distribution inside of their boundingbox. The dimensions and weights of all model parts with have a bounding boxare listed in table 3.7. The dimensions were determined using [Model] and theweights were estimated based on table 3.1. The bottom part of table 3.7 showsthe correspondence between blocks (table 3.1) and model parts.

Name Figure 3.3 Size [mm] Mass [g]X Y ZBody red 80 100 183.6 613Head tilt ∅ 24 40 24 15Head pan ∅ 30 30 30 10Head roll yellow 68 68 103.5 190Leg J1 ∅ 1 40 40 50 (4x)Leg J2 blue 32 70 40 73 (4x)Leg front J3 light green 32 70 32 73 (2x)Leg rear J3 dark green 32 80 40 73 (2x)Paw touch sensor ∅ 15 15 15 0 (4x)Total 1612

Body = Bus connector + Battery + Tail block + 50g from Head blockHead tilt + Head pan + Head roll + 50g from Body = Head block

Leg J1 + Leg J2 + Leg J3 (front or back) + Paw touch sensor = Leg block

Table 3.7: Webots Model Nodes with Bounding Box

Webots allows the user to create complex environments for mobile robotsimulations using advanced hardware accelerated OpenGL technologies. Fur-thermore, Webots can import 3D models from most 3D modelling softwarethrough the VRML97 standard. The graphical representation of every part wasimported from [3D]. Taking [Model] as a reference, most parts did not have thecorrect size. They were therefore scaled in order to obtain a correspondencebetween the graphical representation and the bounding box as it is shown onfigure 3.3.

Figure 3.4 shows the hierarchy relations between all model nodes, includingthose not mentioned yet. These nodes are explained in the following paragraphs.The colored and the grey nodes are those listed in table 3.7. The complete modelis stored in the Webots world file aibo_ers210.wbt and appears in Webotsas shown on figure 3.6. The integration into Webots of the graphical userinterface for simulation control (described in the next chapter) and the modelconstruction were done in parallel. The interface for manual control has provento be a convenient tool for model testing.

LEDs

The Webots LED node is used to model Aibo’s LEDs. The shape of the lightemitting part of the LED device is defined as a Solid node in the ’children’ listof the LED node. Upon activation, the ’emissiveColor’ field of the first Materialnode in this Solid node will be changed to the color specified by the ’color’ fieldof the LED node. It is possible to add extra colors that can then be selectedfrom the LED programming interface.

16

Figure 3.3: Main Webots Model Nodes

17

Figure 3.4: Webots Model Node Hierarchy

Figure 3.5: Simulated Position Sensing Device

18

Only Aibo’s head LEDs are included in the current version of the model. Agrey appearance is defined for the Solid node in the ’children’ list of the LEDnode. This represents the OFF state. A single color was put in the ’color’ fieldof each LED. When this color is selected as the ’emissiveColor’, the LED is inON state. The color was chosen to be as close as possible to the color of thereal illuminated LED. For the red LEDs, the default color given by Webots wasadopted.

Joints

The Webots Servo node models a servo motor, which is adequate for the simula-tion of Aibo’s joints. A servo can be controlled in position, maximum velocity,maximum force and acceleration from the Servo programming interface. Allvalues must be expressed in radians. The ’acceleration’ field defines the accel-eration used by the position controller. It should be set to a value smaller thanthe ’maxForce’ field to achieve smooth and slow movements. This parameterdoesn’t specify the actual force of the servo, but rather the acceleration usedby the position controller. Hence, to achieve a slow and smooth movement,it is better to set a small value to the ’acceleration’ field rather than to the’maxForce’ field. A small ’maxForce’ parameter may indeed result in a servounable to move or to maintain a desired position because of the weight it hasto support. The ’acceleration’ field was introduced in the course of this projectin order to take into account JointMover’s trajectory generation on Aibo.

Every joint of Aibo is represented by a Webots Servo node. The minimumand maximum positions of a joint are stored in the ’minPosition’ and ’maxPo-sition’ fields. ’maxVelocity’ contains the maximum speed whereas ’maxForce’contains the maximum allowed acceleration for a joint. Those values must notbe changed in order to maintain a correct simulation of Aibo’s capabilities ofmovement. It is an open question, whether the values in ’maxForce’ are suffi-ciently high to accurately model the force of a servo. The minimum speed andacceleration cannot be included in a Webots model. The speed limit currentlyset for a servo (between the minimum speed and ’maxVelocity’) has no field inthe model, contrary to the ’acceleration’ field, which contains the accelerationlimit that was selected from the Servo programming interface. ’acceleration’initially contains the default acceleration.

All the parts of the model having a mass, except the body (table 3.7), cor-respond to the bounding box of a Servo node. The boxes of the head tilt andpan servos and of the four leg J1 servos are not shown on figure 3.3. The leg J1boxes are very flat and lie under the surface of the body bounding box. Theirweight simulates the unmovable part of the leg blocks (figure 3.1), with the onlydifference that they follow the rotation of their associated joint J1. The box ofthe head tilt joint was dimensioned in a way that it covers Aibo’s neck. Theweight of the head pan servo is used for adjustment in the back part of the head.The two tail servos do not have a bounding box and no mass either.

Plungers

There is no Webots node specially designed for the simulation of Plungers.These special Aibo servos with only two possible positions are represented byServo nodes with numerically very high values for ’maxVelocity’, ’maxForce’

19

and ’acceleration’. When they are commanded, only two positions should beused. This results in a very fast movement between these two positions. Dueto the device position and orientation in the Webots model, the ON positioncorresponds to ’0.0’ radians and the OFF position to ’1.2’ radians. Althoughthis is not the idea of Plungers, it cannot be excluded that a user sets othervalues (including speed and acceleration) via the Servo programming interface.

Sensors

The Webots DistanceSensor node is used to model sonar sensors, infra-red sen-sors and laser range finders. It uses a ray casting algorithm to detect collisionbetween the sensor ray and Solid nodes in the world. Simulated infra-red sen-sors have a special property: They are color sensitive and will see better lightor red obstacles than dark or black ones.

Aibo’s distance sensor is modeled by a DistanceSensor node whose ’type’ fieldis set to “infra-red”. In the field ’loopkupTable’ the returned values (and thecorresponding noise in percent) for different distances are defined. For distancevalues not specified in the lookup table, the simulator will perform a linearinterpolation to compute the value returned by the sensor and its associatednoise. In the case of an “infra-red” sensor, the value returned by the lookuptable is modified by a reflection factor depending on the color properties of theobject hit by the sensor ray. Figure 3.5 shows the simulated return values forsome distance values (including minimum and maximum values for the noise).These values are approximately those returned by the real robot when a whitepiece of paper is placed at the given distance. The exact behavior of the realsensor is rather difficult to reproduce. Tests with several Aibos should be done.

The Webots TouchSensor node is used to model bumper sensors. A bumpersensor will detect the collision with any Solid object in the world. Collisiondetection is based upon the ’boundingObject’ field of the TouchSensor nodeand the ’boundingObject’ field of other Solid nodes.

Aibo’s paw touch sensors are modeled by a TouchSensor node whose ’type’field is set to “bumper” and which returns binary values. They are the onlyparts in the model that do have a bounding box but a mass of zero. Because onlythe paw touch sensors are of interest for the simulation of Aibo’s movements,the back sensor, the chin switch and the two (non-binary) head sensors are notincluded in the model.

Acceleration and thermo sensors do not have corresponding Webots nodesand can therefore not be simulated. Every Servo node also includes a positionsensor, this means that joint positions can easily be measured in the model.

Miscellaneous

There is no major interest in simulating Aibo’s battery. The current MTN keyframe can be obtained during simulation but is not included in the robot model.

20

Fig

ure

3.6:

Sim

ulat

ion

ofA

ibo

ER

S-21

0in

Web

ots

(wit

han

dw

itho

utD

ista

nceS

enso

rra

y)

21

3.2 wxWindows

wxWindows is a C++ framework providing GUI (Graphical User Interface)and other facilities on more than one platform.2 Version 2 currently supportsall desktop versions of Windows, Unix with GTK+, Unix with Motif, and MacOS.

wxWindows was originally developed at the Artificial Intelligence Applica-tions Institute, University of Edinburgh, for internal use, and was first madepublicly available in 1992. Version 2 is a vastly improved version. According tothe developers, wxWindows was developed to provide a cheap and flexible wayto maximize investment in GUI application development and meets all of thefollowing criteria:

• low price (free)

• source availability

• simplicity of programming

• support for a wide range of compilers

As open source software, wxWindows has benefited from comments, ideas,bug fixes and enhancements of users. This gives wxWindows a certain advantageover its commercial competitors, plus a robustness against the transience ofone individual or company. This openness and availability of source code isespecially important when the future of a developed application may dependupon the longevity of the underlying class library.

The importance of using a platform-independent class library cannot beoverstated, since GUI application development is very time-consuming, andsustained popularity of particular GUIs cannot be guaranteed. Code can veryquickly become obsolete if it addresses the wrong platform or audience. Al-though wxWindows may not be suitable for every application, it provides accessto most of the functionality a GUI program normally requires, plus many extrassuch as network programming, PostScript output, and HTML rendering. Someother features of wxWindows are:

• simple-to-use, object-oriented API

• flexible event system

• constraint-based and sizer-based layouts

• toolbar, notebook, tree control, advanced list control classes

• common dialogs for file browsing, printing, colour selection, etc.

• support for many file formats (e.g. BMP, PNG, JPEG, GIF, XPM)

• over 50 sample programs

• over 1000 pages of printable and online documentation2Visit the web page [wxWin] to learn more.

22

3.3 Graphical User Interface

Webots and the Aibo Remote Control application both use wxWindows fortheir graphical user interface. Nevertheless, the integration into Webots madeseveral internal changes on the remote control necessary, mainly to keep the codeconsistent with already existing parts of Webots and to further ensure platformindependendy. Changes on the client software (or the software running on Aibo)are explained in the chapters and sections where the functionality that madethe changes necessary is treated. That’s why almost all modifications on Aibo’ssoftware are explained in the chapter about cross-compilation.

The former main application (AiboApp) doesn’t exist anymore. The inter-face (AiboFrame) was given the name A_Aibo and inherits from A_RobotWindow.When a graphical interface for a certain robot is implemented in Webots, it isalways a subclass of A_RobotWindow.3 The control window is opened by a dou-ble click on the robot model (depending on the already shown windows, thescene tree window is opened) or by choosing the corresponding menu entry.Every simulated Aibo has his own instance of A_Aibo, i.e. multiple robotscan be controlled. The networking component (NetworkConnector) was re-named to A_AiboConnector. One of the classes implementing interface elements(A_Picture) is now called A_AiboPicture.

The data structures used in the first version of the graphical interface werereplaced by structures commonly used in Webots. In order to maximize thereusability of the interface elements, the explicit reference of the parent framestored in some interface classes (sliders, LEDs) for doing callback invocationswas omitted. A similar functionality is achieved by taking advantage of theevent passing system in wxWindows. Similar to native wxWindows interfaceelements, the interface classes now generate events that are propagated to theirparent frame (where a corresponding event handler can be defined).

Only minimal changes were made on the visual appearance. The new check-box “Simulation” allows the user to turn simulation control on and off. Figure3.7 shows the initial state of all controls. On figure 3.8 it can be seen how theinterface and the controlled model look like after some interaction.

The interface is divided into three parts with logically grouped controls.The next sections follow approximatively this division. They describe how thesimulated robot model is controlled by the different interface elements. Onlyadditional information to the explanations is [RC] is presented.

3.3.1 Networking, Shutdown and Reboot

It is not necessary to “connect” to the simulation. The simulation is alwayscontrolled when no connection to a real Aibo is open. The state of the checkbox“Simulation” can’t be changed. Once a connection is established, the checkboxcan be used to turn on and off the simultaneous control of the simulation andthe real robot. When the connection is closed, the checkbox is automaticallychecked and manual state changes are again disabled.

In the current version of Webots, there is a simulation problem when con-nected to a real Aibo. Performance of the simulation drastically drops uponconnection or the simulation even completely stops. The reason seems to be

3It is possible that users implement their own interfaces as part of a controller program.The interface explained here has nothing to do with a controller program.

23

Figure 3.7: ScreenShot Graphical User Interface (disconnected)

24

Figure 3.8: Simulation controlled by Graphical User Interface

that controller programs are not executed correctly when Webots is connectedto Aibo (a void controller advances simulation when no other controller is spec-ified). Until the problem is solved, it is recommended to turn simulation controloff when connected to Aibo.

Webots’ simulation system uses virtual time, making it possible to run sim-ulations much faster than it would take on a real robot. There is a mechanismprepared in the user interface which sets simulation mode to real time uponconnection to Aibo and resets it to the previously selected mode when the con-nection is again closed. When leaving real time simulation mode turned off, theabove mentioned performance problem is slightly reduced. For this reason, themechanism is not yet enabled.

LEDs and Plungers are set to their initial values by Aibo when a connectionis established. The interface reflects this by resetting them to these values upona click on the “Connect” button and by commanding the simulation accordingly.Because speed and acceleration values are obtained from Aibo upon connectionin order to set the corresponding sliders to the correct initial position, thesevalues are also used to set the target velocity (different for all servos) and theacceleration of the simulated joints. No synchronization of joint positions isdone between the real robot and the simulation. When the previously men-tioned performance problem is solved and simultaneous control works properly,synchronizing the simulation and the real robot once a connection is establishedwould be an interesting feature. Symmetrically, synchronization could also bedone when simulation control is turned on when a connection already exists. Ithas to be tested, which synchronization direction is more convenient and usefuland whether the procedure should be automatic or manually controllable, e.g.by a button for each synchronization direction.

“Shutdown” and “Reboot” are disabled when only the simulation is con-

25

trolled. These two commands do not have a simulated equivalent. The initialposture stored in the file aibo_ers210.wbt is a standing position. The realrobot goes to this position after booting.

3.3.2 Sensors

The reading of all sensors (simulated and real) is not done in the handler ofthe wxWindows idle event anymore. Instead, a method executing the sameinstructions as the event handler is repeatedly called by the simulation scheduler.Continuous sensor reading therefore only works when the simulation is running.The basic simulation time step (and thus the sensor reading frequency) can beadjusted to suit the user’s needs (precision versus speed). The optimal valuedepends on the client computer and network performance.

At instantiation time of the graphical user interface, references of the simu-lated devices (servos, sensors, LEDs) are obtained from the simulation systemvia the CPC Primitive Locators of the robot components. Internally, Webotsuses an object-oriented programming interface that offers functionalities similarto the controller programming interface. Sensor values are obtained by callingthe appropriate methods of the objects representing simulated primitives. Forthe simulated sensors (including joint position reading) to work, they have tobe enabled in the Webots controller program. The default controller for Aibois called ers210 and does sensor enabling.

When Webots is not connected to a real Aibo, the servo sliders display thesimulated joint positions and the distance sensor text corresponds to the simu-lated Position Sensing Device. The four paw pictures are activated according tothe returned values of the four simulated paw touch sensors. The other picturesdo not change their state and the text for the other sensors is set to “[sim]”.When a connection is opened to Aibo, observation of all sensors is automaticallyturned on and the interface displays the values obtained from the real robot.Some interface elements were optimized in order to prevent unnecessary displayrefreshing, i.e. the display is only refreshed when a visible value change hasoccurred.

3.3.3 LEDs and Plungers

Simulated devices that can be commanded have appropriate methods in theobjects they are represented by. Depending on the state of the interface, theLEDs are turned off or on by setting their emissive color to none or the firstcolor that is defined for a given LED. Ear servos are commanded either to theposition ’0.0’ radians (ON) or ’1.2’ radians (OFF).

The handler dealing with the LED or Plunger interaction event (contains thecurrent state) checks the current connection state and the state of the “Sim-ulation” checkbox (connected, simulating, both) and sends a command to thereal robot and/or calls the appropriate method of the simulated device. Thisapproach is used for all primitives that can be controlled in simulation.

3.3.4 Joints

The events generated by the joint sliders contain the current slider position.The target positions of the simulated servos are set accordingly (after unit con-

26

version). There is no field in the model showing the target position. Of course acertain redundancy exists between the limits of the position sliders (given by aunique file defining all constants associated with a certain robot type) and theminimum and maximum positions defined in the Webots model file. It is notpossible to manually command a position exceeding the limits defined in themodel file. Those limits are important for trajectory generation and when thesimulation is commanded by a controller program.

Speed and acceleration sliders generate events containing the new globalmaximum values. The limits of the acceleration slider are valid for all jointsand therefore the values can be passed to the simulated device without anyfurther tests. The sliders are used to set the ’acceleration’ field visible in themodel file. ’maxForce’ remains untouched. The maximum velocity is differentfor all joints. When the slider position is higher than the maximum velocitydefined for a joint, the joint limit is set as the current maximum speed. Thetests are not done with the limit values in the model file but with the values inthe file defining the robot type constants. It is not the ’maxVelocity’ field in themodel file that is modified by the velocity slider but the target velocity whichis not included in the model.

At instantiation time of the graphical user interface, when references of thesimulated servos are obtained from the simulation system, target speed andacceleration of every joint are set to their initial values because the target speedis equal to ’maxVelocity’ after the loading of the model file. For the acceleration,this should not be necessary because the ’acceleration’ field in the model isalready set to the initial value. It is mainly done for symmetry reasons, also forthe ear servos where the target speed is indeed always equal to ’maxVelocity’.This shows another kind of redundancy between the constant values used inthe graphical interface and the model file. The setting of model fields based onprogram constants could of course be extended to other values.

It still has to be verified that the Webots Servo node is implemented cor-rectly, especially the trajectory generation with a given speed and acceleration.Some minor problems could already be corrected due to the direct comparisonwith the real robot. Once it is sure that the trajectory generation is correct,it cannot be excluded that some Servo parameters, especially ’maxForce’ and’maxVelocity’, have to be adapted in order to accurately simulate Aibo’s physi-cal capabilities. Currently, these values correspond to the maximum limits usedfor trajectory generation but they are not imposed by Aibo’s hardware.

3.3.5 MTN files

For the interpretation of the MTN file format on the client side, Sony’s imple-mentation used in the the MoNet sample (see [Samples]) is used. Slight mod-ifications had to be made in order to make the files usable on systems otherthan Aibo’s OPEN-R environment. In particular, byte order conversions wereintroduced when necessary to make the files readable on big-endian machines.

The dialog opened by “Choose...” in the bottom part of the interface pro-poses the directory controllers/ers210/mtn_files in the Webots home di-rectory as the starting point. The buttons for the upload and deletion of MTNfiles are disabled when only the simulation is controlled. When the “Playback”button is pushed, the chosen MTN file is loaded and speed and acceleration ofthe simulated joints are set to a numerically high value, overriding the limits

27

defined in the model file (after the MTN playback the speed and accelerationvalues are set back to their old values). This is necessary because the MTNplayback algorithm on Aibo does not respect these limits and only takes intoaccount the given position at a given time. Despite the fact that the upper speedlimits were taken from [FAQ], Sony’s MTN files do not seem to respect them.As the acceleration limit was chosen in the context of manual servo control, itdoes not make any sense for MTN playback. Once the real hardware limits canbe determined, they should be used for ’maxForce’ and ’maxVelocity’. Duringplayback, ’acceleration’ and the target speed should be set to these values.

The method called by the simulation scheduler for sensor reading is alsoused for MTN playback. The simulated time elapsed between consecutive callsto the method is determined and the current MTN key frame is calculatedaccordingly. The execution of the file starts at the next method call after theactivation of the “Playback” button, no movement to the starting position ofthe file is previously done. The positions corresponding to the current simulatedkey frame are read from the loaded file and are used to command the simulatedservos and to set the positions of the normally manually moveable joint sliderpart (manual control is disabled for the duration of the playback). The first keyframe commanded has the index zero. When the simulation time step is chosentoo high, the movement does probably not correspond to what is intended bythe values in the file. The MTN key frame rate is 16ms. With a simulation timestep bigger than 64ms, too many key frames are likely to be skipped and theresulting movement is incorrect.

Because the MTN file format is interpreted on the client side, only the cur-rent MTN key frame needs to be returned by the real robot when the observationof the MTN key frame is turned on (true during MTN playback on real Aibo).The content of the array returned by Aibo when all sensor values are requestedis shown in table 3.8.

0 to 18 0 to 6 0 to 7 0 to 2 0 or 1joints binary sensors sensors misc MTN key frame

Table 3.8: Transmission of multiple Sensor Values including MTN Key Frame

When an MTN file is executed on Aibo, the simulation must be running inorder to periodically receive the current MTN key frame (together with othersensor data) on the client side and to guarantee that the interface exits theplayback state after the last key frame. Simulation control does not need tobe enabled. The commanded positions corresponding to the received MTNkey frame are read from the loaded file and the target positions of the jointsliders are set accordingly. If simulation control is activated, they are alsoused to command the simulated servos. At the instant a specific MTN keyframe and the corresponding measured positions are received, the real robothas already reached that position. The simulation is thus commanded with acertain delay and is not absolutely synchronized with the real robot. The delayis compensated in logfile writing.

28

3.3.6 Logfile

The MTN playback can be logged to the file LOG.LOG in the Webots directory.The file format was extended in order to be able to represent three possiblecases: simulated playback, playback on Aibo, synchronized playback. Table 3.9shows how one key frame is stored. The logfile contains a series of such keyframes, they do not necessarily all have the same total size. The header of 8characters which is written at the beginning of the file is AIBO_1.1.

1 short 15 short · · ·MTN key frame commanded positions

· · · 1 byte 0 or 19 short · · ·# measured (0 or 19) measured: 15x position + 4x paw

· · · 1 byte 0 or 19 short# simulated (0 or 19) simulated: 15x position + 4x paw

Table 3.9: MTN Playback Logfile Format

Physical quantity Unit (precision)Angular Position 10−2 degsON / OFF 1 / 0Counter integer

Table 3.10: Physical Units in MTN Playback Logfile

The values are written in little-endian format and in the units given by table3.10, i.e. the values are used as they are received from Aibo. The order of thejoints is given by table 3.4, the order of the paw sensors by table 3.5.

When the simulation is commanded (given a simulated MTN key frame orthe current key frame received from Aibo) by setting the goal positions of thesimulated servos to the position in the MTN frame, the simulated measuredvalues can’t be immediately written to the logfile because the simulation wasn’tallowed to reach the commanded position. The simulated positions are measuredand written to the logfile only at the next key frame. This is similar to themeasuring procedure on Aibo which guarantees that the measures are made atthe very instant the robot is supposed to be in the position given by the currentkey frame index.

3.4 Controller Program

Once Aibo’s model was defined, nothing had to be changed in Webots in order tomake the simulation controllable by Webots controller programs. Using Webot’sdocumentation on [Webots] and the explanations in section 3.1.3, it should bestraightforward to write controllers. It has to be remembered that the names(necessary to obtain a DeviceTag) of the model parts correspond to their CPC

29

Primitive Locators and the restrictions on the joints’ capabilities given by table3.4 must be taken into account. For the simulated sensors (including jointposition reading) to work, they have to be enabled in the Webots controllerprogram. The default controller for Aibo does sensor enabling. It is calleders210 and can be found in the directory webots/controllers/ers210.

3.5 Possible Extensions

As explained in section 3.1.3, there are features of Aibo which are not includedin the model, either because there’s no interest in modeling them or becauseit is not possible to simulate them. Those features could be integrated in themodel by implementing new simulated device types.

With an extended Aibo model, it would be desirable to have control over theadditional features via the graphical user interface. Even the current versionof the interface has some capabilities which are unused because the simulatedAibo does not have all features of the real robot. These missing features shouldbe integrated into the model with priority in order to fully take advantage ofthe already existing user interface.

On the other side, there are devices which are included in the simulation sys-tem but which are not controllable via the Remote Control System and thereforenot integrated in the user interface, e.g. the camera. A fundamental extensionof Aibo’s model will certainly induce an extension of the graphical interface, thecommunication protocol and the OPEN-R software on Aibo.

Of course the addition of other simulated primitive types (specific or not toAibo) to Webots or the implementation of new functionalities on the real robotwill make it inevitable to extend the Webots controller programming interface.

30

Chapter 4

Remote Control

4.1 Graphical User Interface

After the integration into Webots and the minor changes explained in the pre-vious chapter, the graphical user interface still has the functionalities describedin [RC] when connected to Aibo. A certain number of internal changes weremade on Aibo’s OPEN-R software objects. They are described in the followingsections. Figure 4.1 shows how the interface and the controlled model look likewhen both the simulation and the real robot are commanded.

Figure 4.1: Simulation and Robot controlled by Graphical User Interface

31

4.1.1 Initial Position

The initial posture for simulation is a standing position. The real robot goes tothis position after booting (instead of the posture called “broadbase” by Sony).For this purpose, additional states were defined in the JointMover OPEN-R ob-ject on Aibo. JointMover can be in different states. A simplified state diagramis shown on figure 4.2. The actual implementation has more states. They areused during initialization, e.g. to fill a first command vector with valid data,to set joint gains and to reach several intermediate positions before the stand-ing posture is reached. After system startup, JointMover moves the joints tothe initial position via certain intermediate states. First, the posture “broad-base” is commanded. Then Aibo goes to the positions “sleeping”, “crouching”and finally “standing”. For this initial movement, the same algorithm as forall manually commanded movements is used. The only difference is, that thegoal positions are set by the program itself. When Aibo has reached its initialposture, it starts accepting commands.

Figure 4.2: JointMover State Diagram

4.1.2 MTN Playback

The MTN playback is not done anymore using Sony’s algorithm provided withthe MoNet sample (see [Samples]). An algorithm with less overhead was in-stead implemented (a much simpler wrapper class for MTN files can be used).One of its main advantages is, that the state “Finish MTN” has become obso-lete because the position corresponding to the last key frame is always exactlyreached. As soon as an MTN playback command comes in, JointMover leavesthe state “Accepting Commands” and goes to the state “Prepare MTN” (seefigure 4.2). In this state, Aibo moves to the position specified by the first keyframe in the file (using the algorithm for manual commands). Afterwards, theMTN file is executed as many times as loops are desired (state “MTN”). WhenAibo has successfully reached the final posture, JointMover goes again to state“Accepting Commands”. To make the enchaining of different MTN files moreconvenient and for the MTN playback in cross-compilation, where no loops canbe specified, the “Prepare MTN” state is skipped when the difference of the

32

current position and the position specified by the first key frame is sufficientlysmall, i.e. when the maximum difference is smaller than 45 degrees. Extensiveusage will have to show the adequateness of this value. It works fine for theMTN files currently available. When at least one joint has a superior difference,the preparation phase is executed for every joint.

Sony’s interpolation algorithm constructs two command vector frames perMTN keyframe (as the key frame rate is 16ms). The first frame has the positionvalue of the current key frame and the second frame has an interpolated valuein the middle between this and the next key frame. The interpolation processstops at the second last key frame, the motion therefore stops at a positionbetween the second last and the last keyframe. This explains why the positionafter playback is not exactly the position corresponding to the last key frame.Table 4.1 shows how command vectors are created by this algorithm when twokey frames are placed into one command vector and when there are six keyframes in the MTN file. At the end of the MTN file, a command vector mightcontain less than the maximum number of allowed frames.

kf 0 kf 1 kf 2 kf 3 kf 4 kf 5f 0 f 1 f 2 f 3 f 4 f 5 f 6 f 7 f 8 f 9cv 0 cv 1 cv 2

kf: key frame, f: frame, cv: command vector

Table 4.1: Command Vector Construction by Sony’s MTN Algorithm

The new algorithm makes the the hypothesis that the playback starts whenAibo is at the position corresponding to key frame zero (is reached in the “Pre-pare MTN” state). Two frames per key frame are calculated . The first framecontains a value between the current and the next keyframe and the secondframe has the value of the next key frame. This guarantees that the last keyframe is reached and improves comparability between measured and commandedpositions. When position measures are done between the execution of commandvectors, it is certain that Aibo was commanded to a position corresponding toa key frame position in the MTN file (the last frame in the preceding commandvector). This interpretation is useful for the playback of several files where thelast key frame of one file is equal to the first one of the next file or for loopplayback where the last key frame is equal to the first key frame of the samefile. Unfortunately this is not the case for the two walking files currently avail-able. The number of key frames per command vector can be specified. Thisdetermines the measuring frequency for logging. Uneven values are roundeddownwards to the next even integer. Table 4.2 shows how command vectors arecreated by the new algorithm under the same assumptions as for table 4.1.

kf 0 kf 1 kf 2 kf 3 kf 4 kf 5f 0 f 1 f 2 f 3 f 4 f 5 f 6 f 7 f 8 f 9cv 0 cv 1 cv 2kf: key frame, f: frame, cv: command vector

Table 4.2: Command Vector Construction by new MTN Algorithm

33

4.2 Controller Program

Webots has an integrated callback mechanism for simulated devices. For everysimulated device, a callback function can be specified. This function is calledwhen the value of the device is read or when it is given a command by a controllerprogram. When Webots is not connected to Aibo, no such callback function isset. Upon connection, the same callback function is specified for all devices. Thefunction receives the simulated device in question as an argument. Accordingto the passed device, a different behavior is implemented. The basic idea isto check whether a value was requested or a command was given and to readthe value from the real robot or to propagate the command to the real robot.Commands are given to Aibo in exactly the same way as they are given in theevent handlers of the graphical user interface. In the current implementation,only the functionality described in the following sections is available.

The controller performance problem mentioned earlier makes it practicallyimpossible to exhaustively test the remote control mode. Furthermore, not allWebots nodes correctly implement the callback mechanism. The functionalitydescribed in the following sections is therefore not yet completely integratedand the remote control mode for controller programs is disabled. Based on theexisting code and with some corrections in Webots, it should be straightforwardto use the existing architecture for an extension of the controller remote controlfunctionalities. It will have to be tested, whether the automatic activation ofthe controller remote control mode upon connection to Aibo is convenient orwhether manual enabling should be implemented.

When implementing the callback mechanism, it is important to ensure thatthe callback function is not called when a value is read or set via Webot’sinternal programming interface, e.g. when a simulated servo is commanded bya slider. At all locations in the client program where devices are commandedby the internal programming interface, a command is also sent to the real robotwhen a connection is open. No further messages should be sent due to callbackinvocations.

4.2.1 Sensors

When the device passed to the callback function is either one of the four sim-ulated paw sensors or the distance sensor (other sensor are not included in themodel), it is certain that the value of this sensor was requested in the controllerprogram currently executed. The last values received from the real robot (in themethod continuously called by the simulation system) are always memorized.It is therefore not necessary to send a requesting command to Aibo, becausethe measurements are already available. The values of sensor devices can beset to an arbitrary value via Webot’s internal programming interface. By set-ting the sensor values to the measurements received from Aibo, the simulationsystem is overridden and the object representing the simulated sensor is actu-ally a representation of the real sensor on Aibo. As a consequence, the call tothe sensor reading function in the controller program (and also via the internalprogramming interface) returns the real value.

34

4.2.2 LEDs and Plungers

In the case the device passed to the callback function is one of the simulatedLEDs (tail LEDs are not included in the model), it is sure that a commandwas given to this LED in the controller program, i.e. the value of the LED wasset by the Webots controller programming interface. On the other side, thisvalue can be accessed via Webot’s internal programming interface. It is checkedwhether the value differs from the last value set by the controller program. Ifthis is the case, a corresponding command is sent to Aibo.

Almost the same procedure is applied for the servo devices used to modelAibo’s ear Plungers. Although servo devices could receive other than positioncommands (e.g. setting speed and acceleration or reading value), it is alwayssupposed that a position was set. A position value of ’0.0’ radians is interpretedas ON, all other values are considered to be OFF. If the commanded binaryvalue differs from the one previously commanded, a message is sent to Aibo.

4.2.3 Joints

For a callback of a servo device, four cases must be distinguished: The targetposition was set, the target speed was set, the acceleration was set, the currentposition was requested. It is possible that several of these possibilities are trueat once because the controller program has executed several kinds of instructionsfor the same joint before the simulation step was done. For performance reasons,only one of the cases is taken into account with priorities equal to the order thecases were previously listed. The tests to distinguish the cases are similar to thetests done for the LEDs and Plungers, with the difference that they do not dealwith binary values. The values for target position, target speed and accelerationare accessed via Webot’s internal programming interface. It is checked in thisorder whether the values differ from the last values set by the controller program.The first value that is different determines which kind of command is sent toAibo (position, speed, acceleration). If no value differs from the previous one,the position of the real servo is set as the current simulated position in the samemanner as described for the simulated sensors.

4.3 Possible Extensions

The complexity of the remote control capabilities, the Aibo model, the graphicaluser interface and the controller programming interface are closely related. Al-ready the current implementation of the remote control by a controller programdoes not support the complete controller programming interface (i.e. turning aservo motor off) because there are no remote control messages associated withthem. Furthermore, there are many status parameters accessible on Aibo forwhich reading commands could be defined in the communication protocol. Itwould also be interesting to include control over Aibo’s speaker, microphoneand camera.

Network performance considerations, especially for the controller remotecontrol mode, could make it necessary to send multiple goal values, particu-larly joint positions, at once to Aibo. This might need and extension of thecommunication protocol or even a fundamental redesign of the communicationscheme, e.g. the implementation of asynchronous communication.

35

As already mentioned earlier, there is a certain redundancy between con-stant values used for the graphical interface and the OPEN-R software on Aiboon one side (given by a unique file defining all constants associated with therobot type) and some of the values stored in the model file on the other side,e.g. minimum, maximum and initial joint positions, maximum velocities andforces (accelerations). A simple solution would be to set some of the modelparameters automatically by the user interface according to the constant valuesin the unique file. This would, however, make the model file incomplete and notvery transparent to the user. Furthermore, Webots must be recompiled whenone of the constants is modified. Because most constant values are given bythe properties of the real robot and are thus not likely to change, this seems anacceptable and easy to implement solution.

The other solution would be the extension of the model nodes such that theyare capable of storing all necessary constants. Webots then had to read all valuesfrom the model file upon creation of the interface. Because the same constantvalues are used on the client and the robot side, a good mechanism would haveto be conceived in order to have access to these values in a straightforward andgeneral way in OPEN-R software.

The above discussion did not take into account multiple Aibo types, e.g.Aibo ERS-220 or Aibo ERS-7. Even if different remote control interfaces aredesigned for each type and the OPEN-R software is recompiled for each of them,it is desirable to have only one source code version for both the client and therobot software. Webots is able to read from the model file, what robot typeis currently simulated. The appropriate version of the graphical user interfacecould then be instantiated. Assuming that a file defining all constants is used,the file must be extended and a way of accessing different constant values usingthe same name has to be found. An easy solution would be to define constantinteger identifiers for each robot type. On the client side, the identifier is dynam-ically determined from the model file. On the robot side, the identifier can beobtained by checking the robot design via an OPEN-R function. These identi-fiers could then be used to access different elements in an array of constants, e.g.NUM_PRIMITIVES[model] where model is equal to the identifiers ERS_210 (therobot type currently implemented) or ERS_7, for instance. There are alreadyarrays of constant values in the current implementation for one robot type, e.g.the CPC Primitive Locators for each primitive. Multidimensional arrays mustbe used in these cases, e.g. PRIMITIVE_LOCATOR[model][...]. Internally, thereis a unique identifier (not the commandID used in the communication proto-col) for every physical Aibo component (and some other readable entities). Thenumber of primitives, their names and types vary from robot type to robot type.Furthermore, the identifiers for one robot type must be consecutive. These con-straints could be handled by appending the robot type to an identifiers name.For instance HEAD_CAMERA_ERS_210 equal to ’10’ and HEAD_CAMERA_ERS_7 equalto ’15’ could exist at the same time. Relatively few explicit identifier namesare used outside the file defining all constants. It is proposed to use the ar-ray approach to distinguish different values for different robots in the case thesame name exists for multiple types, e.g. if it has to be checked somewherein the code whether a given identifier is equal to the camera, the given identi-fier is compared to HEAD_CAMERA[model] where HEAD_CAMERA[ERS_210] equalsHEAD_CAMERA_ERS_210 and HEAD_CAMERA[ERS_7] equals HEAD_CAMERA_ERS_7.

36

Chapter 5

Cross-Compilation

5.1 OPEN-R

“OPEN-R” is the interface that Sony is promoting for the entertainment robotsystems to expand their capabilities.1 The “OPEN-R SDK” discloses the speci-fications of the interface between the “system layer” and the “application layer”and makes the development of custom Aibo software in C++ possible.

Practical aspects of the OPEN-R SDK are discussed in [Start]. The presentsection treats the basic concepts of OPEN-R.2 Features of OPEN-R:

• Modularized software and inter-object communicationOPEN-R software is object-oriented and modular. Software modules arecalled “objects”. Processing is performed by multiple objects with vari-ous functionality running concurrently and communicating via inter-objectcommunication. Connections between objects are defined in an externaldescription file. When the system software boots, the description file isloaded and used to allocate and configure the communication paths forinter-object communication. Connection ports in objects are identified bythe service name, which enables objects to be highly modular and easilyreplaceable as software components.

• Layered structure of the software and services provided by the system layerThe OPEN-R system layer provides a set of services (e.g. output of con-trol data to joints and input of data from various sensors) as the interfaceto the application layer. This interface is also implemented by inter-objectcommunication. OPEN-R services enable application objects to use therobot’s underlying functionality, without requiring detailed knowledge ofthe robot hardware. The system layer also provides the interface to theTCP/IP protocol stack, which enables the creation of wireless LAN net-working applications. The IPStack is an OPEN-R system layer object.Objects can use the network services offered by the IPv4 protocol stackby communicating with the protocol stack through normal message pass-ing, i.e. by sending special messages to and receiving special messagesfrom the IPStack.

1Visit the web page [OPEN-R] to learn more.2A more detailed discussion can be found in [Prog].

37

5.1.1 Object

OPEN-R application software consists of several OPEN-R objects. These arenot objects in the common object-oriented way of thinking. The concept of anobject is similar to one of a process in UNIX or Windows operating systems.Characteristics specific to objects:

• An object corresponds to one executable file.An object is a concept that only exists at run-time. Each object has acounterpart in the form of an executable file, created at compile-time. Thefile is put on an Aibo Programming Memory Stick. When the robot boots,the system software loads the file from the Memory Stick and executes itas an object.

• Each object runs concurrently with other objects.Each object has its own thread of execution and runs concurrently withother objects in the system.

• Objects exchange information using message passing.An object can send messages to other objects. When an object receivesa message, the method corresponding to the message is invoked, with thedata in the message as its argument. An important feature of objects isthat they are single-threaded. This means an object can process only onemessage at a time. If an object receives a message while it is processinganother message, the second message is put into the message queue andprocessed later. The typical life cycle of an object (this is an infinite-loop,an object cannot terminate itself):

1. Loaded by the system2. Wait for a message3. When a message arrives, execute the corresponding method. Possibly

send some messages to other objects.4. When the method finishes execution, go to step 2.

• An object has multiple entry points.OPEN-R allows an object to have multiple entry points. Each entry pointcorresponds to a method. Some entry points are common to all objectsand have purposes that are determined by the system, e.g. initializationand termination. Other entry points are specific to a certain object.

5.1.2 Inter-Object Communication

The use of inter-object communication enables each object to be created sepa-rately and later be connected to other objects. This results in a very efficientdevelopment lifecycle. When two objects communicate, the side that sends datais called the “subject” and the side that receives data is called the “observer”.The subject sends a “NotifyEvent” to the observer. NotifyEvent includes thedata that the subject wants to send to the observer. The observer sends a“ReadyEvent” to the subject. The purpose of ReadyEvent is to inform thesubject that the observer is ready to receive data or not. If the observer is notready to receive data, the subject should not send any data to the observer,otherwise messages might be ignored.

38

Figure 5.1: Inter-Object Communication

5.2 Objects on Aibo

The Remote Control System [RC] had to be extended and modified for theexecution of cross-compiled controller programs. The software on Aibo nowconsists of four OPEN-R objects. Figure 5.2 shows them as yellow circles.PowerMonitor is isolated from the other objects. JointMover, RCServer andController, however, have multiple entry points (in addition to the default entrypoints, which are not shown on the figure). The entry points are labeled with theservice names. Red arrows show message passing between objects (the subject isat the beginning of the arrow and the observer is where the arrow points to). Thearrows point in the direction of the notify events. A label shows the type of datatransmitted, e.g. joint and LED commands need to be passed to the robot inthe form of command vectors containing commands for multiple primitives andover a certain period of time. The ready events are passed in the opposite sense(not shown by a separate arrow, contrary to figure 5.1). RCServer possessesadditional entry points necessary for the network communication. These entrypoints are not drawn individually.

PowerMonitor was not modified for the purpose of this projet. The new Con-troller object is discussed in a more detailed manner in the following sections.The modifications made on JointMover and RCServer which were not previ-ously mentioned are explained together with the aspect of Controller that madethem necessary. A fifth object is shown on figure 5.2. OVirtualRobotComm isrepresented by a white circle. This object is part of Aibo’s system software andprovides services for sending commands to the robot, reading sensor values andretrieving images from the camera. The Effector service receives joint and LEDcommands.

5.3 Controller Cross-Compilation

5.3.1 Webots Robot Controller Program

Each Webots robot controller program is built in the same manner. An ini-tialization with the function robot_live is necessary before starting the robot.

39

Figure 5.2: Objects on Aibo

40

A callback function is provided to the robot_live function in order to iden-tify the devices of the robot. Then an endless loop (usually implemented asa for(;;) { } statement) runs the controller continuously until the simulatordecides to terminate it. This endless loop must contain at least one call to therobot_step function which asks the simulator to advance the simulation timea given number of milliseconds, thus advancing the simulation. Before callingrobot_step, the controller can enable sensor reading and set actuator com-mands. Sensor data can be read immediately after calling robot_step. Thencalculations can be performed to determine the appropriate actuator commandsfor the next step. A typical controller program could look like this (not specificto Aibo):

#include <robot.h>#include <differential_wheels.h>#include <distance_sensor.h>

DeviceTag ir;

void my_robot_reset() {ir = robot_get_device(‘‘ir_name’’);

}

int main() {robot_live(my_robot_reset);for(;;) { /* infinite loop */if (distance_sensor_get_value(ir)>100)differential_wheels_set_speed(0,0);

elsedifferential_wheels_set_speed(10,10);

robot_step(64); /* run for 64 ms */}return 0;

}

In this example, the robot is a differential wheeled robot equipped with aninfra-red distance sensor named “ir name” looking forward. The robot will stopmoving if the distance sensor detects an obstacle and restart moving when theobstacle is no longer detected. A similar Java programming interface is alsoincluded in Webots. Moreover, any Webots controller can be connected to athird party software program through a TCP/IP interface.

5.3.2 Cross-Compilation

The basic idea of the controller cross-compilation is rather simple. The sourcecode of a wrapper object for the controller code was created. This OPEN-Robject is called Controller. On Aibo, the Controller object runs together withother objects (see figure 5.2). Because OPEN-R programs are written in C++,it was rather straightforward to combine the controller program file and thesource code files of the Controller object. What was developed for controllercross-compilation is thus not really a cross-compiler but an OPEN-R objectwhich can integrate custom Webots controller programs without modifications

41

on the controller code itself. The actual cross-compiler is Sony’s OPEN-R cross-compiler included in the OPEN-R SDK available from [OPEN-R]. In order totake advantage of controller cross-compilation in Webots, the OPEN-R SDKmust first be installed.3

The Controller code includes the files in webots/include/device and itsMakefile was modified accordingly. The Controller object provides Aibo spe-cific implementations of the most important and most interesting functions ofthe controller programming interface. Table 5.1 lists the functions implementedfor cross-compilation.4

Figure 5.3: Cross-Compilation

The Aibo controller program coming with Webots is placed in the direc-tory webots/controllers/ers210. The controller code is contained in the fileers210.c. There are two Makefiles, Makefile is necessary for the compila-tion of the controller for simulated execution and Makefile.openr creates an

3See [Start] and [Install] for explanations on how to set up the OPEN-R SDK.4Consult [Webots] for the detailed programming interface.

42

robot.h functions Descriptionrobot live executes the passed function oncerobot die stores the passed function

(executed when Aibo is shut down)robot run see 5.3.3robot get device returns unique identifier as DeviceTagrobot get mode always returns ’1’ (cross-compilation)robot battery sensor enable does nothingrobot battery sensor disable does nothingrobot battery sensor get value returns remaining capacity [%]led.h functions Descriptionled set see 5.3.4servo.h functions Descriptionservo motor off see 5.3.6servo enable position does nothingservo disable position does nothingservo get position see 5.3.5servo set position see 5.3.4 and 5.3.6servo set velocity see 5.3.6servo set acceleration see 5.3.6servo set force does nothingdistance_sensor.h functions Descriptiondistance sensor enable does nothingdistance sensor disable does nothingdistance sensor get value see 5.3.5touch_sensor.h functions Descriptiontouch sensor enable does nothingtouch sensor disable does nothingtouch sensor get value see 5.3.5mtn.h functions Descriptionmtn new see 5.3.7mtn get error see 5.3.7mtn play see 5.3.7mtn get length see 5.3.7mtn get time see 5.3.7mtn is over see 5.3.7mtn delete see 5.3.7

Table 5.1: Implemented Functions for Cross-Compilation

43

object executable on Aibo. The source code files to be compiled by both Make-files are listed in Makefile.sources. For cross-compilation, Makefile.openrcalls a Makefile in a subdirectory of webots/transfer/openr. This directorycontains the intermediate cross-compiled files of the Controller OPEN-R ob-ject. The Makefile called in this directory compiles all source code files specifiedin Makefile.sources (*.c, *.cc or *.cpp) using Sony’s cross-compiler andlinks them with the already existing files. If there is not yet a directory calledOPEN-R in the controller directory, a default OPEN-R directory is copied fromwebots/transfer/openr. Calling make -f Makefile.openr clean will againremove the OPEN-R directory. The Controller binary file CONTROLL.BIN resultingfrom the controller cross-compilation is then placed into OPEN-R/MW/OBJS (thebinary files of the other Remote Controls System objects are also in this direc-troy) in webots/controllers/ers210 (by default CONTROLL.BIN correspondsto an empty controller program). This and the other files in the OPEN-R di-rectory must not be modified, renamed, moved or deleted. It is this OPEN-Rdirectory that has to be copied to the Memory Stick in order to install the com-plete Remote Control System (four MTN files are already present and thus donot have to be uploaded later). Figure 5.3 shows a summarized schema of thecross-compilation process.

Before putting the Remote Control System on the Memory Stick, the defaultconfiguration has to be installed following the instructions in [Start] and [Install].The default configuration is also contained in a directory called OPEN-R. TheOPEN-R directory copied from webots/controllers/ers210 must not replacethe already existing directory on the Memory Stick. The contents of the twodirectories have to be merged.

Aibo’s wireless LAN configuration file is located in OPEN-R/SYSTEM/CONF onthe Memory Stick (was copied with the configuration). A copy of WLANCONF.TXTis situated at the same place in the controller directory. The file can be mod-ified according to the user’s needs and it is automatically copied to the rightlocation on the Memory Stick when the complete OPEN-R directory is put onthe stick. The easiest way to get wireless communication between Aibo and aclient computer is to leave the configuration file untouched and to set up theclient station according to the settings in WLANCONF.TXT.5

5.3.3 Aibo specific Programming Scheme

The procedure described in the previous section creates binary code which is in-deed executable on Aibo. Upon execution, Controller is notified by JointMoverover the MainExecution entry point (see figure 5.2) when the initial standingposition was reached. Controller then calls the Webots controller main functionand starts executing the controller program. The major problem is the infiniteloop in the controller main function. Aibo’s programming scheme is contrary towriting infinite loops in order to have a program running without interruption.In OPEN-R code a completely different approach has to be used. In section5.1, the life cycle of an OPEN-R object was presented. The whole life cycle ofan object is an infinite loop, but it is an implicit infinite loop, consisting of thecontinuous reception and sending of messages from and to other objects. In-structions (placed in a method) are only executed when an appropriate message

5Consult [Start] and [Install] to learn more.

44

is received, e.g. the notification on the MainExecution entry point of Con-troller which induces the execution of the controller main function. If there isan explicit infinite loop in the main function (and thus in the method handlingthe notification event), the Controller object never finishes the execution of thehandling method and can’t receive any other notifications because objects aresingle-threaded and can process only one message at a time. Messages receivedwhile processing another message are processed later. Introducing waiting in-structions does not solve the problem.

With respect to this particularities, it must always be guaranteed that amethod dealing with a notification event (this definition covers all OPEN-Rmethods) terminates. RCServer and JointMover are implemented according tothis programming scheme. In order to make Controller compatible with theOPEN-R programming scheme, the Webots controller programming interfacehad to be extended. The function robot_run was defined. The controller pro-gram shown above then becomes:

#include <robot.h>#include <differential_wheels.h>#include <distance_sensor.h>

DeviceTag ir;

void my_robot_reset() {ir = robot_get_device(‘‘ir_name’’);

}

int my_robot_run(int k) {if (distance_sensor_get_value(ir)>100)differential_wheels_set_speed(0,0);

elsedifferential_wheels_set_speed(10,10);

return 64;}

int main() {robot_live(my_robot_reset);robot_run(my_robot_run);return 0;

}

The robot_run function starts the control loop for a robot. The argument ofthe robot_run function is the function that shall be executed endlessly. Thisrun function receives an integer as an argument. It is ’0’ the first time thefunction is called and it takes a possibly different value on subsequent calls. Insynchronous simulation mode, this value is always ’0’. In asynchronous mode,this value may be different from ’0’. The integer value returned by the runfunction is the requested time step expressed in milliseconds. This time stepdefines the duration of an iteration of the control loop. The return value of therun function replaces the call to robot_step, which has become obsolete.

The simulation in Webots translates this code to a program equivalent tothe first version containing an infinite loop. This is not yet integrated in the

45

current Webots version and different controllers have to be used for simulationand cross-compilation. In the implementation of robot_run on Aibo, the func-tion passed as a parameter is executed only once (similar to the implementationof robot_live). Further invocations of the run function are done in the methodhandling the ready event coming from OVirtualRobotComm on the Led entrypoint. On Aibo, the integer argument of the run function is always ’0’ (equiva-lent to synchronous simulation mode). It has to be considered that robot_runnever returns in simulation and that instructions after robot_run are neverexecuted, contrary to the execution on Aibo.

The alternative to defining a new function would have been a conversion ofthe source code in the precompilation phase. This solution is much less generaland reusable and problems with scopes of variables arise almost certainly.

5.3.4 Timing, LEDs and Plungers

The most convenient and precise way to obtain a difference in milliseconds equalto the time step (the return value of robot_run) between consecutive calls to therun function is to take advantage of the period for LED commands. The usageof a wait function being part of Aibo’s underlying Aperios system (OPEN-Ris an extension of Aperios) did not lead to acceptable results and is, althoughpossible, not conform to the OPEN-R programming scheme.

For every LED value in a command vector (multiple LEDs with multiplevalues possible), an illumination duration (period) can be specified in Aibo’sbasic time unit (frame of 8ms). The last commanded values of each LED areput as the first and only elements in the LED command vectors. Their peri-ods are all set to the time step. For this reason the time step on Aibo shouldbe a multiple of 8ms. The first command vector is passed to Aibo immedi-ately after the execution of the controller main function. The ready event sentback by OVirtualRobotComm arrives after the specified period of time. That’swhy further invocations of the run function are done in the method handlingthe ready event coming from OVirtualRobotComm after the execution of LEDcommands. The last instruction in the handling method again sends a LEDcommand vector to the robot and the loop goes on. Commands given to LEDsin the run function are buffered and written into the command vector beforeit is passed to Aibo. When there was no modification on the LEDs’ state, theprevious command vector is sent again in order to continue the loop. Using thisapproach, it is possible to implement an infinite loop with exact timing whilerespecting Aibo’s programming scheme.

The function led_set is used to command LEDs. If a value different fromzero is passed to the function, the specified LED is turned on by writing the ap-propriate value to the buffer which will be copied to the next LED command vec-tor. As there is no simulated Plunger device and therefore no programming in-terface specific to servos with two possible positions, the general servo program-ming interface has to be used. If the DeviceTag passed to servo_set_positionrepresents a Plunger, the device is commanded in a similar way to a LED. Con-trary to LEDs, a value of ’0.0’ stands for ON. Any other value is interpreted asOFF. This is due to the implementation of the Webots model.

Plunger commands given in the run function are also buffered and sent atthe same time as the LED commands, but in a separate command vector, only ifthere was a modification in their state and only when Aibo is ready to accept a

46

new Plunger command. The ready check doesn’t need to be done for the LEDsbecause new commands are only sent when the ready notification is received.When Aibo is not ready to receive a new Plunger command, it is attempted tosend the command vector at the next loop iteration. The Plungers are treatedseparately from the LEDs because the ears always need about 128ms to becomeready again. When LED commands are given in the same command vectoras Plunger commands, the ready notification for both only occurs after 128ms.This would prevent time steps smaller than 128ms. In any case, the time stepshould not be too small for performance reasons (huge amount of messagespassed), e.g. 32ms or 64ms seems to be appropriate.

For consistency reasons, all LED and Plunger commands are sent to Aibo bythe Controller object. It would be possible for the RCServer to still send it’s owncommands to OVirtualRobotComm. Because both RCServer and Controllerwould then have to be informed of the commands given by the other object(in order to prevent overwriting of the LED state by the other object), it ismuch clearer to have only one objet commanding the LEDs. Because Controllerabsolutely needs to give LED commands for timing reasons, it is this object thatis responsible of passing command vectors to Aibo. To keep the architecturesimple, Controller is also responsible for the Plungers. RCServer sends the LEDand Plunger state as it was commanded over the network to Controller on theLedEar entry point. Controller fills the values in its buffer. Values commandedin the run function have priority over those coming from RCServer, i.e. theyoverwrite the buffer in any case.

5.3.5 Sensors

The functions to enable and disable sensors have no effect. Contrary to thesimulation, sensor values can be read in any case by calling the appropriatefunction. Only sensors for which a controller programming interface exists canbe accessed, i.e. thermo and acceleration sensors are not readable.

The position of any joint can be read with servo_get_position. The valuereturned has unit radians. Even though Plungers use parts of the Servo pro-gramming interface, their positions are not readable. For all invalid Servo de-vices, ’0.0’ is returned by default. All other Sensor reading instructions alsoreturn zero when the requested device does not exist.

Despite the fact that there is only one distance sensor on Aibo, the DeviceTagpassed to distance_sensor_get_value must be Aibo’s head PSD. The unit ofthe returned value is millimeters.

touch_sensor_get_value does not only work with the four paw sensorsthat are simulated, but also with the other touch sensors (head front and back,back, chin). The two head sensors are assumed to be binary sensors. As soonas the force applied exceeds half of the upper limit, ’1’ is returned.

5.3.6 Servos

The unit for all servo commands is radians. In a way analogue to LED andPlunger commands, target positions, velocities and accelerations are stored in abuffer. The limits for positons, velocities and accelerations given in table 3.4 arerespected by the commands servo_set_position, servo_set_velocity and

47

servo_set_acceleration and values beyond these limits are adapted accord-ingly. At every time step (if there were values changed), the whole buffer arrayis passed to the JointMover object. The entry point BatchMover was created forthis purpose (see figure 5.2). Every value which differs from the correspondingpreviously received value on this entry point is treated in a way equivalent to theprocedure done when a command for a single joint is received over the Moverentry point. Commands can be given by RCServer and Controller in parallel, itis the last received value that is taken into account. RCServer is not informed ofspeed and acceleration changes commanded by Controller. The values returnedby RCServer when such a value is requested, e.g. upon connection to determinethe initial slider positions, reflect the last values set manually.

The Mover entry point is never used for single joint commands (position,speed, acceleration) given to JointMover by Controller. For the playback ofMTN files, however, a playback command is generated by Controller and sentto JointMover on the Mover entry point as it is also done by RCServer (seesection 5.3.7).

An interesting feature only available for cross-compilation and simulation,but not included in the communication protocol for remote control and in thegraphical interface, is the function servo_motor_off. The gain for the specifiedservo is turned off, i.e. motor power is turned off and the joint can be freelymoved, e.g. to record position values or to command other servos. Callingservo_set_position of a disabled joint turns servo gain on again and theservo moves to the position given in the command. There are is other way ofturning servo gain on, i.e. giving commands via remote control has no effect.

5.3.7 MTN Playback

The Webots controller programming interface for MTN playback had to be de-signed specially for Aibo. The current version of Webots does not yet implementthe MTN related functions and there is no internal programming interface toaccess them. That’s why the simulated MTN playback is currently handled inthe sensor reading method. Aibo, however, can already be programmed usingthe MTN programming interface.

mtn_new returns a pointer to a simple MTN file wrapper class. All MTNfunctions except mtn_get_error take as their single argument such a pointer.A filename must be passed to mtn_new and the function then loads the MTNdata from the Memory Stick and returns a pointer to an instance of the wrapperclass. Only filenames, no complete paths should be passed as a parameter tomtn_new. If there was any error when loading the file, a null pointer is returnedand an error message string can be accessed by calling mtn_get_error. Theerror messages is reset to null upon a new call to mtn_new. To liberate thememory reserved for a loaded file, mtn_delete has to be called. This does notdelete the MTN file on the Memory Stick.

The files are supposed to be in the directory /MS/OPEN-R/MW/DATA/P, this isthe directory where uploaded MTN files are placed by RCServer. This should beconsidered when custom file handling is implemented in a controller program.6

The directory delimiter is /. A path must begin with /MS/. This represents theroot of the Aibo Programming Memory Stick. Relative paths are not allowed.

6See [Prog] for a list of system calls available on Aibo and their specific behavior.

48

A filename must have the 8+3 format. The base name and the extension mustbe separated by a period character (.). Lowercase letters are automaticallyconverted to uppercase letters.

An MTN file is executed by calling the function mtn_play. The appropriateplayback message is then sent to JointMover. The playback works independentlyof the chosen time step. The number of loops can’t be specified, looping behaviorhas to be explicitly programmed in the controller program. For this purpose,the functions mtn_get_length, mtn_get_time and mtn_is_over are available.mtn_get_length returns the duration of the movement stored in the MTN file.The duration is equal to the number of key frames minus one (the number ofelementary movements) times the key frame rate (16ms). The value returned bymtn_get_time represents the current position in the MTN file during playback(current key frame times the key frame rate). If the key frame is invalid (equal to’-1’), it is supposed that a previous playback has occurred and the total durationis returned. For both length and time, zero is returned when the specified filedoes not exist, i.e. the pointer is null. mtn_is_over checks whether the playbackof a file has finished. It simply compares the file length and the current time andreturns ’1’ in case of equality and ’0’ otherwise. This implies that files havingan invalid current key frame or non existing files are considered to be over.

Controller has, like RCServer, an entry point called SetValue (see figure 5.2).At every invocation of the MTN playback algorithm in JointMover, the com-manded values in the MTN file and the positions measured are passed togetherwith the current MTN key frame index (’-1’ is explicitly passed after the lastloop) to both RCServer and Controller. Controller only uses the current MTNkey frame in order to update the current key frame of the file being executed.

5.4 Controller Transfer

Cross-compiled controllers do work absolutely independently of the client partintegrated in Webots. It is not necessary to be connected to Aibo in order to runcontroller programs on the robot. All that has to be done is to call the appro-priate Makefile and to copy the binary files to the Aibo Programming MemoryStick. It would, however, not be very convenient to put new CONTROLL.BINfiles onto the Memory Stick manually when frequent recompilations are done.That’s why the networking section of the graphical user interface integrated inWebots has a button called “Cross”.

The “Cross” button induces the wireless transfer of the file CONTROLL.BIN tothe directory /MS/OPEN-R/MW/OBJS on Aibo. This directory contains all binaryfiles of the Remote Control System. When Aibo boots the next time, e.g. aftera click on the “Reboot” button, the new binary file is loaded and executed.Only the file CONTROLL.BIN is transfered, not the complete OPEN-R directoryin webots/controllers/ers210. The file must be at its exact location for the“Cross” button to work correctly.

A new command message was defined for the controller upload in orderto distinguish the destination directories for binary and other files (currentlyonly MTN files). The answer message is the same for all upload commands.Furthermore, a field had to be added to the FileCommand message type (field’extra’) in order to make the commands more versatile. Table 5.2 shows thenew FileCommand format.

49

Even if there is data available on the communication channel (the clientsends the complete file at once), it has been observed, that the packets receivedby Aibo do not always have maximum size (512 bytes in the current implemen-tation). This effect in general only occurs when transferring files much biggerthan average MTN files, e.g. binary files. It was therefore not sufficient anymoreto specify the size of an uploaded file as the number of packets necessary forits transmission. Aibo was already prepared to receive packets of size smallerthan the maximum size, even in the middle of a transmission, but the case whenthe cumulated differences result in additional packets needing to be transferedwas not foreseen. In the current version of the communication protocol, theexact file size is included in the upload commands for the cost of one additionalparameter. The field ’option’ contains the number of complete packets that canbe filled and ’extra’ contains the number of remaining bytes. These values arereconverted back to the exact file size after reception of the command on Aibo.It still results a theoretical file size that is twice as big as the capacity of anempty Aibo Programming Memory Stick (16 MB).

In the file reception method in RCServer, the recursive call was replacedby a simple loop until the received file size is equal to the expected file size.Network communication and file handling on Aibo is rather slow. In order toreduce the danger of timeouts on the client side when sending large files, thetimeout on send and receive operations defined in A_AiboConnector was raisedto ten seconds. Furthermore, file data is not sent at once but in chunks of 1000512 byte packets. After each chunk, a one second wait was introduced.

5.5 Possible Extensions

As stated earlier, the complexity of the remote control capabilities, the Aibomodel, the graphical user interface and the controller programming interfaceare closely related. With some exceptions, only functions in the controllerprogramming interface providing a functionality similar to the capabilities ofthe Remote Control System are implemented for cross-compilation. There aredevices on Aibo, for which it would be very interesting to have a convenientprogramming interface, even if these devices are not simulated, e.g. thermo andacceleration sensors, microphone, speaker. Except for the challenging task ofimplementing all simulated devices, it is probably easiest to obtain equivalencebetween a (complete) Aibo model controlled by a simulated controller and a realAibo running the same cross-compiled controller. The implementation of fullwireless remote control by a controller running on the client side would implyan important extension of the communication protocol. A generalization of thecallback mechanism for simulated devices might become necessary. Dependingon the data read from Aibo’s devices (e.g. camera images) and the amount ofother data transferred to and from the robot, the network performance mightsoon become a limiting factor. Finally, designing a graphical interface giving ac-cess to all the newly implemented functionalities would certainly not be trivial.A limitation to the most important commands or an interface with customizablelevels of complexity might be a solution.

50

File

Com

man

dB

asic

Ans

wer

Des

crip

tion

char

ucha

rch

ar[1

2]us

hort

shor

tch

aruc

har

shor

tco

mm

and

flag

filen

ame

opti

onex

tra

com

man

did

enti

fier

valu

eU

igno

red

8+3

files

ize

files

ize

u0

0up

load

file

to/MS/OPEN-R/MW/DATA/P

Dig

nore

d8+

3ig

nore

dig

nore

dd

00

dele

tefil

ein

/MS/OPEN-R/MW/DATA/P

Pfr

ames

/vec

tor

8+3

loop

sig

nore

dp

0lo

ops

play

back

file

in/MS/OPEN-R/MW/DATA/P

Cig

nore

d8+

3fil

esiz

efil

esiz

eu

00

uplo

adbi

nary

to/MS/OPEN-R/MW/OBJS

Tab

le5.

2:M

essa

ges

ofT

ype

File

Com

man

d

51

Bibliography

[RC] Documentaion for the project “Wireless Remote Control andMonitoring of an Aibo Robot” by Lukas Hohl.Can be downloaded on http://birg.epfl.ch/page43234.html

[Webots] Official Webots web page:http://www.cyberbotics.com/

[Start] “Aibo ERS-210: QuickStart Manual” by Lukas Hohl.Can be downloaded on http://birg.epfl.ch/page43234.html

[OPEN-R] Official Sony AIBO SDE web page:http://openr.aibo.com/Direct link to the English web page:http://openr.aibo.com/openr/eng/index.php4

[Download] Section “OPEN-R SDK Download” on [OPEN-R]

[FAQ] Section “OPEN-R SDK FAQ (Frequently Asked Questions)” on[OPEN-R]

[Docs] File “OPEN-R SDK Documents English”:OPEN_R_SDK-docE-XXX.tar.gz7 in [Download]

[Install] Document “Installation Guide”:InstallationGuide_E.pdf in [Docs]

[Prog] Document “Programmer’s Guide”:ProgrammersGuide_E.pdf in [Docs]

[Model] Document “Model Information for ERS-210”:ModelInformation_210_E.pdf in [Docs]

[3D] File “ERS-210/220 3D model data”:RS200-3Dmodels-XXX.tar.gz in [Download]

[Samples] File “Sample Programs”:OPEN_R_SDK-sample-XXX.tar.gz in [Download]

[wxWin] Official wxWindows web page:http://www.wxwindows.org/

7“XXX” in a filename stands for the current version number of the file (not the same forall files)

52

Appendix A

MTN File Format

The file format as it is show here is defined in the file MTN-FFORM-E.txt comingwith the MoNet sample program included in [Samples].

- function: Provides motion data information for a robot.

- byte order: little endian

- data type definition:

type: Strings are composed by the following definition.

offset size type contents data-------+-------+-------+-------------------------------+----

0 1 uc length1 x ch characters

length: The number of characters in a stringcharacters: String data

- File Format:

offset size type contents data-------+-------+-------+-------------------------------+----

0 1 ch magic 0 ’O’1 1 ch magic 1 ’M’2 1 ch magic 2 ’T’3 1 ch magic 3 ’N’4 24 cp Section 028 x cp Section 1x x cp Section 2x x cp Section 3

magic 0-3: file magic

53

1. Section 0Section0 is the header of fundamental items.It has a fixed length.

offset size type contents data-------+-------+-------+-------------------------------+----0 4 lw Section Number 04 4 lw Section Size 248 4 lw Number of Sections 412 2 wd Major Version 114 2 wd Minor Version 216 2 wd Number of keyframes18 2 wd Frame Rate 1620 4 lw Reserved 0

Section Number: Serial Number of this sectionSection Size: The size of this section (in bytes)Number of Sections: The number of sections in this fileMajor Version: The major version number of FormatMinor Version: The minor version number of FormatNumber of keyframes: The number of keyframes in this fileFrame Rate: Motion replay speed [msec/frame]

2. Section 1This header describes the fundamental names.It has a variable length.Note: Use padding when the Section Size is less than 4 bytes.

offset size type contents data-------+-------+-------+-------------------------------+----0 4 lw Section Number 14 4 lw Section Size8 x Strings Motion Namex x Strings Creatorx x Strings Design Labelx 0-3 Padding 0

Section Number: Serial Number of this sectionSection Size: The size of this section (in bytes)Motion Name: file name without the mtn extensionCreator: Author nameDesign Label: The design label of the robot

3. Section 2This header describes the Joint list.It has a variable length.Note: Use padding when the Section Size is less than 4 bytes.

54

offset size type contents data-------+-------+-------+-------------------------------+----0 4 lw Section Number 24 4 lw Section Size8 2 wd Number of Jointsx x Strings PRM 0x x Strings PRM 1x x ....x x Strings PRM (Number of Joints -1)x 0-3 Padding 0

Section Number: Serial Number of this sectionSection Size: The size of this section (in bytes)Number of Joints: The number of Joints used in this filePRM: CPC Primitive Locator name of a Joint used in this file

4. Section 3This head describes the type of motion data.It has a variable length.Note: Use padding when the section Size is less than 4 bytes.

offset size type contents data-------+-------+-------+-------------------------------+----0 4 lw Section Number 34 4 lw Section Size8 4 lw Data Type 012 x cp Data Section

Section Number: Serial Number of this sectionSection Size: The size of this section (in bytes)Data Type: The kind of Motion Data (0 : angle line data)

4-1. Data SectionThe following is the data format when the Data Type is 0.

[Roll] [Pitch] [Yaw][Joint Data]{[Number of Interpolate frame][Roll] [Pitch] [Yaw][Joint Data]

} X (Number of Keyframes - 1)

offset size type contents data-------+-------+-------+-------------------------------+----0 4 lw Roll4 4 lw Pitch8 4 lw Yaw12 x cp Joint Data Sectionx 4 lw Number of Interpolation frames

55

.....Roll: Robot’s BODY roll (in Micro radian)(when it leans to the right, plus)

Pitch: Robot’s BODY pitch (in Micro radian)(when it leans backward, plus)

Yaw: Robot’s BODY yaw (in Micro radian)(when it turns to the left, plus)

Joint Data Section:The joint data is associated with the corresponding PRMString in Section 2.

Number of Interpolation frames:The number of interpolation frames with the former frame.

4-1-1. Joint Data Section

offset size type contents data-------+-------+-------+-------------------------------+----0 4 lw Joint Data 04 4 lw Joint Data 1

.....x 4 lw Joint Data (Number of Joints)

Joint data: The position of a joint (in Micro radians)

56