Through the Eye of the Needle:

Perceiving Affordances for Minimally Cognitive Morphological Change

Sarah Barber

Submitted for the degree of Evolutionary and Adaptive Systems MScUniversity of SussexSeptember 2011Declaration

1

I hereby declare that this thesis has not been submitted, either in the same or different form, to thisor any other university for a degree.

Signature:

Acknowledgements

2

Thanks to Dr Luc Berthouze for useful supervision and support, and for ensuring that the aims of the project were clearly laid out from the start. Thanks also to Dr Inman Harvey and Dr Andy Philippides for useful discussion. The biggest thanks goes to David Angove for both emotional and nutritional support throughout this endeavour.

Abstract

3

This paper analyses an evolved Continuous-Time Recurrent Neural Network -based controller of an artificial legged agent that changes its morphology in order to traverse a simple environment whilst avoiding falling into chasms. The controller of an agent whose morphology and locomotion are coupled is compared with that of an agent whose are decoupled.

Keywords: morphology, locomotion, Continuous-Time Recurrent Neural Network, genetic algorithms, minimally cognitive behavior.

4

ContentsAbstract ................................................................................................................................................. 3

Contents ................................................................................................................................................ 5

Introduction .......................................................................................................................................... 5

2 Background and Related Work ........................................................................................................... 6

2.1 Field Robotics and Legged Robots ............................................................................................... 6

2.2 Behaviour-Based Robotics ........................................................................................................... 7

2.3 Perceiving Affordances ................................................................................................................ 9

2.4 Dynamical Systems and Continuous-Time Recurrent Neural Networks ..................................... 10

2.5 Minimally Cognitive Behaviour .................................................................................................. 11

3 Methodology ................................................................................................................................... 12

3.1 The Agent ................................................................................................................................... 12

3.2 The Controller ............................................................................................................................ 13

3.3 The Environment ....................................................................................................................... 14

3.4 Evolution .................................................................................................................................... 14

3.5 List of Experiments ..................................................................................................................... 15

7 Bibliography ...................................................................................................................................... 28

I ntroduction

This project aims to create an artificial agent that can traverse a simulated environment, travelling along narrow ridges while detecting and avoiding chasms. The agent will have to make use of techniques such as changing its bodily dimensions in order to achieve this task. It will do this by drawing its legs underneath its body to move between two chasms, or by pushing its legs out as far as possible to travel over a single chasm. The creation of this agent may have applications in the area

5

of field robotics, where it may be useful for a legged robot to change its morphology to traverse an environment that is unsafe for humans. The principle of changing a legged robot’s morphology can be applied to avoiding falling down chasms, and also squeezing through tight apertures.

An understanding of the relation between body morphology and locomotion patterns is important for designing a walking robot. This paper attempts to contribute to this understanding in a small way: although the agent is technically a wheeled agent, it is possible for the legs to move through 1 degree-of-freedom in a way that changes the pattern of locomotion.

In order to contribute to a robot that could be robust enough to cope with a varied environment, the author follows a bio-inspired approach, as many insects are highly robust and are able to deal with numerous changes in their environments. A Continuous-Time-Recurrent Neural Network (CTRNN) controls the agent’s locomotion and morphology, and one aim of this paper is to contribute to an understanding of the dynamics of CTRNN-controlled autonomous agents, particularly those agents that change shape according to their sensory input.

The paper is also of interest because the ability to change one’s dimensions according to the aspects of environment that one perceives is a behaviour that is ordinarily associated with living beings, and as such a cognitive ability, albeit a minor one. This project therefore aims to contribute to work on minimally cognitive behaviour, much of which has been completed by Randall Beer and his colleagues.

The structure of this paper is as follows: Section 2 gives an overview of work with legged robots, clarifies the behaviour-based approach to robot design which this project is heavily influenced by, and explains the type of controller as well as justifying its selection. It also informs the reader of the achievements in the field of minimally cognitive behaviour. Section 3 establishes the configuration of the agent and its specific control system, while specifying the exact environment that the agent will traverse, and explaining the genetic algorithm that will be used to evolve the controller. It also reveals the specific experiments that the agent will undergo.

Section 4 contains the results of the experiments, and section 5 includes analysis of the dynamics of the CTRNN controller, and of the experiments from section 4.

2 Background and Related Work

2.1 Field Robotics and Legged Robots

The area of field robotics requires mobile robots to perform tasks in dynamic and unstructured environments that are difficult or unsafe for human beings to work in; such as underwater, in a volcano, in a nuclear environment or for the military. It is difficult to design a mobile robot that can navigate under such a wide variety of conditions when perception of the environment is limited. It is important to achieve useful locomotion patterns in order to allow a robot to traverse a complex environment.

6

There have been a number of attempts to design a robot that is robust enough to walk across rough terrain1 2 3 4 : the most well-known of these is Big Dog, made by Boston Dynamics5. Other areas of importance in field robotics include the ability to move through tight spaces, such as through apertures in a ‘wall’, or underneath a low ‘ceiling’, as well as the related ability to move across ridges without falling into chasms on either side.

The solution to these problems is very much dependent upon the type of mobile robot, and the reasons behind wanting to solve this problem. One solution would be to design a micro robot that was small enough to move through most apertures. However, a disadvantage of this method could be that a small robot would not be able to cover large areas quickly. Some medium-sized modular ‘snake-like’ robots have been designed for the task or moving through apertures or covering rough terrain6 7, and have had success due to their robustness and comparatively low financial price.

Despite the added cost and complexity, walking robots are still popular, particularly robots with 4 or more legs, due to their stability8 9 10 11. A legged robot might be useful in different situations to a snake robot. A snake robot would indeed be very good at a simple task such as finding something trapped underneath rubble. But it would perhaps be less useful for the carrying of equipment, as part of what makes the snake so versatile is the fact that it can move its entire body, and any part of its body may touch the ground. A legged robot would be more suitable for carrying equipment (just as Big Dog carries army equipment), as it would be able to carry equipment on top of itself in a stable manner, and it has an advantage in that the robot body is carried above the ground and as such can be protected from the environment, whether it be moisture on the ground or a burning hot surface.

2.2 Behaviour-Based Robotics

When considering how to get a robot to perform a useful task such as passing through an aperture, it is necessary to consider some different approaches in the field of artificial intelligence and robotics. Robotics initially began as a way of automating a process: the first autonomous robot was created in 1961, and was used to remove hot pieces of metal from a die casting machine12. The first robots were not mobile and operated in very fixed environments, such as factories. Once researchers began making mobile robots, they looked to the field of artificial intelligence for ways to get a robot to behave intelligently. Artificial intelligence researchers at this time were initially more concerned with disembodied reasoning machines than embodied movement, and in the 1960s and 70s focused on building reasoning computer programs such as chess programs. Mobile robotics researchers were influenced by this, and it seemed natural to work separately on a) intelligence b) perception and c) action, and to try to integrate them. Such work led to the creation of slow mobile robots that could only move in very restricted environments, such as Shakey13. Shakey took a long time to ‘think’ before acting, due in part to available processing power at that time, but also because of the way it have been designed to perceive, reason and act separately.

1 Pearson & Franklin (1984)2 Urwin-Wright, Sanders & Chen (2002)3 Erez & Smart (2007)4 Vernaza et al (2009)5 http://www.bostondynamics.com/robot_bigdog.html6 Zhang and Guo (2009)7 Hirose (1990)8 Donner (1987)9 Pearson & Franklin (1984)10 Delcomyn, Nelson & Cocatre-Zilgien (1996)11 Brooks (1989)12 Nof, S.Y. (1999)13 Nilsson, N.J. (1984)

7

The above methods were characterised as the sense-model-plan-act framework by Rodney Brooks, who criticised the framework for being too dependent on traditional AI principles of disembodied reasoning14. Brooks did not think traditional AI was a good model of biological intelligence, because it depends on the robot pausing to think every time new sensory information is acquired in order to decide on an appropriate action. Brooks thought that this was a very stilted kind of intelligence, and argued that many of these AI-style mobile robots had specially designed environments, never had to deal with the real world, and indeed would never be able to. He argued that human beings and animals continuously sense; our sensing is not paused so that we can perform an action, and that mobile robots should be built according to these principles.

Brooks thought that the AI researchers were mistaken in trying to apply top-down reasoning to an embodied robot: “Most of what people do in their day to day lives is not problem-solving or planning, but rather it is routine activity in a relatively benign, but certainly dynamic, world”15. He argued that any model of intelligence should be built from bottom-up principles, rather than top-down. This led to a new framework that Brooks developed called the ‘subsumption architecture’:

Fig. 1. Sense-model-plan-act framework16

Brooks’ subsumption architecture17

Brooks’ subsumption architecture18 is a robotics framework that involves building capabilities on top of each other. For example, a robot might have to initially avoid contact with objects. Once this capability is successful, it might have to wander around whilst avoiding objects. And once this has been done successfully, it might be expected to explore certain reachable locations while wandering around and avoiding objects. This is a framework where each new activity subsumes the old activity. The success of such a robot is judged by the behaviour of the robot: “[subsumption architecture] makes no distinction between peripheral systems, such as vision, and central systems. Rather the fundamental slicing up of an intelligent system is in the orthogonal direction dividing it into activity producing subsystems”19. This framework is part of a tradition that has become known as behaviour-based robotics. The behaviour-based approach favours a design methodology based on the behaviour that will be produced by the agent, and as such the success of the agent is evaluated by looking at the agent’s behaviour. The agent created by this paper will be designed according to the behaviour-based framework, because the design follows a bottom-up approach to robotics and the success of an agent will be judged by studying its behaviour.

Many behaviour-based robots are built very simply, and are also reactive, which means that they do not build an internal modelling of the environment; the actuators are determined in some way by the

14 Brooks, R.A. (1991a)15 Brooks, R.A. (1991a)16 Brooks, R.A. (1986)17 Brooks, R.A. (1986)18 Brooks, R.A. (1986) 19 Brooks, R.A. (1991b) §4.2

8

immediate information coming through the sensors. Brooks popularised this approach, but there had already been some robots designed in a similar way, including W. Grey Walter’s tortoises20 and Braitenberg’s vehicles21. Braitenberg’s vehicles were the result of a thought experiment that focused on some very simple experiments with imaginary vehicles, whose sensors and motors were connected by simple wired connections that led to seemingly complex behaviours such as fear, aggression and love. His paper led to many simple mobile robots being controlled by basic feed-forward neural networks. Most behaviour-based robots have very minimal sensors, and react to the detection of changes in their environment, with no ‘understanding’ of what these changes mean. One of the cornerstones of Brook’s approach was that any agent had to be situated in an environment and embodied. This meant that agents could not perform abstract reasoning: the intelligent behaviour came from interactions between a body (containing a brain or control system of some kind) and its environment.

2.3 Perceiving Affordances

J.J. Gibson was an ecological psychologist who studied perception in humans and animals. He argued that animals perceive the world directly, without extensive modelling of the environment, which holds many parallels with Brooks’ views on intelligent robot design. Gibson thought that what an animal perceives is based on their ‘ecological niche’, which means their particular relationship with the environment. He was interested in the study of ‘affordances’ – a term he defined himself: “the affordance of anything is a specific combination of the properties of its substance and its surfaces taken with reference to an animal”22. An object in the environment affords different things to different creatures depending on their ecological niche. For example, a bicycle may have ridability for a human, but not for an ant, in part due to size and morphological differences. According to this view, agent and environment are inextricably linked by the concept of affordances: "An affordance cuts across the dichotomy of subjective-objective and helps us to understand its inadequacy"23. Gibson’s view was that animals make use of their environment in order to achieve goals, and that they are able to do this because they are part of a sensory-motor feedback loop. An example of this is the bluefin tuna24, which uses currents in its environment to travel fast: its biological makeup does not allow the tuna to travel at such high speeds, however other fishes that have not adapted to this ecological niche would not travel fast in these currents as the bluefin tuna does, as it is the combination of the body of the tuna plus its environment that allows for such fast travel.

Gibson’s view on perception was that it was not based on what was objectively in the world, but on affordances, and he therefore thought that such perception depends on the body of the perceiver, which exists to allow an animal to achieve a goal. Because many goals do not require complex sensory information, an animal need not perceive highly defined images in order to complete a simple task. The perception could be as simple as noticing a relevant change in the environment. The agent that this paper will soon describe and experiment on will be designed according to this principle: it will have an array of simple sensors, the purpose of which will not be to identify objects, but to merely detect changes in the environment.

Uğur and Şahin made use of the concept of perceiving affordances when designing a robot that was able to traverse an environment with a number of obstacles and environmental constraints such as gaps and ramps25. Their paper investigated the possibility of a robot learning to perceive affordances, and as a

20 Walter, W. G. (1963)21 Braitenberg, V. (1986)22 Gibson, J. J. (1977)23 Gibson, J. J. (1977)24 Clark, A. (1999)25 Uğur, E. and Şahin, E. (2010)

9

result learning which apertures could be navigated through. Their method consisted of two phases: exploration and learning. In the exploration phase, the robot “executed all of its behaviours within a given environment and recorded its experience”26. In the learning phase, the robot calculated the relevant features that were extracted from an image produced by its range finder, and then used this information in a trained Support Vector Machines to decide whether traversability of the environment was afforded.

Uğur and Şahin argued that their paper shows that “three of the main attributes that are commonly associated with affordances, that is, affordances being relevant to the environment, providing perceptual economy, and providing general information, are simply consequences of learning from the interactions of the robot with the environment”. It is very probable that learning in this case was responsible for these aspects of affordance perception, and learning methods would be very useful for many aspects of traversability, however this paper intends to focus primarily on traversing a ridge with chasms on either side. It would be difficult for an agent to learn not to fall into a chasm, because it need only fall into a chasm once for its life to end. This is a drawback to ontogenetic (lifetime) learning. For this reason, the author intends to use phylogenetic learning (evolution), where a population of agents ‘learn’ through evolving rather than a single agent learning over the course of a single lifetime.

2.4 Dynamical Systems and Continuous-Time Recurrent Neural Networks

Dynamical systems are complex systems that include variables that interact over time. The world can be viewed as a dynamical system, but in practice, certain variables are abstracted from a larger dynamical system to create a smaller dynamical system worthy of study. This allows us to study the relation between particular variables without confusion27. An organism can be viewed as a dynamical system, but W.Ross Ashby, the cybernetician, argued that it was useful to conceive of organisms plus their environment as dynamical systems: “Given an organism, its environment is defined as those variables whose changes affect the organism, and those variables which are changed by the organism’s behaviour”28. He was a proponent of the idea that brain, body and environment can be treated as a single dynamical system, as well as individual systems. Chiel and Beer29 extended this idea in their work. They believed that “the nervous system cannot be the exclusive focus for understanding adaptive behaviour [because] it continuously receives and responds to feedback both from the movements that it induces in its own periphery and from the surrounding environment”30. Fig. 2 shows Chiel and Beer’s conception of the three dynamical systems of nervous system, body and environment all dynamically affect one another, thus creating a further over-arching dynamical system. Beer thought that dynamical systems as control systems for intelligent robots were superior to computational methods31.

26 Uğur, E. and Şahin, E. (2010), p26327 Ashby, W.R. (1952), p1628 Ashby, W.R. (1952), p3629 Chiel, J. H. and Beer, R. D (1997)30 Chiel, J. H. and Beer, R. D (1997), p55531 Beer, R. D. (1995a)

10

The view that the entire system contributes to behaviour is held by researchers in different fields, including biology32 33, philosophy34 35, ecological psychology36 and cybernetics37.

Some have used the tools of dynamical systems theory to study the behaviour of animals38, and it has also been a fruitful set of tools for studying the behaviour of simple robots and electronic agents. Beer39

popularised the Continuous-Time Recurrent Neural Network (CTRNN), which was extended from Grossberg’s work40, as a particular dynamical systems tool for studying the behaviour of adaptive agents. Fig. 3 describes the equation of a CTRNN. Such neural networks can be useful for studying adaptive behaviour, because they are universal approximators of smooth dynamics41 whilst also being a very simple dynamical system model.

Fig. 3 y = state of each neuron

τ = time constant of each neuron (τ > 0)wji = strength of connection jth to ith neuronθ = bias of each neuron

σ(x) = logistic activation function = 1 (1 + e - x)

I = external input

2.5 Minimally Cognitive Behaviour

Yamuachi & Beer argued that “many tasks, such as landmark-based navigation and maze negotiation, require that an agent also be able to integrate perceptual information over time in order to determine the appropriate course of action”42. Similarly, in order for an agent to navigate an environment whilst perceiving affordances, simple reactive behaviour will not suffice. Its behaviour must be related to past behaviour to some extent. This can to some extent be achieved with a CTRNN, which integrates over time.

Beer was interested in studying minimally cognitive behaviour, which he defined as “the simplest behavior that raises cognitively interesting issues”43. Such issues for him included categorical perception, attention, persistence, and the ability to manipulate objects in the environment.

His experiments in minimally cognitive behaviour44 45 were performed on circular agents with an ‘eye’ that consisted of between 5 and 7 distance sensors that were uniformly distributed over an acute angle. These experiments mostly related to visually-guided behaviour. Some agents were controlled by feedforward neural networks, and others were controlled by CTRNNs. A discrimination experiment46

investigated an agent’s ability to discriminate between two different types of falling objects: in one

32 Maturana, H.R. and Varela, F.J. (1980)33 Varela, F.J., Thompson, E. and Rosch, E. (1991)

34 Noë, A. and O'Regan, J.K. (2002)35 Clark (1999) p34536 Gibson, J. J. (1977)37 Ashby, W.R. (1952)38 Henson, M.S. and Hayward, J.L. (2010)39 Beer, R.D. (1995b)40 Grossberg, S. (1969)41 Funahashi, K. and Nakamura, Y. (1993)42 Yamauchi, B. M. and Beer, R. D. (1994) p243 Beer, R. D. (1996) p244 Beer, R. D. (1996)45 Slocum, A.C., Downey, D.C. and Beer, R.D. (2000)46 Beer, R. D. (1996)

11

experiment, agents were evolved to discriminate between circles and diamonds, and in another, between circles and horizontal lines. The agent had to ‘catch’ the circles and avoid the other shapes. The agents were controlled by feedforward neural networks. The best performing agents tended to foveate the object, scan it, and then either remain in place or avoid the falling object depending on whether it was a circle or otherwise. This was a complex task, and was achieved using an evolutionary algorithm. The ability took much longer to evolve than simpler tasks. The best evolved agent had a very high mean fitness (99.83%) and all objects were correctly identified.

Another experiment of Beer’s focused on a simple agent’s ability to perceive affordances. In this particular experiment, the agent had to judge the pass-ability of a narrow opening, created by two walls that fell towards the agent from above, as shown in Fig. 447. If it was possible to pass through the gap, the agent had to move to centre the gap so that it passed through. If the agent judged the gap to be impassable, it had to move horizontally far enough to avoid the falling walls. Successful agents were clearly able to distinguish between passable and impassable apertures.

The most common way to configure a CTRNN is through evolutionary algorithms; the most common of these is the real-valued genetic algorithm. Beer argued that “one significant advantage of an evolutionary approach is that it minimizes a priori theoretical assumptions and thus allows the space of possible brain-body-environment systems capable of generating a particular behavior to be explored”48

CTRNNs are a useful robot controller because they allow for easy transfer from simulation to hardware reality. As the aim of this paper is to contribute to the long term goal of creating a physical robot that can move across a variety of terrains and through apertures, it is essential that in theory the controller can transfer from software to hardware. It has been show by Vigraham & Gallagher49 that it is possible to transfer a CTRNN from software to hardware, and implementations of this are already available50 51. For these reasons, and those stated above, a CTRNN will be used as a controller for this project.

3 Methodology

3.1 The Agent

The agent is simulated within a 2D environment, and can be viewed from a top-down ‘bird’s eye’ perspective. It is comprised of a circular torso with radius 1.5cm, and two transparent legs with wheels of radius 0.6cm attached to the end of them. Despite the 2D representation, the agent can in some small sense be considered to be in a three dimensional environment, as the wheels are treated as ‘lower’ than the torso and legs of the agent, and so only the wheels touch the floor of the environment. The agent has an array of 5 distance sensors at the front at a range of ±50o, which act as an eye, as well as two distance sensors at angles of 165 o and 195 o at the back. Each wheel has one motor, and can move either forwards or backwards.

Each leg is comprised of two sections: an upper limb of length 1cm and a lower limb of length 3cm. They are connected by a joint that has 1 degree of freedom with a range of between 0 and π/2. Two different styles of agents will be compared: the first will be an agent where the morphology and locomotion pattern of the agent are decoupled. The second will be where they are coupled. This will be achieved by

47 Slocum, A.C., Downey, D.C. and Beer, R.D. (2000)48 Beer, R.D. (1997) p2249 Vigraham, S.A. and Gallagher, J.C. (2006)50 Vigraham, S.A., Garverick, S. and Gallagher, J.C. (2006)51 Boddhu, Vigraham & Gallagher (2006)

12

varying the direction of the degree of freedom in each case; effectively creating two different agents with different joints for two different experiments. This is shown in Figs. 5 and 6.

Fig. 5. Fig. 6.

Agent 1Morphology and locomotion are de-coupled

Translational joints mean the lower limbs can be pulled in under the agent. From a bird’s eye view this is shown as the lower limbs moving in and out for the distance of the dotted line.

Agent 2 Morphology and locomotion are coupled

Rotational joints mean the limbs can bend in front of the agent, at an angle as far as the dotted line. This changes the direction of the wheels, which move perpendicular to the lower limbs.

The agent’s wheels can move in a forward or backward direction, which prevents the agent from turning on its side and moving sideways to traverse the environment. This allows us to analyse the behaviour of an agent which is changing its shape, which is important because legged robots in the real world might not be able to rely on moving sideways to get through a tight space, as they will have more than 2 legs, and might not be able to fit them all through a gap or along a ridge.

The joint angles will vary throughout the trial of the agent, as will the wheel speeds: each will be dependent on sensor information as well as being dependent upon each other. If the agent’s wheels touch the chasm at any point, the agent ceases to move, because it has fallen into a chasm and can no longer traverse the environment. If the agent touches a wall of the environment, it moves back to the position it was at during the previous time-step.

3.2 The Controller

The agent is controlled by two different Artificial Neural Networks: the main controller is a Continuous-Time Recurrent Neural Network (CTRNN), but there is also a feed-forward neural network that connects the sensors to the neurons of the main controller. The motors and joint angles are each related to a single neuron in the CTRNN by a logistic activation function.

A CTRNN is useful as a controller because it can approximate all smooth dynamics, yet is relatively simple. It is also a closer model to neurobiological systems than a standard artificial neural network.

The CTRNN has fixed weights in the range ±16, and time constants had a range of 0.5 to 10. The rate of decay is

13

0.1. The feed-forward ‘sensor weights’ are also fixed in the range ±16. Fig. 7.

Centre-crossing conditions are satisfied in order to calculate the bias values. The null-surfaces of each neuron in a centre-crossing recurrent neural network cross exactly at their centres of symmetry, so the value of centre-crossing is that it prevents the neurons from saturating too early. The bias terms, θ , are calculated using the following formula:

where wji is the weight from the jth to the ith neuron, and N is the number of neurons in the CTRNN 52.

3.3 The Environment

The agent will be placed inside a walled rectangular environment of width 29cm and height 35cm. Inside this environment there will be a number of chasms, and the agent will be assigned a variety of tasks which involve moving from one side of the environment to the other whilst avoiding falling into the chasms. The agent can be considered to have fallen into a chasm if either of its wheels touches the chasms. This gives the agent a number of different ways of traversing the environment, including passing between two chasms, extending its legs to pass over a chasm, and travelling around a chasm. Fig. 8.

3.4 Evolution

The task of creating an agent that can change its shape to traverse its environment is suited to phylogenetic learning (evolution) rather than ontogenetic learning (learning over the course of a lifetime), because some tasks would be very difficult to learn without ‘dying’. For example, if an agent falls down a chasm and cannot get out, it will not have the opportunity to learn during its lifetime alternative solutions.

Evolutionary algorithms53 are a common method for choosing effective weights for robots controlled by CTRNNS54 55 56, particularly real-valued genetic algorithms. This paper makes use of the simple Microbial Genetic Algorithm57, a tournament-based algorithm with elitism built in. The Microbial GA works in the

52 Mathayomchan and Beer (2002)53 Holland, J. (1992)54 Beer, R.D. & Gallagher, J.C. (1992)55 Bongard, J., Zykov, V & Lipson, H. (2006) 56 Laksanacharoen, S., & Fife, A. (2003)57 Harvey, I. (1996)

14

following way: a population of agents is created, where each agent is represented by a genotype, which is a vector containing values that represent the weights and time constants of an agent. Two genotypes are selected at random from the population. The fitness values of each genotype are calculated according to a fitness function, and then compared. The genotype with the highest value is the “winner” and the other is the “loser”. The losing genotype has some of its genes replaced by the genes of the winner according to a recombination rate. Some of the loser’s genes are also mutated according to a mutation rate. A single tournament has been described, but the GA will continue for many more tournaments until the fitness is substantially higher. The fitness function should be designed so that the higher the fitness, the closer an agent is to completing a desired task.

A population of 30 agents was used in this case. The number of neurons in the CTRNN was 4 and the number of sensors was 7. This led to 28 sensor weights, 16 CTRNN weights, 4 tau values and 4 bias values, which could have created a genotype with 52 parameters. This is a large number of parameters to evolve and would take a long time. The bias values can be removed from the genotype by taking advantage of the centre-crossing condition, as this calculates bias values depending on the weights, so there is no need to evolve them. This would reduce the number of parameters to 48. In addition to this, a large number of parameter can be reduced by designing the CTRNN to be bilaterally symmetric58. This means that many parameters can be used twice in the same system, thus reducing the number of parameters again by almost half to 26.

3.5 List of Experiments

The agents with both translational and rotational joints were subjected to a series of experiments, where they had to traverse the environment which had chasms in different numbers and locations. It was decided to have chasms to avoid rather than walls, because this set-up allows an agent to traverse an environment in different ways: it can draw its limbs in to pass between two chasms, or stretch them far out to travel over them. Using chasms allows us to view multiple behaviours in a 2D environment.

The agent will be presented with a three different increasingly challenging environments that it must traverse successfully:

Fig. 9. Fig. 10. Fig. 11

1 Chasm Environment

0 5 10 15 20 25

0

5

10

15

20

25

30

2 Chasm Environment

0 5 10 15 20 25

0

5

10

15

20

25

30

3 Chasm Environment

0 5 10 15 20 25

0

5

10

15

20

25

30

58 Slocum, A.C. Downy, D.C., & Beer, R.D. (2000)

15

In the 1 chasm environment, the agent must extend its legs in order to pass over the chasm. In the second environment, it must do the opposite: retract its legs underneath it or tuck them forwards to traverse the ridge between two chasms. In the 3 chasm environment, it must first retract and then extend its legs based on its sensory input.

Each agent will be tested with 3 variations in staring position, as shown in fig. 12. The reason for this is to begin investigating whether the controller of the agent can generalise so that the task can be achieved from different starting positions. The three conditions are fairly similar to ensure the agent has a likelihood of moving through the gap over a short number of generations: it is very unlikely that an agent will evolve to move through a gap with no reward on the other side, unless it is positioned in such a way as to be able to do so without much difficulty. Fig. 12.

Once the controller can generalise through the 3 conditions, the controller will be tested 20 times starting from different locations within the range of the 3 different previous starting points. Differences between behaviour of agents with translational joints and rotational joints will be recorded, and the above statistics should help determine which is the most successful agent. Comparisons between length of time to evolve will be also be taken.

As stated previously, the controller of the agent will be evolved so as to produce the best agent for completing the task of traversing past chasms. The fitness function to be used will be a combination of three different terms:

1.) The distance between the agent’s final position and the ‘finishing line’. The finishing line is the maximum line of the furthest chasm in the environment. The agent’s position in this case is calculated to be the part of the body or wheels that is furthest away from the finishing line.

2.) In order to convince the agent to remain in the centre of the x axis, the difference between the x co-ordinate of its body and a ‘centre line’ (12.5cm) is calculated. The mean of these differences is another potential term in the fitness function.

3.) It seems likely that a successful agent will move in a relatively straight path, therefore another term for consideration in the fitness function will be the mean of the difference between the forwards-facing angle of the agent and the angle it would be at if it were moving parallel to the y axis.

All terms will be normalised to be between 0 and 1, therefore the maximum the overall fitness could be would be 1. It is often possible when evolving robot behaviour to get unexpected or unwanted behaviours, and so for this reason, a variety of fitness functions will be used that include one or more of these terms, in the interest of producing the best possible behaviour. A fitness function that contained all three terms could be expressed as:

Φ = δ ∗ χ ∗ α Where δ = distance , χ = centre line difference and α = angle difference

All scenarios that the agents will be presented with will be possible to traverse. There will be no ridges too narrow and no chasms to wide to traverse.

3.6 Implementation

16

0 5 10 15 20 25

0

5

10

15

20

25

30

The agent and environment were both simulated in MATLAB. Fig. 13 shows the structure of the program used for the simulation. Some of the main functions are explained below:

Fig. 13

GenesToCTRNN – Converts the Genotype into CTRNN weights and tau values, as well as sensor weights for the feedfordward neural network. It adds gain values, and reshapes a vector-shaped genotype into a number of matrices.

WallDist – Calculates the distance between each sensor and the wall ahead of it, to be used as sensory input.

ChasmDist – Calculates the distance between each sensor and the chasm ahead of it, if any, also to be used as sensory input.

UpdateBody – Wheel speeds are determined by nodes in the CTRNN. The distance an agent moves in one timestep is calculated, as well as the direction it is moving in.

CalcAgentLimbs – The length of each leg segment and the angle of each joint are calculated

WallBounds – Stops the agent from crossing the wall line.

ChasmBounds – If an agent’s wheels touch a chasm, the agent is paralysed.

17

4 Results

Key:

CTRNN graphs

Node 1: left wheel force

Node 2: left joint angle

Node 3: right joint angle

Node 4: right wheel force

Agent Position 1: Body x co-ordinate=9.8953;Body y co-ordinate =0.2279;Forward facing angle =pi/18*8;

Agent Position 2: Body x co-ordinate =12.5;Body y co-ordinate =0;Forward facing angle =pi/2;

Agent Position 3: Body x co-ordinate = 15.1047;Body y co-ordinate = 0.2279;Forward facing angle = pi/18*10;

1 chasm environment

Agent with translational joints:

Fig. 14 Position 2 – agent behaviour

0 5 10 15 20 25

0

5

10

15

20

25

30

Fig. 15 Position 1 – agent behaviour

0 5 10 15 20 25

0

5

10

15

20

25

30

The behaviour of agent 3 was the mirror image of that of agent 1. The chasm in the diagrams above is obscured, as the agent is moving over it. In each case the agent moves roughly in a straight line, rather than turning and facing the chasm head on.

18

0 5 10 15 20 25 30 35 40 450

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

node1node2node3node4

The agent was able to generalise its behaviour to some extent, as over 20 test runs, each beginning with a random starting position in the range between position 1 to position 3, the agent could successfully traverse the environment. The successful agent in this case increased its nodes to the maximum value very quickly (see fig. 16). Its speed was consistently maximum speed, and its legs very quickly extended to allow the agent to move over the chasm. However, the behaviour did not generalise when the agent was positioned much further outside of the range (see fig. 17), because its insistence in moving in a straight line regardless of sensory input meant that at certain angles, it would fall into the chasm. The agent in this case adapted its joint angles to adapt to the environment, but not the speed of each wheel, and therefore its ability to turn. Very similar behaviours were shown, when the fitness functions of Distance, and Distance * MeanCentreDifferences were used.

Fig. 16

0 5 10 15 20 25 30 350

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

time

activ

atio

n of

nod

e

node1node2node3node4

Fig. 17

0 5 10 15 20 25

0

5

10

15

20

25

30

Agent with rotational joints:

The agent with rotational joints has a very similar behaviour to the agent with translational joints, when traversing a 1 chasm environment.

Fig. 18 Position 1 Fig. 19 Position 2 Fig. 20 Position 3

19

0 5 10 15 20 25

0

5

10

15

20

25

30

0 5 10 15 20 25

0

5

10

15

20

25

30

0 5 10 15 20 25

0

5

10

15

20

25

30

This agent was able to generalise its behaviour to the same extent as the agent with translational joints, as each have entended their legs out as far as possible, so they work as the same agent and display the same behaviour.

2 chasm environment

Agent with translational joints:

The agent in position 1 was able to evolve an interesting behaviour that allows it to pass across the ridge between two chasms: it moves in a straight line, but as its left wheel approaches the left chasm, it turns around and travels through the chasm backwards (see fig 21). After the turn it is facing a more appropriate angle at which to proceed. The agent starting at position 3 showed very similar behaviour, due to the bilateral symmetry of the agent’s control system.

Fig. 22 shows a plot of the activation of the CTRNN nodes (y axis) over time (x axis). The blue and green lines represent the nodes that related to the wheel speeds of the agent. So in the 2 chasm environment, the wheel speeds seem to adapt and not the joint angles, whereas the reverse is true in the 1 chasm environment.

Fig. 21 Position 1 Fig. 22 Position 1

0 5 10 15 20 25 30 350

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

20

It took a relatively small number of tournaments to evolve this behaviour: Fig. 23 shows the maximum fitnesses increasing over the number of tournaments of the genetic algorithm. The graph shows the fitness rise sharply between 0 and 200 tournaments, and then rise again at further points after periods of punctuated equilibria. The maximum fitness is not reached, because one of the terms of the fitness function is the mean distance from the centre line: unless the agent permanently moved along the centre line, it would not score a fitness of 1.

Fig.23

0 1000 2000 3000 4000 5000 60000.92

0.93

0.94

0.95

0.96

0.97

0.98

number of tournaments

max

imum

fitn

ess

Fig. 24 Position 2

The agent that begins at position 2 retracts its legs just the right amount to be able to move along the ridge, and then continues in this position all the way across the environment, as shown in fig. 24. The

21

agent can generalise this behaviour to a small extent, in a similar way to the agents traversing the 1 chasm environment, as shown in figs. 25-27. The unusual behaviour of positions 1 and 3 when evolved in isolation are no longer present. However, the agent is only able to move in a straight line, as with the 1 chasm environment. Therefore it will not generalise when the starting position is moved far.

Fig. 25 Position 1 - generalised

0 5 10 15 20 25

0

5

10

15

20

25

30

Fig. 26 Position 2 - generalised

0 5 10 15 20 25

0

5

10

15

20

25

30

Fig. 27 Positon 3 - generalised

0 5 10 15 20 25

0

5

10

15

20

25

30

Agent with rotational joints:

The agent in positions 1 and 3 successfully moved across the ridge by turning gently in its path.

Fig. 28 Position 3

0 5 10 15 20 25

0

5

10

15

20

25

30

Fig. 29.

0 5 10 15 20 25 30 350

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

The green line represents the left wheel force, which slows a little towards the end due to the agent turning. The blue line shows that the right wheel was moving at full speed constantly. The

22

red and black lines show at the agent takes a number of timesteps to tuck its legs forward.

The agent in position 2 tucks its legs forward and moves across the ridge (se fig. 30). Fig. 31 shows the activation of the CTRNN nodes (y axis) over the number of timesteps (x axis). The green and blue lines which represent the wheel speeds move immediately to 1, which means the vehicle is moving at full speed, but conversely the red and black lines move immediately to 0, so the legs are tucked in as far as they can go.

Fig. 30 Position 2 Fig. 31

0 5 10 15 20 25 30 350

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

The agent was able to generalise its behaviour a small amount (see fig. 32), but, like the agent with translational limbs, would only move in a straight line when doing so.

Fig. 32 Position 1 using generalised genetic algorithm.

0 5 10 15 20 25

0

5

10

15

20

25

30

3 chasm environment

Agent with translational joints

The agent with translational joints was not able to traverse the more complex environment after evolving a controller for 200 generations, with the fitness function Φ = δ ∗ χ.

23

Fig. 33

0 5 10 15 20 25

0

5

10

15

20

25

30

Fig. 34

0 5 10 15 20 25

0

5

10

15

20

25

30

Fig. 35

0 5 10 15 20 25

0

5

10

15

20

25

30

Agent with rotational joints:

Only the agent in position 2 was able to evolve to traverse this environment (see fig. 36), and it was not possible to get this agent to generalise its behaviour. Fig. 37 shows the dynamics of the CTRNN over time. The green and blue lines (blue is hidden underneath green) show the wheel speeds at maximum throughout the trial. The black and red lines represent the joint angles. Between 0 and 15 units of time the legs are immediately tucked in, and after this they are very quickly extended so the agent can travel over the third chasm, which is obscured the the agent in fig. 36.

Fig. 36

0 5 10 15 20 25

0

5

10

15

20

25

30

Fig. 37

0 5 10 15 20 25 30 350

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

24

0 5 10 15 20 25

0

5

10

15

20

25

30

Despite positions 1 and 3 not evolving to traverse the environment, some interesting behaviours were displayed: For example, the agent starting from position 1 turned around and travelled through backwards (fig. 38). If it had travelled forwards with its legs tucked in front of it, it would have hit the 3rd

chasm earlier, so it is a useful strategy in this case to travel backwards, although if the agent had reached its final destination, travelling backwards would not have provided any extra benefits.

FFig. 38

5 Analysis and Discussion

The results show that it was possible to evolve a CTRNN-controlled agent to adapt its morphology to the environment, but this was only achieved in a fairly limited way. The controllers could be said to generalise to different starting positions, but only in a limited way: In all cases of generalisation, the agent was simply moving in a straight line, and did not seem to respond to different sensory input ( see figs. 25-27).

However, despite this, it has now been shown this it is possible to evolve an agents joint angles to respond to the environment, and also that its wheel speeds can be evolved for the same purpose. Fig. 21 shows an example where wheel speeds and joint angles change over time to traverse an environment. For all 3 environments it was possible to evolve at least one agent that could traverse it.

Agents with rotational joints, i.e. agents whose morphology and locomotion patterns were coupled, were slightly more successful than the agents whose morphology and locomotion patterns were decoupled. An agent with rotational joints was more successful than the translational jointed agent when traversing the 3 chasm environment, although without more data it is not possible to say whether rotational joints were significantly more successful than translational joints. A qualitative analysis of the behaviour, however, shows that the rotational-jointed agent tended to produce more interesting behaviour. For example, fig. 38 shows an agent turning backwards, in order to avoid its legs pointing forwards and hitting the 3rd

chasm earlier. This behaviour increases the fitness of the agent.

Fig. 21 shows a translational agent moving forwards in a straight line, then turning around and travelling backwards at a slightly different angle to before the turn, in order to move along the ridge between 2 chasms. It works, but in the real world, travelling backwards would not be ideal, due to fewer sensors on the back of the agent. In order to stop the agent turning in this way, a term in the fitness function could be introduced that discourages the agent from turning drastically.

25

In cases where a good solution did not evolve, it is likely that the simulation was not run for enough tournaments. The number of tournaments was capped at 6000, which is roughly 200 generations, due to a population size of 30. The reason for capping the number of generations at this number was time constraints brought about by the slow nature of MATLAB. The author regrets not choosing a faster language such as C, and does not recommend MATLAB as a useful language for long evolutionary processes. Given further time, it might also be useful to increase the length of simulation time, thus allowing slower moving agents the chance of success.

Although the agents were often successful at moving across the environment when evolved separately, it was difficult to evolve interesting behaviour when trying to create an agent that could generalise its behaviour to different starting points. When the agents were successful, they always moved in a straight line. If the position of an agent was moved slightly out of the accepted range, it would fall into a chasm. As stated above, it is possible that a higher number of generations would solve this problem, but it is likely also that the fitness function was not adequate. Another method that could improve the movement of the agents could be to introduce a specific goal location (a point in space) that the agent needs to reach. This way agents in positions 1 and 3, for example, would be more likely to employ a curved trajectory, if its goal was a specific point at the top centre of the environment.

It is difficult to decide on a good fitness function, because many of the terms conflict with each other, and it is hard to know what weight of importance each term should have in the function. The best fitness function in this instance was Φ= δ ∗ χ. An evolved solution is difficult to obtain for complex situations, and it may also be useful to compare the success of this evolutionary approach with that of a learning algorithm. For example Ugur & Sahin59 programmed a robot that learnt to traverse the environment, including moving through gaps, which is a similar task to the one of this paper, although this robot only traversed simple environments: it did not have to deal with a variety of chasms in the same environment. It would be useful to find out if such a robot would be better at generalising its behaviour to deal with the sorts of scenarios presented in this paper.

One feature that all observed agents had in common was that their legs moved in or out very quickly. Usually after the first 3 timesteps, the agent had moved its legs to the position they needed to be in to traverse the environment. This means that the neurons controlling the leg positions change very abruptly to the maximum or minimum values. The same is also true of the nodes that control the wheel speeds: they often move very quickly to 0 or 1. Evolved agents tend to be agents that move, so most agents nodes will be consistently at 1. The result of this is that the agent will be unlikely to turn, as the ability to turn rests on one wheel moving faster than another. This explains why the agents usually moved in a straight line, following the path of least resistance. It would therefore be very useful to stop the agent’s joint angles from changing so quickly, which could perhaps be tried by varying the value of tau – the rate of decay of the nodes of the CTRNN.

In order to make the agents in this experiment simpler and easier to analyse, the CTRNN was restricted to 4 nodes: one for each wheel and one for each joint angle. However, without this restriction, more nodes may have allowed for faster evolution, and in fact, it may be that the small number of modes meant that certain solutions were not possible to be found. A CTRNN in a universal approximator of smooth dynamics, but only providing enough nodes are used. A problem with using CTRNNs is that one does not know what an adequate number of nodes is. Evolving a CTRNN can be satisfying when it produces interesting behaviour, but frustrating when it does not, because there are few guidelines for what to do when something does not work.

59 Uğur, E. and Şahin, E. (2010)

26

6 Conclusion

The behaviour of the most successful agents in this paper is still too restrictive to attempt to move the agents into a real-world environment, however it has been possible to evolve controllers that can cope with specific situations, and can traverse different types of environments, including, in one case, an agent that moves between 2 chasms and then over the top of another one, each time adapting its joint angles.

Further work will be necessary to determine whether the coupling of morphology and locomotion patterns improve behaviour, although preliminary findings suggest that coupling the two may be an improvement. It would be useful to analyse the gait of a walking robot that has to change its morphology in similar experiments.

It would be interesting to adapt an agent that is also able to respond differently if it is not possible to move through a gap: all the experiments in this paper have involved environments which were possible to traverse, but it is also important to stop trying when a situation is clearly impossible, and to try something new.

Alternative evolutionary algorithms could be used to encourage agents to improve their fitness. For example, a netcrawling algorithm has been proven to be more effective than a population-based evolutionary algorithm in some types of complex environments60, through their use of travelling along “neutral ridges” of the fitness landscape. However, it would also be useful to try a non-evolutionary technique for comparison, such as the support vector machines used for learning to perceive affordances in Ugur & Sahin’s Traversability61 paper.

60 Barnett, L (2001)

61 Uğur, E. and Şahin, E. (2010)

27

7 Bibliography

Ashby, W.R. (1952). Design for a Brain. London, UK: Chapman & Hall.

Barnett, L (2001) Netcrawling – Optimal Evolutionary Search with Neutral Networks. Proceedings of the 2001 Congress on Evolutionary Computation CEC2001 (pp. 30-37). IEEE Press.

Beer, R.D. & Gallagher, J.C. (1992). Evolving Dynamical Neural Networks for Adaptive Behavior. Adaptive Behavior, 1(1) 91-122.

Beer, R. D. (1995a). Computational and Dynamical Languages for Autonomous Agents. In Port, R. F. & van Gelder, T. (Eds.), Mind as Motion: Explorations in the Dynamics of Cognition. Massachusetts: Bradford Books, MIT Press, 121-147.

Beer, R.D. (1995b). On the Dynamics of Small Continuous-Time Recurrent Neural Networks. Adaptive Behavior, 3(4), 469-509.

Beer, R.D. (1996). Toward the Evolution of Dynamical Neural Networks for minimally Cognitive Behavior. In P. Maes, M. Mataric, J. Meyer, J. Pollack and S. Wilson (Eds.), From animals to animats 4: Proceedings of the fourth international conference on simulation of adaptive behaviour. MIT Press, 421-429.

Beer, R.D. (1997). Dynamical Systems and Embedded Cognition. Chapter 12 in K. Frankish and W. Ramsey (Eds.), The Cambridge Handbook of Artificial Intelligence.

Boddhu, S., Vigraham, S.A., and Gallagher, J.C. A reconfigurable analog neural network for evolvable hardware applications: intrinsic evolution and extrinsic verification. In Gary G. Yen, Simon M. Lucas, Gary Fogel, Graham Kendall, Ralf Salomon, Byoung-Tak Zhang, Carlos A. Coello Coello and Thomas Philip Runarsson (Eds.), Proceedings of the 2006 IEEE Congress on Evolutionary Computation, Vancouver, Canada.

Bongard, J., Zykov, V. & Lipson, H. (2006). Resilient Machines Through Continuous Self-Modeling. Science, 314( 5802) 1118-1121.

Braitenberg, V. (1986). Vehicles: Experiments in Synthetic Psychology. Massachusetts: Bradford Books, MIT Press.

Brooks, R.A. (1989). A robot that walks: Emergent behaviors from a carefully evolved network.28

Neural Computation, 1:253-262.

Brooks, R. A. (1991a). Intelligence Without Reason. Technical Report, Massachusetts Institute of Technology.

Brooks, R. A. (1991b). Intelligence without representation. Artificial Intelligence, 47, 139–159.

Chiel, J. H. and Beer, R. D (1997). The brain has a body: adaptive behaviour emerges from interactions of nervous system, body and environment. Trends in Neuroscience, 20, 553-557.

Clark, A (1999). An Embodied Cognitive Science? Trends In Cognitive Sciences 3(9), 345-351.

Delcomyn, F., Nelson, M.E. and Cocatre-Zilgien, J.H (1996). Sense organs of insect legs and the selection of sensors for agile walking robots. International Journal of Robotics Research 15, 113-127.

Donner, M. (1987). Real-Time Control of Walking. Cambridge, Massachusetts: Birkhauser Boston.

Erez, T. and Smart, W.D (2007). Bipedal Walking on Rough Terrain Using Manifold Control. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 1539-1544.

Funahashi, K. and Nakamura, Y. (1993). Approximation of dynamical systems by continuous-time recurrent neural networks. Neural Networks, 6, 801-806.

Gibson, J. J. (1977). The Theory of Affordances. In R. Shaw & J. Bransford (Eds.), Perceiving, Acting, and Knowing: Toward an Ecological Psychology (67-82). Hillsdale, New Jersey: Lawrence Erlbaum.

Grossberg, S. (1969). On learning and energy-entropy dependence in recurrent and nonrecurrent signed networks. Journal of Statistical Physics 1, 319-350.

Harvey, I. (2011). The Microbial Genetic Algorithm. Advances in Artificial Life: ECAL 2009, Part II, LNCS (5778), 126-133.

Henson, M.S. and Hayward, J.L. (2010). The Mathematics of Animal Behavior: An Interdisciplinary Dialogue. Notices of the American Mathematical Society, 57(10), 1248-1258.

Hirose,S. (1990). Design and Control of a Mobile Robot with an Articulated Body. The International Journal of Robotics Research, 9(2) 99-114

Holland, J. (1992). Adaptation in Natural and Artificial Systems. Cambridge, MA. : MIT Press.

Jakobi, N. (1997). Evolutionary Robotics and the Radical Envelope-of-Noise Hypothesis. Adaptive Behavior 6. p325.

Laksanacharoen, S., & Fife, A. (2003). The Feasibility of CTRNN Controllers for an Insect-like Robot's Leg Model. The Journal of KMITNB, 13(2).

Mathayomchan and Beer (2002) Center-Crossing Recurrent Neural Networks for the Evolution of Rhythmic Behavior. Neural Computation, 14(9) 2043-51.

Maturana, H.R. and Varela, F.J. (1980). Autopoiesis and Cognition: The Realization of the Living. Boston Studies in the Philosophy of Science, Vol. 42, D. Reidel Publishing Company.

Nilsson, N.J. (1984). Shakey The Robot. SRI AI Center Technical Note 323.

29

Noë, A. and O'Regan, J.K. (2002). On the brain basis of visual consciousness: a sensorimotor account. In Noë A. and Thompson, E. (Eds), Vision & Mind: Selected Readings in the Philosophy of Perception. Massacheussets: MIT Press.

Nof, S.Y. (1999). Handbook of Industrial Robotics (2nd Edition). John Wiley & Sons, 3-5.

Pearson, K.G. and Franklin, R. (1984). Characteristics of leg movements and patterns of coordination in locusts walking on rough terrain. International Journal of Robotics Research, 3, 101-112.

Slocum, A.C. Downy, D.C., & Beer, R.D. (2000). Further Experiments in the Evolution of Minimally Cognitive Behavior: From Perceiving Affordances to Selective Attention. From Animals to Animats 6: Proceedings of the Sixth International Conference on Simulation of Adaptive Behavior, 430-439.

Uğur, E. and Şahin, E. (2010). Traversability: A case study for learning and perceiving affordances in robots. Adaptive Behavior, 18(3-4), 258-284.

Urwin-Wright, S., Sanders, D. and Chen, S. (2002). Terrain prediction for an eight legged robot. Journal of Robotic Systems, 19(2), 91-98. ISSN 0741-2223.

Varela, F.J., Thompson, E. and Rosch, E. (1991). The Embodied Mind, MIT Press

Vernaza, P., Likhachev, M., Bhattacharya, S, Chitta, S., Kushleyev, A. and Lee, D.D. (2009). Search-based planning for a legged robot over rough terrain. Robotics and Automation, 2380-2387.

Vigraham, S.A. and Gallagher, J.C. (2006). CTRNN-EH in Silicon: Challenges in Realizing Configurable CTRNNs in VLSI. In Gary G. Yen, Simon M. Lucas, Gary Fogel, Graham Kendall, Ralf Salomon, Byoung-Tak Zhang, Carlos A. Coello Coello and Thomas Philip Runarsson (Eds.), Proceedings of the 2006 IEEE Congress on Evolutionary Computation, Vancouver, Canada, 2807-2813.

Vigraham, S.A., Garverick, S. and Gallagher, J.C. (2006). A low voltage nonlinear dac for programming on-chip ctrnns. In Gary G. Yen, Simon M. Lucas, Gary Fogel, Graham Kendall, Ralf Salomon, Byoung-Tak Zhang, Carlos A. Coello Coello and Thomas Philip Runarsson (Eds.), Proceedings of the 2006 IEEE Congress on Evolutionary Computation, Vancouver, Canada.

Walter, W. G. (1963). The Living Brain. W. W. Norton and Company, Inc. (Second Edition).

Yamauchi, B. M. and Beer, R. D. (1994). Sequential Behavior and Learning in Evolved Dynamical Neural Networks. Adaptive Behavior, 2(3), 219-246.

Zhang, S. and Yi Guo, Y. (2009). Bio-inspired locomotion for a modular snake robot. Proc. SPIE 7321, 73210E.

30