PartⅠ Research Trends in the ICT Inspired by Life · tems, we must not only investigate the...

1 Why Draw Inspiration from Life?

The reasons for drawing inspiration fromlife are self-evident. Take, for example, thebrain, the seat of intelligence and awareness.The product of four billion years of sustainedevolution, this unparalleled object is most def-initely not an overnight creation.

1.1 Proximate and Ultimate FactorsTo be “inspired by life” and to build intel-

ligent information and communication sys-tems, we must not only investigate the func-

tions of the organisms currently found onEarth, but penetrate deep to discover how thediverse and ingenious functions observedtoday were acquired over the long course ofevolution. In other words, we need to lookback at the developmental stages in the evo-lution of living organisms — an ultimate fac-tor — in addition to proximate factors, proxi-mate factors being the adaptation of variousorganisms to their immediate environments.The relationship between proximate factorsand ultimate factors is briefly discussed below.No understanding of the historical background

3SAWAI Hidefumi

PartⅠ Research Trends in the ICTInspired by Life

“Where the world ceases to be the scene of our personal hopes and wishes,where we face it as free beings admiring, asking and observing, there we enterthe realm of Art and Science.”

—Albert Einstein—

1 The ICT Inspired by Life－ The Design of ICT System Inspired by Evolution of Life and Brain Functions －

SAWAI Hidefumi

The significance of “biologically inspired paradigm” as the creation of a new paradigm isdescribed showing the scientific position in the course of biological evolution and the hierarchyof nature by referring to the Darwinian theory of evolution as a change of paradigm in the scien-tific revolutions. The structure and functions of brain as a by-product of biological evolution, thevarious information processing models with time and space structures by deducing from thebrain functions, the genetic and evolutionary algorithms as the models of evolution itself, and thealgorithm based on sexual selection, are described. Furthermore, as the results of trend surveytoward the recently developing complex network science research, the useful design guidelinesfor constructing the future information and communications society can be obtained.

KeywordsLife, Evolution, Brain, Information, Network

4 Journal of the National Institute of Information and Communications Technology Vol.55 No.4 2008

and significance of modern life science orcognitive science is complete without anunderstanding of this relationship.

1.2 Nature’s Hierarchy“The most beautiful thing we can expe-rience is the mysterious. It is the sourceof all true art and science. He to whomthis emotion is a stranger, who can nolonger pause to wonder and stand raptin awe, is as good as dead: his eyes areclosed.”


It is believed that the universe was createdabout 15 billion years ago and the Solar Sys-tem and the Earth approximately 4.6 billionyears ago. Primitive life forms are believed tohave appeared around 3.8 – 4 billion yearsago. Chemical evolution during the first600 million years after the formation of Eartheventually led to the origin of life.

Table 1 presents the hierarchy of natureand the disciplines and research topics associ-ated with each level of this hierarchy. At thesmallest scales are elementary particles,

atoms, and molecules. The topics discussed inthis special issue that correspond to this levelof the hierarchy are artificial chemistry inChapter 2 and molecular communications inPartⅡ. The next level of the hierarchy isgenetic. Section 3 of Chapter 1 introducesgenetic algorithms and algorithms based ongenetic duplication. Beyond this is the amino-acid level of this hierarchy; topics associatedwith this level include chemical genetic algo-rithms (CGAs) and chemical genetic program-ming (CGP), presented in Section 4. PartⅡdiscusses molecular communication andmotor proteins as topics at the protein level ofthe hierarchy.

As issues at the cellular level, Section 2 ofChapter 1 discusses neural network modelingbased on the model of neurons and synapses.Note that Ca ion diffusion, a topic discussed inPartⅡ, is also associated with this hierarchicallevel. At the level of tissues and organs,researchers are currently pursuing studies ofbrain machine interfaces (BMIs) to extract andapply neuronal activity outside the brain aspart of efforts intended to investigate con-

Table 1 Nature’s hierarchy

*ESS: Evolutionary Stable Strategy

5SAWAI Hidefumi

sciousness, the state emerging from the brainfunctions. The Origin of Species by Darwin isassociated with the organism level of the hier-archy. Section 3.4 of Chapter 1 discussesalgorithms inspired by the theory of sexualselection. At the level of populations andspecies, perhaps the most characteristic topicis co-evolution involving multiple agents; theacquisition of behavioral strategy by multipleagents in CGP discussed in Section 4.2 isindeed based on the mechanism of co-evolu-tion. At the ecosystem level, the topic present-ed is parallel distributed processing for para-meter-free genetic algorithms (presented inSection 3.2 of Chapter 1), a model inspired bythe migrational strategies of various popula-tions. At the level of the Earth, environmentalproblems constitute the most pressing topics.At the level of the universe, when we considerthat the origin of life is a lifeless molecule, weunderstand that the vast scale of the universeis closely linked to small-scale structures atthe level of elementary particles, atoms, andmolecules. We also consider how what weobserve of the universe (nature’s hierarchy)reflects a mode of observation unique tohumans.

A satisfactory Theory of Everything, orTOE, that provides a unified explanation forall levels of the hierarchy presented in Table 1has yet to be established. Micro-scale theoriessuch as elementary particle theory and quan-tum mechanics and macro-scale theories suchas electrodynamics, Newtonian dynamics, andEinstein’s general theory of relativity haveresolved many of nature’s mysteries. But thehandling of meso-scale problems remain achallenge, and the respective theories remainworks in progress, despite the emergence ofstudies on complexity (complex science)through chaos theory. As is well-known, thediscovery of DNA’s double helix structure byWatson and Crick in 1953 triggered rapidprogress in molecular biology and has pushedthe discipline to its current prominence. In theinformation sciences, research has declined inthe area of artificial intelligence, applicationsof which include expert systems that seek to

embody human expertise. However, studiescontinue on artificial neural networks, artifi-cial life, and artificial chemistry, which havecome to be regarded as fundamental areas ofstudy. Chapters 1 and 2 of this issue intro-duces recent research achievements in thisarea. While Descartes’s formulation of themind-body duality has sequestered physicaland spiritual or mental issues as problems intotally distinct dimensions, recent progress incognitive science and the development oftechnologies for non-invasive measurement ofbrain functions such as f-MRI, MEG, NIRS,and EEG have transformed the study ofhuman consciousness and subjective percep-tion into subjects of natural science (cognitivescience). This is a potentially epochal event inthe history of scientific and technologicaldevelopment.

1.3 Scientific Revolutions as ParadigmShifts

“Possibly we shall know a little morethan we do now. But the real nature ofthings, that we shall never know,never.”


Let’s examine how scientific revolutionscan be viewed as paradigm shifts, drawing onThomas Kuhn’s The Structure of Scientific Rev-olutions[1]. According to Kuhn’s definition,“paradigms” are “universally recognized scien-tific achievements that for a time provide modelproblems and solutions to a community of practi-tioners.” Kuhn states that “when the professioncan no longer evade anomalies that subvert theexisting tradition of scientific practice—thenbegin the extraordinary investigations that leadthe profession at last to a new set of commit-ments, a new basis for the practice of science.The extraordinary episodes in which that shift ofprofessional commitments occurs are the onesknown in this essay as scientific revolutions. Theyare the tradition-shattering complements to thetradition-bound activity of normal science.” As aprime example of a “scientific revolution” hegives the discovery of the general theory of


relativity by Albert Einstein to explain thenature of space and time — a theory that com-pletely overturned the concept of absolutespace and time presented by Newton[2]. New-tonian dynamics failed to explain Mercury’sperihelion movement (precession), the gravita-tional redshift of the light spectrum, or thegravitational diffraction of light, all phenome-na successfully and quantitatively explainedfor the first time by expressing spaces inwhich gravitational fields are found as distor-tions of space by metric tensor {gij} in Rie-mannean space and deriving the equation ofthe gravitational field based on Einstein’s gen-eral theory of relativity.

Sir Arthur S. Eddington, a Britishastronomer, observed that the amount of dif-fracted light arriving from the opposite side ofthe Sun during a solar eclipse curved by thegravitational field of the Sun exactly matchedthe values predicted by Einstein’s field equa-tions, demonstrating the validity of Einstein’stheory. This example illustrates how scientificrevolutions involve a shift from an old para-digm to a new one. But what does the creationof a new “inspired by life” paradigm mean?To understand this, we must first learn fromthe theory of Darwin, the father of evolution.

1.4 Darwin’s Theory of Evolution“The most incomprehensible thingabout the world is that it is at all com-prehensible.”


In his book, The Origin of Species pub-lished in 1859, Darwin writes the following[3]:

It is interesting to contemplate a tangledbank, clothed with many plants of many kinds,with birds singing on the bushes, with variousinsects flitting about, and with worms crawlingthrough the damp earth, and to reflect that theseelaborately constructed forms, so different fromeach other, and dependent on each other in socomplex a manner, have all been produced bylaws acting around us. These laws, taken in thelargest sense, being Growth with Reproduction;Inheritance which is almost implied by reproduc-

tion; Variability from the indirect and directaction of the conditions of life, and from use anddisuse: a Ratio of Increase so high as to lead to aStruggle for Life, and as a consequence to Natur-al Selection, entailing Divergence of Characterand the Extinction of less-improved forms. Thus,from the war of nature, from famine and death,the most exalted object which we are capable ofconceiving, namely, the production of the higheranimals, directly follows. There is grandeur inthis view of life, with its several powers, havingbeen originally breathed by the Creator into afew forms or into one; and that, whilst this planethas gone cycling on according to the fixed law ofgravity, from so simple a beginning endless formsmost beautiful and most wonderful have been,and are being evolved.

Richard Leakey, an evolutionary biologist,justifies Darwin’s status as the father of themodern theory of evolution as follows[4].First, Darwin meticulously and systematicallycategorized each and every piece of evidenceassociated with the problem of evolution. Inhis youth, Darwin boarded the surveying ves-sel HMS Beagle as a naturalist and spent fiveproductive years aboard the vessel along itsvarious journeys (1831–1836). Introducinghim to diverse geological and biological phe-nomena and giving him time to accumulateknowledge, and formulate ideas, this longvoyage was crucial to his future emergence asan eminent naturalist. As early as 1837, hebecame convinced that species were neitherpermanent nor fixed. During the periodbetween 1837 and 1859, he read widely, medi-tated deeply, and carried out carefullydesigned experiments. This extended period ofpreparation may explain why The Origin ofSpecies succeeds in covering such a widerange of topics and at such depth. Second,claims Leakey, Darwin was able to present amechanism that convincingly explained howchanges occur in species—namely, NaturalSelection. Darwin first conceived the idea ofnatural selection in 1838, inspired by An Essayon the Principle of Population by ThomasRobert Malthus, a parson and socio-economistof the early 19 th century. While Malthus was

7SAWAI Hidefumi

primarily interested in population, his essaypoints out that the number of offspring pro-duced by an organism is greater than the num-ber expected to live to reproductive age, iden-tifying this as a general principle in nature.

Based on Darwin’s theory of evolution,John Holland in the 1960s advanced the ideaof genetic algorithms (GA) as a computationalmodel[6]. Genetic algorithms and their evolu-tionary forms (e.g., genetic programming andevolutionary computation) are discussed ingreater detail in a later chapter.

1.5 The Phylogenetic Tree“The distinction between past, present,and future is only a stubbornly persis-tent illusion.”


In 1837, Darwin realized that the evolu-tionary genealogy between organisms couldbe more clearly represented using a tree dia-gram. (Figure 1 is a schematic drawing of aphylogenetic tree.)

Over 10 million rich and diverse speciespopulate the globe today, all descended from asingle primordial cell existing some four bil-lion years ago. Advances in fossil survey andDNA analysis techniques have made it possi-ble to put the time at which humans divergedfrom orangutans at some 13 million years ago;from gorillas at some 6.5 million years ago;and from chimpanzees at some five millionyears ago. Gradual accumulations of small dif-

ferences in genotype created by mutationseventually result in significant differences inphenotype, in due course leading to brains andintelligent organisms capable of adapting totheir environment. The term “intelligence” asused here will refer both to the adaptive func-tions of an organism that affect its survivaland the exquisite functions emergent from thebrain.

2 Information Processing Basedon Brain Function Modeling

2.1 Brain Structures and Their Functions

This section presents a summary of thestructure and functions of the brain. As shownin Fig. 2, the brain is divided into left andright hemispheres. The brain can also bedivided into the following units: the cerebrum,consisting of frontal, parietal, temporal, andoccipital lobes; the cerebellum; the brain stem;and the spinal cord. Each part has a modularstructure consistent with its function. Thebrain constitutes a system of immense com-plexity, composed of some 100 billion neurons(counting both the cerebrum and cerebellum),with each neuron connected to other neuronsby several thousand to tens of thousands ofsynapses. This complex organ is the productof evolution. Figures 3 and 4 present the struc-tures of a neuron and a neural network,respectively. As shown in Fig. 5, a synapse inessence is the space between two neurons

Fig.1 Schematic diagram of a phyloge-netic tree Fig.2 Brain structure


across which electric pulses traveling along anaxon are relayed to an adjacent dendrite viachemical transmitters.

A neural network is an example of aninformation processing model based on brain

function. Presented below are modeling anddesign methods for neural network architec-tures suitable for speech or image patternrecognition.

Special focus will be given to the featureextraction technique tailored to the temporalstructure of speech and the spatial structure(information structure) of images.

2.2 Basic Architecture of Neural Networks

Figure 6 is a model of the neuron proposedby McCulloch & Pitts in 1943. The input sig-nal xi (i = 1, 2, …, n) is input to the neuron,with the links representing synapses weightedby wi. The resulting internal potential u is thesum of these products, or ∑wixi. The sum isinput to the threshold function y = f(u), whichis y = 0 when the value of u is below a certainthreshold value of θ and y = 1 when u isgreater than the threshold. Thus, this neuronmodel adopts a weighted majority logic basedon threshold value. The threshold function y =f(u) is generalized into a differentiable sig-moid function y = 1/{1+exp(–x+θ)} andapplied to the learning rule by error back-propagation for the artificial neural network(ANN) shown in Fig. 7.

2.3 Architecture of Time-Delay NeuralNetworks

Figure 8 shows a unit used in a time-delayneural network (TDNN)[7]. The architectureshown here was proposed to process time-series data — for example, speech. To the leftis the input unit, which is linked to the hostoutput unit via synapse W, which can be anon-time-delay concatenation or a time-delayconcatenation with a delay of D1, D2, …, Dn.This architecture is suitable for processingspeech signal patterns that have temporalstructures. Weighted summation (∑) is per-formed on the input signals, and the resultingvalue is sent to the sigmoid function F(x) =1/{1+exp(–x)} and the result output.

Figure 9 shows the architecture of aTDNN designed to distinguish the voiced plo-sives /b, d, g/ in the Japanese language[7].

Fig.3 Neuron (Nerve cell)

Fig.4 Neural network

Fig.5 Synapse

9SAWAI Hidefumi

From bottom to top are the input layer, hiddenlayer 1, hidden layer 2, and output layer. Theinput layer consists of a total of 240 units, or15 and 16 units in each of the x- and y-axisdirections.

The x- and y-axis correspond to the time-

and frequency-axis, respectively. The time-frequency spectrum (sound spectrum) pro-duced by frequency analysis performed every10 ms is input to the input layer. This time-fre-quency spectrum is shifted by a time windowof 30 ms (3 units) and after being multipliedby the weighting factors of synapses havingtime-delay, it is linked to the host unit in hid-den layer 1. As in the input layer, the total of40 units in hidden layer 1, five units in the x-axis (time-axis) direction, and the eight unitsin the y-axis (frequency-axis) direction arelinked to hidden layer 2 with the time-delay.In hidden layer 2, each of the nine units in thex-axis direction is assigned to one of three cat-egories /b, d, g/, and concatenated to the out-put unit with the time-delay. This TDNN istrained by error-propagation. The results ofdiscrimination testing for input phonemes notused for training indicate that the TDNNachieves a speaker-dependent recognition rateof approximately 98 – 99 %, representing areduction in misrecognition rate to approxi-

Fig.6 Formal neuron in McCulloch & Pitts

Fig.7 Artificial neural network (ANN)

Fig.8 Unit of the time-delay neural net-work (TDNN)

Fig.9 Time-delay neural network (TDNN)architecture


mately 1/4 that of the conventional HMM(Hidden Markov Model) widely used forspeech recognition applications. (HMM isassociated with a recognition rate of91 – 97 %.) By expanding the phonemic cate-gory in a similar manner, we should be able tobuild a TDNN for all Japanese phonemegroups.

Figure 10 shows the modular architectureof a TDNN capable of discriminating the18 consonants of the Japanese language[8]. Asthe figure shows, the 18 consonants are divid-ed into six groups — the voiced plosives /b, d,g/, unvoiced plosives /p, t, k/, nasals /m, n, N/,

fricatives /s, sh, h, z/, affricates /ch, ts/, andliquids and semi-vowels /r, w, y/. A TDNNcapable of distinguishing between the sixphoneme groups is designed so that the resultsof discrimination processing within eachphoneme group and those from group discrim-ination processing can be linked in the outputlayer.

Figure 11 shows a TDNN system with aTDNN capable of distinguishing between theJapanese language vowels /a, i, u, e, o/ and aTDNN capable of distinguishing between thesix consonant groups and the vowel groupadded to the TDNN in Fig. 10, in addition to aspeaker-dependent recognition TDNNexpanded to operate as a speaker-independentrecognition network[9]. As the figure shows,this TDNN is a large-scale network with a 3Dstructure. Its scale enables speech recognitionof all Japanese consonants and vowels inspeaker-independent mode. Adding a discrim-ination unit (Q) for the presence/lack of voice(voiced/unvoiced) makes it possible to auto-matically recognize Japanese phonemes(called phoneme spotting) simply by scanningvocalized speech along the temporal direction.Readers are referred to reference[9] for moreinformation on the recognition performance ofthese networks.

Figure 12 outlines a different approach

Fig.10 Modular architecture of all conso-nant network

Fig.11 Large-scale TDNN architecture forspeaker-independent recognition

Fig.12 Large-scale TDNN architecturewith speaker-adaptive neural net-work

11SAWAI Hidefumi

that also results in a speaker-adaptive neuralnetwork[10]. The three lowermost layers areneural networks that perform speech spectramapping to adapt the speech of an unknownspeaker to that of a standard speaker used intraining. This approach — providing the map-ping to the TDNN in advance — may repre-sent a highly effective strategy for applying aTDNN trained to a standard speaker to recog-nize the speech of unknown speakers.

2.4 Expansion of Time-Delay NeuralNetwork[11]

Figure 13 shows an expanded architecturemodel of a new TDNN designed to absorbtemporal and frequency fluctuations in speech.Time and frequency windows are installed inthe input layer, and extractions of featurequantities (the feature quantity associated withtemporal fluctuations and those associatedwith frequency fluctuations) are integrated inhidden layer 1. The signal is ready for outputby the output layer after integration in hiddenlayer 2. The figure is an expanded TDNNdesigned to make precise distinctions within acategory consisting of six phonemes associat-

ed with high misrecognition rates: the voicedplosives /b, d, g/ and nasals /m, n, N/. Therecognition performance of this TDNN is dis-cussed in reference[11].

Figure 14 is a neural network with blockwindows inspired by the neocognitron, exten-sively investigated for applications in hand-written letter recognition. As with handwrittenletter recognition, it is necessary to absorbfluctuations in the absorption along the time(x-axis) and frequency (y-axis) directions inspeech pattern recognition. Thus, we can buildan architecture that promotes such absorptionby installing block-shaped windows in thelower layers so that the feature quantities canbe integrated in succession as connections aremade upwards.

2.5 Architecture for Rotation-InvariantPattern Recognition[12]

The architecture in Fig. 15 is an expandedNN having axial symmetry created by expand-ing the translation invariance of TDNN to

Fig.13 Frequency-time-shift-invariantTDNN architecture

Fig.14 Block-windowed neural network(BWNN) architecture


rotational invariance.The synapse weighting factors having par-

allel assignments from the bottom to upperlayers retain the original values acquiredthrough training. This architecture means thatif the synapse weighting factors can be learnedusing error back-propagation for every patternof the letter pattern class categories (A – Z)input at a given rotational position (0 deg., forexample), the neural network can correctlyrecognize the class category and rotation anglewhen an arbitrary class category is input atany angle. The figure shows a neural networkarchitecture capable of recognizing the 26 let-ters of the alphabet at rotation angle intervalsof 45 degrees. From bottom to top are theinput layer, hidden layer 1, hidden layer 2,output layer 2 (for rotation angle recognition),and output layer 1 (for class category recogni-tion). This neural network has an axially sym-metric structure. Proper execution of therecognition function requires careful align-ment of the center of the pattern of the image

fed to the input layer.

3 Information Processing Inspiredby Evolutionary Computation

An information processing model inspiredby biological evolution is a computationalalgorithm based on genetic algorithms andevolutionary computation. Table 2 is a compi-lation of the evolutionary computation algo-rithms introduced in this and subsequent sec-tions.

In 1859, Charles Darwin released hisbook, The Origin of Species, which dealt withthe genetics and evolution of organisms.Prompted by the book, John Holland advancedthe idea of genetic algorithms (GA) in the1960s. Genetic algorithms have been appliedto problems in numerous fields, includingfunctional optimization, combinatorial opti-mization, and parameter optimization inmachine design. In recent years, genetic algo-rithms have been integrated with other tech-niques such as Evolutionary Strategies (ES)by Rechenberg and Evolutionary Program-ming (EP) by L. Fogel to form a research dis-cipline known as Evolutionary Computation(EC).

3.1 Parameter-free Genetic Algo-rithms Inspired by Disparity Theoryof Evolution[13]

This section discusses a new algorithm,the parameter-free genetic algorithm (PfGA),which requires no initial setting of geneticparameters such as initial population size,crossover rate, or mutation rate.

The PfGA was inspired by and builds onthe disparity theory of evolution proposed byFurusawa et al.[20], which itself is based onmutations in the double strands of DNA(Fig. 16). According to disparity theory, whenthe double strands of DNA unwind and a copyof each is created, a difference emerges in therate of mutation between leading and laggingstrands. This is because the direction of repli-cation in the former is the same as the direc-tion of unwinding, while the direction of repli-

Fig.15 Axially symmetric neural networkarchitecture

13SAWAI Hidefumi

cation in the latter is in the opposite direction.While mutations are generally rare in the lead-ing strand (conservative), the lagging stranddisplays comparatively high mutation rates(innovative). Disparities in replication errorsaccumulate through crossovers and mutationsover generations, creating DNA diversitywithin a single population via DNA strandsexperiencing little or no mutation and strandswith accumulated mutations. The former typepromotes the stability of the population, whilethe latter promotes flexibility. In the case ofPfGA, the former corresponds to the opti-mized individual at a specific point in time,while the latter corresponds to the offspringproduced by crossover and mutation. Thus,under the disparity theory of evolution, themechanism by which diversity is retainedwhile maintaining a balance between exploita-tion (localized search) and exploration (globalsearch) can be understood as a balancebetween genetic stability and flexibility.

In PfGA, the population is regarded as aset of all possible solutions in the whole

search space. In this whole search space S, alocal sub-population S’ is set. Two individualsare selected from this sub-population S’ asparents. The parents are subjected to crossoverand mutation to generate a family (S”) of four,with two offspring. The fitness of the fourindividuals within this family is then evaluat-ed to select or eliminate individuals to evolvelocal population S’ and to execute a search forthe solution.

Below are the steps in the basic algorithmfor the PfGA (See Fig. 17).

1. An individual is randomly extractedfrom S and is regarded to be initial localpopulation S’.

2. An individual is randomly extractedfrom S and added to the local popula-tion S’.

3. Two individuals are randomly extractedfrom local population S’ for use as par-ent 1 (P1) and parent 2 (P2) in the multi-point crossover.

4. Of the two individuals generated by thecrossover, one is randomly selected andinverse mutation is applied at a randomnumber of points at random positions.

5. Selection and elimination is performedfor a total of four individuals (referredto as a family) consisting of the twogenerated offspring (C1 and C2) and thetwo parents (P1 and P2), by selectingeither one or three members of the fami-ly to be returned to local population S’,based on the calculated fitness.

6. If the local population size |S’| ≥ 2, thenreturn to Step 3; if |S’| = 1, then returnto Step 2 and repeat the cycle.

Table 2 Some examples of evolutionary computation algorithms

Fig.16 Hypothesis based on the disparitytheory of evolution


Multi-point crossover is used as thecrossover mode in PfGA. In multi-pointcrossover, both the number and positions ofcrossover points are determined randomly, andcrossover is executed between chromosomesof two different individuals. Mutations use theerroneous copy of chromosomes generatedduring the crossover. Thus, of the two off-spring produced, one is randomly selected anda partial inversion of the gene sequence car-ried out to create mutations at a random num-ber of points at random positions. Here, of thetwo offspring produced by the crossover, oneis left untouched by mutation to allow one off-spring to retain at least a portion of the par-ents’ traits. In this manner, the PfGA is imple-mented to execute genetic manipulation basedon random numbers and to minimize ad hocchoices.

For selection and elimination, to retain thediversity of the local population while main-taining a balance between global search(exploration) and local search (exploitation),the fitness of the four members of the family(S”) are compared and selections made in themanner presented in the following four casesas shown in the left plot of Fig. 18. Local pop-ulation S’ is evolved while dynamically main-taining a balance between global and local

search and concurrently changing the size ofthe local population S’ based on an implicitset of rules for switching between Cases 1 – 4,depending on the relative superiority of the fit-ness of family members. This featureimproved the search efficiency of PfGA overother GAs of fixed population size, since iteliminates the need to perform unproductivesearches. In addition, the best individualamong the four family members is alwaysreturned to the local population S’. Thus, thefamily may be said to be adopting an elite-pre-serving strategy: The algorithm guarantees theretention of the best individual at a given pointin time while simultaneously performing anactive search over a very wide (neighborhood)space. If a better individual than the currentlybest individual is generated, the center of thesearch transfers to that offspring; if not, thecurrent best individual is retained. This avoidsfitness degradations during the course of evo-lution.

3.2 Parallel Distributed ProcessingTechniques for Parameter-freeGenetic Algorithms[14]

This section describes techniques for par-allel distributed processing related to the para-meter-free genetic algorithm (PfGA) inspiredby ecosystems. In general, the main goal ofparallel processing in any processing, includ-ing GAs, is to increase processing speed.However, we can dramatically enhance the

Fig.17 Flowchart of parameter-free GA(PfGA)

Fig.18 Population and selection rule inPfGA

15SAWAI Hidefumi

efficiency of search problem processing basedon a GA by introducing interactions betweenindividuals by migration, rather than simplydividing up the task. In the case of a coarse-grained parallel GA, the local population istreated as the processing unit, and individualsare migrated between local populations atappropriate frequencies. In a fine-grained GA,the neighborhood of a given individual istreated as the processing unit, and overlaps areset among neighbors. The former is frequentlyreferred to as the island model, wherein a sin-gle local population constitutes the deme of asingle species. This paper uses this as themodel.

Two types of parallel processing architec-tures are used: the uniformly-distributed typeand the master-slave type. The uniformly-dis-tributed type corresponds to a situation inwhich all local populations have the same roleand local population monitoring functions areabsent.

On the other hand, in the master-slavetype, a master local population is equippedwith a function for monitoring the processingof other local populations, called slaves.Fig. 19 shows the uniformly-distributed-typePfGA architecture. From the whole searchspace S, an N number of local populationsS’i (i = 1, …, N) is derived; in each local popu-lation S’i , there exists a family (S”i) that per-forms PfGA crossovers, mutations, and selec-tion. Migration of individuals may occurbetween any of the local populations.

Figure 20 shows the master-slave architec-ture. S’0 is the master local population, whileS’i (i = 1, …, N) are the slave local popula-tions. The master S’0 consistently (or at con-stant intervals) seeks to identify the best indi-vidual in all slave populations.

Several migration strategies may come tomind, but here we adopt the following two. Inthe first, an individual in a given local popula-tion is copied and distributed to other localpopulations only when a good individualemerges. This is called the direct migrationtype. The disadvantage of this method is thatthe same individual is retained by other localpopulations after migration, threatening sys-tem diversity. We adopt the second strategy, inwhich good individuals are gathered frommultiple local populations and two individualsare arbitrarily selected to be the new parents.They bear two offspring (by crossover andmutation), and one to three members of thefamily are distributed according to the selec-tion rules in PfGA to arbitrarily selected localpopulations. Since this migration methodimplements meta-level PfGA operations fromthe perspective of the local population, it iscalled the hierarchical migration type(Fig. 21).

We performed parallel processing for thefour different combinations of parallel archi-tecture (uniformly-distributed/master-slave)and migration type (direct/hierarchical) toinvestigate the effects of migration.

An evaluation of search performance

Fig.19 Uniformly-distributed type archi-tecture Fig.20 Master-slave type architecture


showed that increasing the number of localpopulations reduced the number of evaluationsrequired before success by a ratio of 1/N.Among the four types of architecture/migra-tion method examined, search performance,from high to low, fell into the followingsequence: UD1, MS1, MS2, UD2. We con-firmed that increasing the number of localpopulations increases the chance of successthrough the effects of migration relative to ser-ial processing.

3.3 Evolutionary Computation Basedon Gene Duplication[15]This section discusses the gene-duplicat-

ing GA (GDGA) inspired by the theory ofgene-duplication proposed by Susumu Ohnoin the 1970s. The theory of gene-duplicationclaims that the replication and reuse of genefragments in the evolution of all organisms,from viruses and plants to animals, fuels adrive toward life forms of ever-growingsophistication.

Ohno distills this phenomenon into a suc-cinct formula: “a single creation and one hun-dred plagiarisms.”[16] Gene duplication isassumed to occur by unequal crossoverbetween chromatids on a single chromosome,unequal crossover between homologous chro-mosomes during the meiotic process, and par-tial repetitive duplication of DNA. Inspired bythis gene duplication mechanism, we haveproposed four gene duplication models: gene

concatenation, gene-prolonging, gene cou-pling, and extended gene coupling.

This computational method is based on adivide-and-conquer GA in which a givenproblem is broken down into sub-problems,which are then combined to obtain the solu-tion for the original problem. Each individualconcatenates the partial solution each hadaccumulated up to that point in time, then theindividual migrates between the local popula-tions. This strategy makes it possible to obtainthe solution more efficiently and quickly.

Gene duplication, a powerful tool in solv-ing multi-dimensional functional optimizationproblems, is a genetic operator applicable toindividuals of different gene lengths. It isapplied by first coding variables to genes foreach subdimension, then setting the fitnessfunction for each subspace and running theGA to obtain the (quasi) optimum solution. Byconcatenating individuals owning the genecorresponding to this (quasi) optimum solution,we can solve an optimization problem in higherdimensions. This algorithm is implemented byhaving individuals with differing gene lengthsmigrate between local populations. Overall, thealgorithm performs crossover, mutation, andselection within a local population; betweenseparate local populations, it performs dupli-cation and migration, in that sequence.

In a simulation evaluating the search per-formance of the four types, we used a func-tional optimization benchmark problem tocompare success rates, probabilities, and con-vergence performance in obtaining the opti-mum solution. We found that increasing thenumber of migrating individuals increasespopulation diversity, thereby confirmingimprovements in convergence performanceand the effectiveness of this computationmethod.

3.4 Evolutionary Computation Inspiredby Sexual Selection[17]

The theory of sexual selection seeks inpart to explain the extensive differencesbetween the phenotype and behavior of thetwo sexes in certain sexually-reproducing

Fig.21 Migration strategy selectionmethod: UD1 (top left), UD2 (topright), MS1 (bottom left), and MS2(bottom right)

17SAWAI Hidefumi

organisms. Certain well-known examplesinclude competition between males (generat-ing antlers) and female preferences (generat-ing peacock plumage).[18][19] Numeroushypotheses based on female preference seek toexplain the evolution of traits that appear dis-advantageous from the perspective of naturalselection.

Two famous hypotheses are the runawayhypothesis and the excellent gene hypothesis.The former assumes that female preferenceswithin a population are always biased and thatmale traits are always variable due to randommutations. In such cases, males with the pre-ferred traits are more likely to father largenumbers of offspring, regardless of fitness interms of natural selection, thereby conferringan indirect advantage to the gene manifestingthat trait and resulting in the rise of the maletrait preferred by females. The latter hypothe-sis claims that the male shows off the qualityof his genes even if doing so comes at somecost, with the result that the trait and thefemale preference for that trait grow morecommon. However, the actual processes atwork in sexual selection remain unclear.

The genetic assignment of sex to individu-als has led to a state in which an individual ofone of the sexes effectively observes andchooses the phenotype of the other, and wewill focus on the effect of such asymmetricroles of males and females on the process ofevolution.

We will also examine the role of mutation(the simplest transition rule) as a genetic oper-ator that drives organisms toward the directionof evolution based on the fitness landscape. Amodel is assumed in which a mutation rate iscoded in the gene as a parameter for mutationand in which fitness varies.

We will focus in particular on the interac-tion between sexual selection and mutation.

Working from these perspectives, weintroduce an evolutionary computationalmodel in which mutation rates are encoded inthe gene and also account for sex and sexualselection. Based on this model, we investigatehow mutation rates become self-adaptive

Fig.22 Gene duplication in a gene-con-catenating model

Fig.23 Gene duplication in a gene-pro-longing model

Fig.24 Gene duplication in a gene-cou-pling model

Fig.25 Gene duplication in an extendedgene-coupling model


within a population and how the direction ofevolution is determined in phenotypic space.[Constructing a computational model]

How does an asymmetric relationshipbetween the sexes whereby one sex observesand selects the phenotype of the other affectmutation rates? We propose a model based onsexual reproduction with its own mutation rateencoded into the genes. For the sake of conve-nience, the observing and observed sex,respectively, are regarded as female and male.

A real-valued genetic algorithm (real-val-ued GA) is used as the evolutionary computa-tional model. The methods of genetic recom-bination are chromosomal exchange betweenindividuals and isotropic mutation of eachgene.

Sexual selection will focus on relativephenotypic value. (Example: a strong prefer-ence for taller individuals or individuals of acertain stronger coloring [e.g., bluer].) In suchmodeling schemes, the direction of the transi-tion of next-generation males in phenotypicspace is determined by the direction of femalepreferences. [Individual phenotype]

Each individual is assigned a sex and withtwo broad phenotypic categories, trait andpreference. Traits are expressed in both sexesand determine the individual’s fitness. In con-trast, preferences are expressed only byfemales and act as a mechanism by whichmales are assessed and selected. Preferencesdo not affect natural selection. Traits and pref-erences are represented in phenotypic spaceby trait vector t = (xt1, xt2, …, xtn) and prefer-ence vector p = (xp1, xp2, …, xpn) in n-dimen-sion Euclidean space.

Sexually-reproducing organisms arediploid in nature, but here, given the emphasison interactions due to preference rather thanthe mode of reproduction itself, the modelassumes that the genotype of each individualis haploid for the sake of simplicity, and anindividual will have three kinds of chromo-somes: sex chromosome, preference chromo-some, and a trait chromosome (Fig. 26).

The sex gene is coded in a single bit and

the other genes are encoded as real values.However, the preference gene in the male isconsidered not expressed, creating a buffer formale preference and sustaining diversity inpreference. Furthermore, we place the geneencoding for the mutation rate (σ) for the traitand preference genes on the sex chromosome,making the expressed mutation rate dependenton sex.[Natural selection]

A known hummingbird species displayssexual dimorphism in beak morphology, withthe males and females of the species feedingon flowers of different shapes. This constituteshabitat segregation in the form of resourcepartitioning. The male and female members ofthis species can be considered to have fol-lowed different paths in natural selection.

Natural selection is posited to operate sep-arately on the sexes. This renders a constantsex ratio while allowing the sexes to generatedifferent traits (sex difference).[Sexual selection]

Under sexual selection, a female chooses amale to form a pair, and the offspring pro-duced are one male and one female to main-tain the constant sex ratio. All females willalways be part of a pair at least once, whilemales are allowed to pair only when selectedby a female as a preferred male. In short, thispopulation is polygamous. (1) The female i observes M numbers of males

at random as potential mates. The averagetrait vector t 0i of the observed male popu-lation is calculated, and the relative traitvector t ’ij = t j – t 0i of male j (j = 1, 2,

Fig.26 Proposed model of sexual selec-tion

19SAWAI Hidefumi

…, M) is used as the selection target.(2) The difference in direction θij between t ’ij

and the female preference vector p i is cal-culated. The strength of preference isdefined as given by cos(θij). The malehaving a trait vector in the direction closerto the direction of the female preferencevector will be strongly favored.

(3) The most preferred male is selected deter-ministically.

By using the relative trait vector as the selec-tion target, we can search for the directiontoward which the male population should shiftin phenotypic space—or the direction of evo-lution—based on the direction of the femalepreference vector.

In sexual selection, males preferred bymore females (i.e., attractive males) gain theadvantage, and selection works directly on themale, manifesting as the number of offspringin the next generation. While females in thismodel are exempt from the direct operation ofsexual selection, through genetic exchangewith preferred males, the offspring of femaleshaving a genetic preference for males withadvantages in terms of natural and sexualselection is likely to have the advantage in thenext generation. This means sexual selectionworks indirectly to the advantage of femaleshaving genes for such preferences.[Genetic Manipulation]

Two types of chromosomal exchange andmutation are used as methods of geneticmanipulation. With respect to interactionsbetween mutation rate and sexual selection,parent chromosomes are exchanged at a con-stant probability when parent genes are copiedto produce offspring.

Mutations are accomplished by impartingperturbations that follow the normal distribu-tion function N(0, σ) on the offspring traitand preference genes xi

t, xip ( i = 1, 2,…, n),

respectively.Here, the standard deviation σ corre-

sponds to the mutation rate, which is variedadaptively by gene encoding.

σ = σ + δ δ～N(0, σ0)

Note here that the standard deviation σ0 ofthe normal distribution function of the pertur-bation imparted to σ is constant.[Steps in simulation](1) Population is initialized at a sex ratio of

1:1.(2) The following procedure is repeated until

the termination condition is satisfied:(a) Natural selection(b) Sexual selection(c) Genetic manipulation

[Problem]Traits and preferences, respectively, are

represented using two-dimensional vectorst = (xt, yt) and p = (xp, yp) in examining the fol-lowing functional maximization problem.

Max f (t ) = xt – 2 sin(πxt) – 0.001yt2 exp(xt)

The initial population is positioned nearthe point of origin, while each individual’s fit-ness is determined by trait alone. In the aboveequation, local optimum solutions may befound on the line yt = 0 with period 2. Theproblem, then, is to find a way to balance thesearch for the local optimum solution and theescape from the local optimum solution.

The effects of the 3rd term become largewith increasing xt as the search progresses, anda slight transition of yt from 0 results in a pre-cipitous decline in fitness, rendering thesearch more difficult. Although there is noupper limit to the function, GA restricted toisotropic mutation alone places a practical

Fig.27 Validation of the proposed modelwith an illustrative problem


limit on the search. This extremely simplemodel is an example of how adaptive acquisi-tion of a direction advantageous to evolution(positive xt direction) concurrent with a searchfor the optimum solution makes it possible tosearch as efficiently as possible.

Natural selection is executed by perform-ing a roulette-wheel selection based on g,where g = exp(af (t )) (α: scaling rate) whenf (t) is fitness. This is selected to make thesearch progress more smoothly when theexponential term in the above equation startsto have a strong effect on the latter part of thesearch and when the average fitness of thepopulation has decreased exponentially.

We use preference vector p = (xp, yp) as theunit vector to normalize the results after eachmutation. This means that the preferenceselection of a male by a female is based solelyon the direction of the trait. [Results of experiment]

As we can see from Fig. 28, the search inthe proposed method progressed the mostamong the three types of strategies (randommating, SGA, proposed method), indicatingthe effects of sexual selection on the search.We see no apparent differences in mutationrates between the sexes for the results of ran-dom mating. Under the proposed method, asthe search proceeded, the male mutation ratesurpassed the mutation rate for females, con-firming that female preference triggersunequal mutation rates.[Search process]

Under the present method, when thesearch process becomes trapped in a localoptimum solution, a runaway situationbetween the male trait and female preferenceresults that intermittently drives explosiveevolution out of the equilibrium state. As thesearch progresses, differences appear in aver-age mutation rates between male and femalepopulations and a division of roles arises. Thesex exercising choice (female) carries out aconservative search with low mutation ratesand the chosen sex (male) carries out an inno-vative search. This results in only the malepopulation performing the search when escap-

ing from the local optimum solution, whilefemales dedicate themselves to maintainingpresent conditions and standing by to move onto better solutions through chromosomalexchange only after they have been found bythe male population. In many sexually-repro-ducing organisms, the production processesdiffer for reproductive cells between the sexes,and sperm cells have higher mutation ratesthan ova. The similarity between the charac-teristics of organisms and the proposedmethod is quite interesting. The advantages ofbalancing an innovative and conservativesearch during the search process through theacquisition of various mutation rates havebeen discussed in relation to the Neo-Darwin-ian algorithm by Wada et al.[20]. Since theerror frequency varies between the two strandsin DNA, they proposed a model in which themutation rate varies within a single individual,thereby permitting a wide range of searches.This realizes a broad range of mutation rateswithin the population, allowing them to per-form an extensive search in problems involv-ing a high level of risk while maintaining pre-sent conditions.

4 Information Processing Basedon the Modeling of Cells in EarlyStages of Evolution

4.1 Chemical Genetic Algorithm(CGA)[21]

The mechanism of cell metabolismemerged over an astoundingly lengthy evolu-tionary process. Modeling this process shouldmake it possible to search efficiently for an

Fig.28 Experimental results

21SAWAI Hidefumi

optimum solution by techniques totally differ-ent from conventional methods. This sectiondiscusses the chemical genetic algorithm(CGA), used for solving difficult problems bydynamically converting them into simplerproblems—in short, by dynamically changingmapping from the genotype to the phenotype,as inspired by the mechanism of cell metabo-lism in the early stages of the evolutionaryprocess. The section will also discuss chemi-cal genetic programming (CGP), in which thissolution method is applied to the evolution ofprogramming, and introduce the problem ofsymbolic regression in artificial intelligenceand describe the results of its application tothe acquisition of multi-agent behavioral strat-egy.[Mechanisms of cell metabolism generated inthe early stages of the evolutionary process]

In the early stages of cell evolution, cellsare believed to have acquired their presentmetabolic processes by dynamically changingthe mapping behavior from genotype to phe-notype mapping (Fig. 29). Fig. 30 presents amodel based on this mechanism.[CGA generation cycle] (See Fig. 31)

The steps in the CGA generation cycle aregiven below:

1. Initialization: First, we prepare a num-ber, N, of cells having the structure pre-sented in Fig. 30. In the initial state, nocell possesses aminoacyl tRNA (aa-tRNA), tRNA, or outputs amino acids.However, they do have random DNAstrands and amino values.

2. Chemical reaction: The following 4-stepreaction takes place in all cells: tran-scription, tRNA-amino acid reaction,translation into internal amino acid, andtranslation into output amino acid. Inthe several generations of the earlystage, this reaction produces new tRNAand aa-tRNA, and their sizes grow.Within the next few generations, weexceed the size of the molecular poolsize.

3. Selection: The fitness of the cell is cal-culated based on the output amino acid,

and cells marking high fitness areselected by roulette-wheel selection.The selected cells are regenerated, andthe complete internal information(DNA, 3 molecular pools) of each cellis copied to the daughter cell.

4. DNA mutation: As in normal GA, pointmutations of genes are performed.

Fig.29 Biochemical reactions for translat-ing genetic information in a cell

Fig.30 Cell structure used in CGA

Fig.31 Entire algorithm of CGA


5. DNA crossover and molecular exchangebetween cells: Gene crossovers occur asin normal GA, and half the moleculesare exchanged between two cells.

6. Calculation of fitness of the cell popula-tion. If the termination conditions aresatisfied, the computation is complete.If not, return to Step 2.

[GA evolvability] (See Fig. 32)By converting the “ragged fitness land-

scape” as seen in the genotype space presentedon the left side of Fig. 32 (a) into a smoothedlandscape on the right side (b), we canimprove evolvability.

Three types of deception problems(TypesⅠ, Ⅱ, and Ⅲ) and 2 benchmark func-tions were used to validate the search perfor-mance of CGA. SGA (simple GA) and PfGA(Parameter-free GA) are used for compar-isons. In the simple TypeⅠ, F(x) assumes themaximum value (optimum value) of 1 whenxk = 1 in all dimensions of k (k = 1, …, n).TypeⅡ is an intermediate complex type, andF(x) assumes the maximum value of 1 onlywhen, in each dimension k, fk(x) randomlytakes a maximum at xk = 0 or xk = 1. In TypeⅢ,F(x) assumes the maximum value of 1 onlywhen fk(x) assumes a maximum of 1 at xk=αk

(αk is a uniformly random number between 0and 1) in each dimension of k. As can be seenfrom Fig. 33, the ratio of the probability off(x) assuming a maximum (optimum value) of1 in each dimension to the probability of tak-ing the localized optimum value of 0.8 is 1: 4.Thus, the probability of f(x) taking optimum

values at all dimensions (k = 1, …, n) is (1/5)n.TypesⅠ and Ⅱ constitute special cases ofTypeⅢ (Fig. 33).[Results of analysis]

Figures 34 and 35 present the evolution ofthe function F(x) of CGA and SGA, respec-

Fig.32 Evolvability in C(GA)

Fig.33 Complex deceptive problem(TypeⅢ)

Fig.34 Evolution of CGA for deceptiveproblem (TypeⅢ)

Fig.35 Evolution of SGA for deceptiveproblem (TypeⅢ)

23SAWAI Hidefumi

tively. The dispersion of the function values islarge for CGA. In contrast, the dispersion issmall for SGA. The optimum value of CGA(CGA best) is 0.8 or higher and attains the sta-tus of the optimum solution. On the otherhand, the optimum value of SGA fails to sur-pass 0.8 and is trapped in a local optimumsolution.

Figures 36 and 37 present the time seriesof amino value histograms for CGA and SGA,respectively. The results are for a 5-dimen-sional Type-Ⅲ deception problem. We see thatfor all αk assuming maximum values at eachdimension of k, the amino value for CGAexceeds a certain value. In contrast, for SGA,the amino value exceeds a certain value in cer-tain dimensions but not in others. This is a

typical example of a search that has fallen intoa local optimum solution. [Performance comparison]

Tables 3 and 4 present comparisons of per-formance evaluation results for SGA, CGA,and PfGA.

Table 3 gives the results for deceptionproblems. A comparison of the success ratesof CGA and SGA shows that the performanceof CGA far outpaces SGA. Furthermore,PfGA is 100 % successful for all types ofdeception problems. Table 4 gives the resultsfor benchmark problems (Shekel’s foxholeproblem, Langerman function), and we seethat CGA gives success rates comparable toPfGA.

Figure 38 shows the changes in basin sizein the case of CGA. The figure shows a sud-den increase in basin size at a certain point inthe evolution (100 generations), considered toreflect the achievement of punctuated equilib-rium associated with the transition stage in theevolution, mapping from genotype (binaryvalue) to phenotype (function value).

Fig.36 Time series of amino value his-togram for CGA

Fig.37 Time series of amino value his-togram for SGA

Table 3 Success rate for deceptive prob-lems

Table 4 Success rate for benchmarkproblems


Figure 39 presents the values of the codon-amino acid translation table. Under the initialconditions of evolution, variations in the val-ues in the table appear to be large changes inamino value per single bit of change in thecodon (lower right). But as evolution pro-gresses, the amino value changes only gradu-ally relative to the change in codon value. Thiscorresponds to the evolution (transition) of thelandscape represented by the left plot ofFig. 32 into the smooth landscape representedby the right plot. The evolvability of CGA hasgreatly increased, indicating that the algorithmgenerates a solution method for difficult prob-lems in an evolutionary manner, while auto-matically (in evolutionary fashion) convertingdifficult problems into easier problems.The method for improving mapping tech-niques from genotype to phenotype throughthe evolutionary process is a highly generaliz-able optimization technique. The following

section discusses a method for expandingCGA to genetic programming (CGP).

4.2 Chemical Genetic Programming(CGP)[22]

Figure 40 shows how CGA is expanded toCGP. A comparison of Fig. 31 and Fig. 40shows how the genes (DNA sequence) inCGA are converted into a combination of therewriting rule numbers and the left sides of therewriting rules. DNA is translated into proteinsynthesized by concatenating amino acids,after which fitness is calculated. At this point,the other portions of the DNA have been tran-scribed into tRNA or translated into aminoacids. Aminoacyl tRNA, produced by theirreaction, acts as a catalyst in this translationprocess. Modeling these metabolic processesinside the cell results in the evolutionary gen-eration of the rewriting rule itself, and gener-ates rules completely different from those inthe initial state to make it possible to acquirerules with higher fitness scores.[Example of application 1] (Symbolic regres-sion problems)

Figure 41 compares the fitness evolutioncurve in CGP and conventional GE (grammat-ical evolution). We see that evolution proceedsfaster in CGP than GE, resulting in a goodsolution after 140 generations. Furthermore,even though the best solution appears faster,the average fitness of the population consis-tently remains below the best value, indicatingthat population diversity is sustained.

Figure 42 compares the solution generatedby CGP and GE. For the target function2x6 + 3x4 + 4x2+100, CGP gives 2x6+ 501,

Fig.38 Increase in basin size for CGA

Fig.39 Final translation table in binary-to-real value mapping for CGA Fig.40 CGP algorithm

25SAWAI Hidefumi

while GE gives 1.9 x6. Since the normalizedfitness values are 0.95 and 0.81 for CGP andGE, respectively, we may conclude that CGPis superior.[Example of application 2] (Behavioral strate-gy of agents)[23]

Figure 43 applies CGP to a multi (2)-agentproblem, the game of “tag.” Two agents useCGP to generate in evolutionary fashion abehavioral strategy for catching the other asquickly as possible, or to evade the other foras long as possible.

Table 5 is a list of basic functions used inCGP.

Figure 44 shows the two agents in motion.S denotes the starting point. The objects num-bered 1 and 2 in the center are obstacles. Wesee how the two agents (pursuer and evader)skillfully avoid the obstacle in the chase.Table 6 presents the behavioral strategies gen-erated by the agents. Although the details arenot given here, we can appreciate how bothagents acquire various behavioral strategiesthrough evolutionary generation.

Fig.41 Evolution curve of fitness

Fig.42 Generated function CGP vs. GE

Fig.43 The game of tag

Fig.44 Results of agent behavior

Table 5 List of basic functions

Table 6 Generated strategies


5 Research Trends in ComplexNetworks

“Man is, at one and the same time, asolitary being and a social being.”


The information and communication sys-tems permeating today’s society grow insophistication and complexity seemingly dayby day, while assuming greater roles in ourlives at the same apparent pace. But the sameinformation and communication systems havebeen shown to be fragile under conditionssuch as accidents and disasters. Developingtechnologies that will overcome such fragilityhas become essential. To resolve these prob-lems, by focusing on network dynamics, weexamined basic technologies for securing self-organized functional network structures thatdo not rely on an infrastructure and remainoperational even in dynamically changingenvironments. Our goal is to propose a proto-model for a next-generation information andcommunications system that is highly reliableand has high affinity for human needs andmodes of interaction. Here, we examinedresearch trends in complex network sciencesand statistical physics, with a special focus onnetworks with self-restoring functions andrequiring only localized information for adap-tive evolution, and on the design principles ofself-organizing networks that feature nodedeployment and load distribution functionsaccording to population distribution and eco-nomic activity levels and network traffic man-agement functions designed to resolve trafficcongestion. In addition, we also examined thespontaneous social network structures createdby relationships between real people and con-sidered a network society desirable in the nearfuture.

5.1 Keywords• Small-world phenomenon (Fig. 45): a net-

work phenomenon emerging from therewiring of a few edges in a regular network;one in which the average path length(described later) is significantly reduced.

This is known in common parlance as “six-degrees of separation.” Observations of thephenomenon date from experiments per-formed in the U.S. by the social psychologistStanley Milgram. From left to right inFig. 45 are a regular network, a small-worldnetwork, and a random network.ρ representsnetwork rewiring probability; higherρ val-ues indicate higher randomness. The figureshows how randomness increases from regu-lar networks to small-world networks andfinally to random networks.

• Scale-free network (Fig. 46): a networkwhose relationship between the degree (ki) ofa node i in the network and degree distributionP(ki) follows the power law P(ki) ∝ ki –γ.Figure 46 shows the Internet in a scale-freecondition, with relatively small numbers ofhubs having nodes with large degrees, andwith numerous nodes having small degrees.The upper right-hand panel in the figureshows the degree and degree distribution,displaying what is known as the long-tailphenomenon (resembling the long tail of a

Fig.45 Small-world network [26]

Fig.46 Scale-free network (for Internet)

27SAWAI Hidefumi

dinosaur). The lower right-hand panel showsthe same data as the upper right-hand panel,but in a log-log plot. The inclination of thelinear line here is equal to –γ. Various scale-free networks in nature and society havetheir own unique values.

• Pareto’s law: A law proposed by the Italianeconomist Vilfredo Pareto, popularly knownas the “80:20 rule.” It states that the signifi-cant 20 % of the element dominates 80 % ofthe whole. It is deeply associated with therelationship between degree ki and degreedistribution P(ki) in the above scale-free net-work.

• Robustness: Adaptability and/or fault toler-ance of life or systems to environmentalchange.

• Fragility, vulnerability: The opposite ofrobustness; the fragility of life or systems(weakness) in the face of environmentalchange.

In Persistent Life by Hiroaki Kitano andKaoru Takeuchi (Diamond Inc., 2007), thetrade-off between robustness and fragility isexplained by taking as examples the 2003New York City blackout, the beef bowl busi-ness strategy of Yoshinoya, and diseases suchas diabetes and cancer.[40]

5.2 Network Analysis Indicators• Network G: Set G = {V, E} consisting of the

set of nodes V = {v1, v2, …, vn} and set ofedges E = {e1, e2, …, em}.

• Degree: Degree ki of node vi refers to thenumber of edges branching out of node vi.

• Average path length/average distancebetween nodes L(ρ): For two arbitrarily cho-sen points vi and vj, the average of the mini-mum number of edges that must be crossedto reach each other.

• Clustering coefficient C(ρ): An index thattakes a value between 0 and 1, which quanti-fies the degree of concentration of edgesconstituting network G.

• Network centrality: The network propertythat plays the central role. When nodes hav-ing higher degrees are more central, the net-work is assumed to have “degree centrality.”

The nodes clustered in the highest concentra-tion along the shortest path between twonode pairs are described as having “between-ness centrality.” The index was proposed bythe sociologist Linton Freeman as an indexfor social network analysis.

• Network density: The higher the number ofedges between nodes, the higher the networkdensity.

Figure 47 is a graph showing the relation-ship between the clustering coefficient C(ρ)and average path length L(ρ) (where ρ is thenetwork rewiring rate). For small values of ρ,both the clustering coefficient C(ρ) and theaverage path length L(ρ) are large. But as ρincreases, they decrease.

At intermediate values of ρ (around 0.01),C(ρ) is large while L(ρ) becomes small, atwhich point the small-world phenomenonemerges.

Fig.47 Clustering coefficient C(ρ) andaverage path length L(ρ)

Fig.48 Robust network architecture resis-tant to both failure and attack


[A network architecture resistant to both ran-dom failure and targeted attack]

Although the question, “Do such networksactually exist?” comes to mind, according to“What Kind of Strong Wiring Design Wouldbe Resistant to Both Failure and Attack?” in“Semi-Special Edition: Spread of ComplexNetwork Science” by Toshihiro Tanizawa(pp. 282-289, Information Processing,March 2008 issue, No. 3, Vol. 49), theanswer is a bimodal distribution networkarchitecture in which only two types ofnodes are present: a small number of large-degree (k2) hubs and large numbers of small-degree (k1, where k1≪ k2) nodes.

5.3 General Trends in Research onNetwork Science

• In Network Science by the U.S. NationalResearch Council (http://www.nap.edu/cata-log/11516.html), the research topics listed inTable 7 are reported to be priority areas.

Based on this table, we find that modelingthe simulation of very large networks is con-sidered critical in the development of real-world tool sets and that human performanceenhancement is the goal of studies of swarm-ing behavior in the development of self-orga-nizing systems and studies on metabolic andgene expression networks. The selection ofbiological mechanisms and human swarmbehavior as research areas for achieving theseresearch goals is noteworthy.

5.4 Research Trends in Social Net-work-Associated Fields[41]-[46]

[Organization network strategy]• According to Global Neighborhoods —

Strategies of Successful Organizational Net-works by Toshio Nishiguchi, (NTT Publish-ing Co., Ltd. 2008), a strong and effectivehuman network may be constructed by com-bining long-distance relationships (= use ofsmall-world by distant but dear acquain-tances) and neighborhood socializing(= rewiring of human networks). Examplesgiven in the book include the robust humannetwork created overseas by former residents

of Wenzhou, China, and the miraculousrecovery of Aisin Seiki Co., Ltd., an affiliatesubsidiary of Toyota, after a fire (self-orga-nizing human network when faced withemergency), as well as the examples ofdefense procurement by the British Ministryof Defense and by Japan. The book presentsactual examples of several interesting net-work topologies. The book concludes bysummarizing such networks as follows: Thesecret to successfully managing a complexsocial network that far surpasses human cog-nitive limits is to establish a neighborhoodsocial network and to weave into it long-dis-tance relationships by some appropriaterewiring at the edges to transform the entiresystem into a small-world, and to take thebest parts of both. Supporting this network isthe social capital that forms the foundationof the relationship of mutual trust and easewith which structural rewiring can be per-formed.

• The Science of Building a Network of HumanConnections — Probing the Power Hidden inHuman-Human Relationships by Yuki Yasuda(Nikkei Inc., 2004)

“Distant people” are more beneficial. Thequality of the links is more important than thenumber. We turn more often to those we sel-dom see for support than frequent friends.Using The Strength of Weak Ties by Mark Gra-novetter and the “small world theory” by Dun-can J. Watts as the basis of her arguments, shereveals the most efficient method for con-structing human connections by networkanalysis.• Network Analysis — What Determines Human

Actions by Yuki Yasuda (Shinyo-sha, 1997)Covering human relationships from lovers

to organization and international networks,Yasuda searches for the common pattern inhuman-human and human-society relation-ships, then explains how network analysistechniques can be used to explore factors thatdetermine human actions.[SNS field]• The current status of social network services

such as Mixi, etc.

29SAWAI Hidefumi

Typical examples of such networksinclude MySpace in the U.S. and Mixi inJapan. These networks have enjoyed wide-spread success by providing services that letusers create blogs and make connections tofriends and other community members.[Information search]• Google’s business strategy

Based on the hypothesis that “quality sitesare sites supported by many other sites”—based on how documents on the WWW arelinked together—Google has successfullyapplied a business strategy that presents thesearch results for a search term using an algo-rithm called PageRank™. [Distribution field]• Amazon’s business strategy

Amazon has applied the long-tail phenom-

enon (power-law = scale-free property) inbook marketing, creating a database that letscustomers search for and purchase on theInternet, books ranking low on the sales list,and has succeeded in making enormous profitsoff such books, going against the conventionaltrend in which best-sellers and long-sellerswere the primary source of revenue for book-sellers.

5.5 Fields of ApplicationOne field of application of Complex Net-

work Science is “congestion studies.” Conges-tion Studies by Katsuhiro Nishinari (ShinchoSensho, 2006) presents a unified discussion ofthe phenomenon of congestion in informationtransferred over the Internet (packets) and oftraffic caused by human and automobile trans-

Table 7 Areas of research in network science, with categories created by the U.S. NationalResearch Council

(Cited from: page 5 of http://www.nap.edu/catalog/11516.html)


portation from the perspective of networks. Heintroduces technology for relieving trafficcongestion while explaining how congestioncan be actively directed to benefit humans (forexample, by preventing the spread of wildfiresor viral infections).

5.6 Conclusions“Imagination is more important thanknowledge. Knowledge is limited.Imagination encircles the world.”


We have witnessed dramatic progress inresearch that analyzes various natural andsocial phenomena from the perspective of net-works. Additionally, Internet businesses haveemerged that skillfully apply the characteris-tics of the small world phenomenon and scale-free networks.

We see that universal network principlesare in play at various hierarchical levels inboth the natural world and human world (soci-ety). The field of complex network sciencehas advanced rapidly from the end of the 20 thcentury to the beginning of the 21 st century,and there are great expectations for its applica-tion to a wide range of academic and businessfields. Our hope is that this field will enlightenall those living in the 21 st century in the cru-cial field of knowledge called network litera-

cy. One of the most important themes for thefuture is to apply this knowledge energeticallynot only in building future ICT infrastructures,such as new-generation network architectures,but in designing and constructing ideal socialsystems as well.

6 Future Prospects

Twenty years have passed since studiesbegan in the late 1980s on information pro-cessing technologies “inspired by life.” Asdiscussed in this chapter, research “inspired bylife” represents a new paradigm, fully worthyof the term “paradigm shift” coined byThomas Kuhn to describe revolutions in thehistory of science and technology and markingan epochal transition of the highest signifi-cance. If we penetrate into the process of “bio-evolution,” which surpasses human wisdom(once again, let me remind readers that evenhuman wisdom is a product of bio-evolution),we may develop a powerful wellspring ofideas for designing and constructing the futureinformation and communication society.

In this chapter, I have discussed how infor-mation processing models “inspired by life”can be applied to solve various real-worldproblems and provided examples of how suchmodels have actually been applied.

References01 Thomas S. Kuhn, “The Structure of Scientific Revolutions”, The University of Chicago Press, 1962.

02 H. Yukawa, “Readings from Einstein”, vol. 1: Special theory of relativity, quantum theory and Brownian

motion, vol.2: General theory of relativity and unified theory, vol. 3: Einstein and his thoughts, Kyoritsu

Shuppan, 1970-1972.

03 Jerry Mayer and John P. Holms, “BITE-SIZE EINSTEIN”, Kotoku-sha, 1997.

04 Charles Darwin, “On the Origin of Species by Means of Natural Selection or the Preservation of Favored

Races in the Struggle for Life”, London, John Murray, 1859.

05 Richard Leakey abridged and introduced.“The illustrated ORIGIN OF SPECIES”, THE ORIGIN OF SPE-

ICES by Charles Darwin, The Rainbird Publishing Group Ltd. 1979.

06 J. H. Holland, “Adaptation in Natural and Artificial System”, The University of Michigan Press, 1975.

07 A. Waibel, “Phoneme Recognition Using Time-Delay Neural Networks”, Report of Speech Committee,

SP87-100, pp.19-24, Dec. 1987.

31SAWAI Hidefumi

08 A. Waibel, H. Sawai and K. Shikano, “Modularity and Scaling in Large Phonemic Neural Networks”, IEEE

Trans. on ASSP, Vol.37, No.12, pp1888-1898, Dec. 1989.

09 S. Nakamura and H. Sawai, “Performance comparison of neural network architectures for speaker-inde-

pendent phoneme recognition”, Trans. IEICE, D-Ⅱ, vol. J74-D-Ⅱ, No.10, pp.1361-1369, Oct. 1991.

10 K. Fukuzawa, Y. Komori, H. Sawai and M. Sugiyama, “A Segment-based Speaker Adaptation Neural

Network Applied to Continuous Speech Recognition”, Proceeding of the ICASSP’92, San Francisco,

55.1, Vol.1, pp.433-436, Mar. 1992.

11 H. Sawai, “Frequency-Time-Shift-Invariant Time-Delay Neural Networks for Robust Continuous Speech

Recognition”, Proceedings of the ICASSP’91, Toronto, S2.1, Vol.1, pp.45-48, May 1991.

12 H. Sawai, “Axially Symmetric Neural Network Architecture for Rotation-Invariant Pattern Recognition,"

IEEE, WCCI(World Congress on Computational Intelligence), Proceedings of the IEEE ICNN’94, Vol.7,

pp.4253-4258, Jun-Jul. 1994.

13 H. Sawai, S. Kizu, and T. Endo, “Parameter-free Genetic Algorithm (PfGA)”, Tech Report, IEICE, Vol.J81-

D-Ⅱ No.2, pp.450-452, Feb. 1998.

14 S. Adachi and H. Sawai, “Effects of migration methods in parallel distributed parameter-free genetic

algorithm”, Trans. IEICE, D-Ⅰ, Vol.J83-D-Ⅰ, No.8, pp.834-843, Aug. 2000.

15 S. Adachi and H. Sawai, “Evolutionary computation method inspired by gene-duplication: application to

functional optimization”, J. of IPSJ, Vol.42, No.11, pp.2663-2671, Nov. 2001.

16 S. Ohno, “Evolution by Gene Duplication”, Springer-Verlag, 1970.

17 K. Omori, Y. Fujiwara, S. Maekawa, H. Sawai, and S. Kitamura, “Evolutionary computation based on sex-

ual selections driving asymmetrical mutation”, J. of the Institutes of Systems, Control and Information

Engineers (ISCIE), Vol.15, No.8, pp.422-429 (2002)

18 M. Hasagawa, “Why is the male peacock beautiful?”, Kinokuniya Shoten, 1992.

19 Y. Iwasa, “Introduction to mathematical biology – investigating the dynamics in biological society”,

Kyoritsu Shuppan, 1998.

20 K. Wada, H. Doi, S. Tanaka, Y. Wada and M. Furusawa, “A neo-Darwinian algorithm: Asymmetrical

mutations due to semi-conservative DNA-type replication promote evolution”, Proc. Natl. Acad. Sci.

USA, Vol.90, pp.11934-11938 (1993)

21 H. Suzuki, H. Sawai and Wojciech Piaseczny, “Chemical Genetic Algorithms – Evolutionary Optimization

of Binary-to-real-value Translation in GAs”, Journal of Artificial Life No.12, pp.1-27, MIT Press (2006).

22 W. Piaseczny, H. Suzuki and H. Sawai, “Chemical Genetic Programming - Evolutionary Optimization of

the Genotype-to-Phenotype Translation Set”, Journal of Artificial Life and Robotics, Vol.9, pp.202-208,

Springer (2006)

23 C. S. Chan , J. C. Tay and H. Sawai, “Environmental Effects on the Coevolution of Pursuit and Evasion

Strategies”, Journal of Genetic Programming and Evolvable Machines, Kluwer Academic Publishers,

Submitting.

24 Albert-Laszlo Barabasi, “Linked: The new science of networks”, NHK Shuppan, 2002.

25 Mark Buchanan, “NEXUS: Small Worlds and the Groundbreaking Science of Networks”, W. W .Norton &

Company, Inc., 2002.

26 Duncan J. Watts, “Six degrees, The Science of a Connected Age”, Hankyu Communications, 2003.

27 Duncan J. Watts, “Small world – The dynamics of networks between Orders and Randomness”, Tokyo

Denki University Press, 1999.

28 N. Masuda and N. Konno, “Introduction to complex networks”, Sangyo Tosho, 2005.


29 N. Masuda and N. Konno, “What is complex network?”, Kodan-sha, 2006.

30 N. Masuda, “How do we connect each other? – Applying network science”, Chuko Shinsho, 2007.

31 Y. Hayashi (Ed.), “How to spread rumor – unraveling the world by network science”, DOJIN Sensho009,

Kagaku Dojin, 2007.

32 Y. Hayashi (Ed.), “Toolbox of network science – untying the hidden phenomena of connections”, Kindai

Kagaku-sha, 2007.

33 “Special issue on expansion of complex network science”, Vol.49, No.3, Proc. of IPSJ, Mar. 2008.

34 N. Kamibayashi, “From Network to Networking – Social and academic significance of network science”,

Vol.51, No.8, pp.348-353, System, Control and Information Society, 2007.

35 “The forefront of network science”, Mathematical Science, No.520-, Oct. 2006.

36 Steven Strogatz, “SYNC – The Emerging Science of Spontaneous Order”, Hayakawa Shobo, 2005.

37 Steven Johnnson, “Emergence – The Connected Lives of Ants, Brains, Cities, and Software”, SOFT-

BANK Publishing, 2004.

38 H. Tanaka, “Life and complex system”, Baihu-kan, 2002.

39 H. Tanaka, “Life – Evolutional molecular network – An introduction to system evolutionary biology”, Per-

sonal Media, 2007.

40 H. Kitano and K. Takeuchi, “Sound life”, Diamond-sha, 2007.

41 T. Nishiguchi, “Companionship in the long distance and the neighborhood – Successful strategy of

organized network”, NTT Shupan, 2008.

42 J. Kanamitsu, “Foundation on the analysis of social network – Towards the social relational capital”,

Keiso Shobo, 2003.

43 S. Nozawa, “Readings on the theory of network – Family, Community and Social relational capital”,

Keiso Shobo, 2006.

44 H. Ohe, “Sustainable community network – Regional management strategy in the information society”,

Nihon Chiiki Shakai Kenkyusho, 2007.

45 Y. Yasuda, “Human network science – Investigation into the power hidden in the relation between

humans”, Nihon Keizai Shinbun-sha, 2004.

46 Y. Yasuda, “Network analysis – What determine acts ?”, Shinyo-sha, 1997.

47 K. Nishinari, “Study on Congestion”, Shincho Sensho, 2006.

SAWAI Hidefumi Dr. Eng.

Director, Project Promotion Office,Kobe Advanced ICT Research Center,Kobe Research Laboratories

Information Science, Computer Science

PartⅠ Research Trends in the ICT Inspired by Life · tems, we must not only investigate the...

Documents

Transcript of PartⅠ Research Trends in the ICT Inspired by Life · tems, we must not only investigate the...