Compressed Linear Genetic Programming: empirical parameter study on theEven-n-parity problem

Johan ParentVrije Universiteit Brussel

Applied Science Faculty, ETROPleinlaan 2 1050 Brussel, BELGIUM

Ann NoweVrije Universiteit Brussel

Faculty of Science, COMOPleinlaan 2 1050 Brussel, BELGIUM

Anne DefaweuxVrije Universiteit Brussel

Faculty of Science, COMOPleinlaan 2 1050 Brussel, BELGIUM

Abstract- This paper presents a parameter study of ourCompressed Linear Genetic Programming (cl-GP) usingthe Even-n-parity problem. A cl-GP system is a lineargenetic programming (GP) which uses substring com-pression as a modularization scheme. Despite the factthat the compression of substrings assumes a tight link-age between alleles, this approach improves the searchprocess. The compression of the genotype, which isa form of linkage learning, provides both a protectionmechanism and a form of genetic code reuse. This textpresents a study of the different parameters of the cl-GPon Even-n-parity. Experiments indicate that the cl-GPperforms best when compressing a small fraction of thepopulation and the length of the substituted substringsis rather short.

1 Introduction

Genetic algorithms (GA) and genetic programming (GP)search the space of possible solutions by manipulatingthe solution representation using genetic operators likecrossover and mutation. Variable length representations al-low the structure of the solution to evolve but are still re-stricted to the genetic primitives used to build the solutions.If GA and GP are to address more complex problems it be-comes necessary to automatically adapt the representationto the problem. Modularization adapts the representation byextending the set of genetic primitives with problem specificfunctionalities. In this text we present a low level modular-ization technique for linear GP system based on compres-sion. The presented compressed linear GP (cl-GP) attemptsto identify usefull combinations of genes in the populationand makes these available as new primitives. The searchprocess benefit from these new functionalities to progressmore quickly through the search space.

The assumption of tight linkage between individualgenes (cfr substrings) was made with the use of a linearrepresentation in mind. The representation of a linear GPsystem is similar to that used by a GA [4][11]. Followingthe approach used by [11] the cl-GP loop consists of a stan-dard (generational) GA on top of which a problem specificevaluator is placed to simulate the program execution.

This paper is structured as follows Section 2 describesrelated work, In Section 3 the compression/substitutionscheme is presented in detail. In Section 4 the experimentsused to evaluate the impact different parameter settings ofour algorithm is presented. Results are presented in Section

5.

2 Related work

The benefits of modularization for the GP search process arewell known [9][1][7]. Modularization fosters code reuse onone hand, and on the other hand allows the GP system toidentify and use high level functionalities. Different modu-larization strategies for tree based GP have been explored.Automatically Defined Functions (ADF) is the most promi-nent approach. It consists of a main tree which evolves to-gether with a predefined number of additional trees. Thoseadditional trees can be called from the main tree and, assuch, complement the set of primitives available to the GPsystem. An alternative method isencapsulation. It replacesa subtree by a new symbol, this symbol is added to the prim-itive set of the system [6][9]. The symbol created in this waycorresponds to a terminal/leaf node as it does not have anyarguments. A third method,module acquisition, works ina similar fashion, but can create both function nodes (mod-ules) and leaf nodes. If the depth of the selected subtreeexceeds a certain threshold module acquisition removes thesubtrees below this level. In this case a function node willbe created by adding an argument for each removed subtree[2][3]. Although modularization has mainly been of interestto the GP community it is of course related to the search ofbuilding blocks in a GA. In [5] amodule acquisition algo-rithm is presented which exploits modularity and hierarchy.Despite the use of a variable length representation, the prob-lem of modularity is approached purely from a GA point ofview. A moduleis defined as a combination of gene valuesthat maximize the fitness. For example, if for the genesg1

andg2 the combinationg1 = v8 andg2 = v3 dominatesall other combinations of genes value, then it is considereda module. As in the work of [5] our algorithm relies on asubstitution scheme but with notable differences in the sub-stitution strategy. Another difference is that the compressedgenotype scheme of the cl-GP was imagined to be used ina linear GP system. The linear encoding of a GP programexhibits a much tighter linkage than is the case for standardGA problems. This assumption is important as it justifiesthe use of the simple compression scheme which supposesa strong dependency between adjacent values.

Proceedings of the 6th WSEAS Int. Conf. on EVOLUTIONARY COMPUTING, Lisbon, Portugal, June 16-18, 2005 (pp70-75)

0. Choose initial population1. Evaluate each individual’s fitness2. Repeat3. Compress individuals4. Select best-ranking individuals to reproduce5. Mate pairs at random6. Apply crossover operator7. Apply mutation operator8. Decompress individuals9. Evaluate each individual’s fitness10. Until terminating condition

Figure 1: The pseudo code of a standard GA with 2 extrasteps. Step 3 adds the compression of the individual priorto the creation of the next generation. Step 8 decompressesthe individuals so that their fitness can be evaluated.

3 Compressed Linear GP

Our cl-GP algorithm consist of the compressed genetic al-gorithm (cGA) together with problem specific evaluator.This section presents the cGA which consists of a GA ex-tended with a substitution/compression based modulariza-tion mechanism. Figure 1 contains the pseudo code for cGAloop. The cGA adds compression and decompression stepsin the GA loop. Compression is applied to the genotypeof individuals in the population prior to the creation of thenext generation. As a result the search process occurs usinga compressed representation. After the creation of the nextgeneration the individuals are decompressed so that their fit-ness can be evaluated.

The schema theory models the probability of disruptionby crossover is proportional to the defining length of thebuilding block. Compression can shorten the representationof the building blocks thus reduce the chance of disruptionby crossover. The cGA uses a substitution coder to com-press the genotype in an attempt to protect substrings thatrepresent building blocks. Protecting the building blocksguarantees the preservation of good allele combinations andis beneficial to the search process. In Subsection 3.1 the ba-sics of the substitution coder used by the cGA are presented.The compression of the individuals is a two stage stochasticprocess: build a dictionary and select individuals for com-pression. Subsection 3.2 explains how the dictionary usedfor the compression is built. Subsection 3.4 describes howcompression is applied to the population of the cGA.

3.1 Substitution coder

A substitution coder is a lossless1 compression algorithm.Compression is obtained by replacing substrings in the in-put by a shorter reference. A substitution coder uses adic-tionarywhich contains the substrings that need to be substi-tuted and associates a placeholder symbol with each entry.Compression involves two phases: 1) building a dictionaryand 2) performing the substitutions. Figure 2 represents the

1Lossless compression means that the original data is restored aftercompression.

0.D = build dictionary();1. current = in; /* in input string */2. out = ””; /* Empty output */3. For e inD /* Loop A */4. For pos = 1 to|in| /* Loop B */5. If match(pos, current, e.str) Then6. out[pos] = current[pos];7. Else8. out[pos] = e.ref;9. pos = pos +|e.str|;10. End;11. End12. current = out;13.End

Figure 2: The pseudo code for the substitution step of thecoder used in the cGA. Given are the dictionaryD and theinput stringin. The coder searches for a match for each dic-tionary entry (loop A). If a match is found the reference isplaced in the output, otherwise the original symbol. The in-ner loop (loop B) can be replaced by the more efficient KMPstring matching algorithm with a linear runtime complexity.

pseudo code of a substitution coder. The content of the dic-tionary is critical for the compression performance. Muchof the research in data compression concerns the develop-ment of algorithms to build the dictionary. Section 3.2 de-scribes how the dictionary is built in the cGA. The substitu-tion step is computationally expensive as it involves search-ing for a match for each dictionary entry in the input string.If a match for substrings is found its placeholder symbolis placed in the output instead of the original symbols. Thedecompression step involves the replacing the placeholdersymbols by the original strings. Decompression is muchfaster since it does not involve any search.

Suppose a dictionaryD = {”101” → α, ”00” →β, ”11” → γ} then a substitution coder would compressthe individual”1010001011101” to ”αβ0α1α”. One canobserve that the order in which the dictionary entries are or-dered is important. If the opposite order had been used theresult would have been{αβ010γα}.

3.2 Building the dictionary

This section explains the stochastic algorithm used in thecGA to build the dictionary as it differs from the algo-rithm(s) used forpure data compression applications. Weconsidered other criteria to build the dictionary since reduc-ing the memory needed to represent the individuals is notour goal. We seek to build a dictionary containing buildingblocks. The problem of identifying building blocks is recur-rent for all the modularization algorithms. In [5] the iden-tification of a set of alleles as a module involves additionalfitness evaluations. These are used to determine whether analternative substring with a better fitness exists. If no suchstring can be found then the substring will be used as a mod-ule. Another approach, used by [10], uses a separateblockfitness functionto evaluate the merits of a subpart of a ge-netic program. This function is presented as scaled down

Proceedings of the 6th WSEAS Int. Conf. on EVOLUTIONARY COMPUTING, Lisbon, Portugal, June 16-18, 2005 (pp70-75)

version of the fitness function for a smaller version of theoriginal problem. The strategy adopted in this work is tosee a GA as the building block discovery process. Sinceby definition building blocks are contributing in a positiveway to the fitness of an individual, they are bound to bepresent in the better individuals of a population. We be-lieve that this approach is more general as it 1) relies onthe information already present in the population, 2) avoidsadditional computations and 3) does not require additionalfitness functions to be engineered. Especially the latter twounacceptable when it comes to real world applications.

The cGA builds its dictionary in two stages. First, a poolof M individuals is selected stochastically from the popula-tion. The genotype of these individual will be used to createthe dictionary in the second stage. These individuals are se-lected using fitness proportional selection. The pool consistmainly of above average individuals, yet with a minimumof diversity. Once the pool has been created every substringof length l of each individual is added to the dictionary.The dictionary only contains strings of the same length2.For example, suppose a non-binary alphabet{a, b, c, d} andthe substring lengthl equal to 3. In this case the individ-ual ”abccdabc” would add the substring (and their respec-tive placeholder)”abc” → α, ”bcc” → β, ”ccd” → γ,”cda” → δ and”dab” → σ to the dictionary. The cGA dic-tionary contains next to the substrings and their placeholdersymbol a counter of the occurrence of every substring in thepool. This counter is used to order entries of the dictionary.We have chosen to sort the dictionary entries based on theoccurrence of each substring, the most frequently occurringsubstring(s) will be substituted first.

The cGA rebuilds the dictionary for each generation.This allows to update the content of the dictionary with in-formation that reflects the evolution of the population. Thisdiffers from [5] where themodule formation algorithmisapplied periodically. This also excludes the presence of dic-tionary entries containing references to other entries.

3.3 Modularization

Applying compression to the genotype can protect buildingblock. But it does not by itself provide modularization com-parable to that of ADFs, encapsulation or module acquisi-tion. Modularization implies that a somehow code reuseshould be present. Code reuse is achieved by the cGA byadhering to several conditions. First of all, the representa-tion used by the cGA is adapted during the search. A newsymbol is added to the alphabet of the genetic system foreach potential building block identified by the cGA. Sec-ond, the genetic operator must be unrestricted: no distinc-tion is made between compressed symbols and symbols ofthe original alphabet. Mutation for instance can replace acompressed symbol by anoriginal symbol and vice versa.The third condition is that no distinction is made betweenindividuals with and without compressed genotype. As willbe explained in section 3.4, all the individuals are not nec-essarily subjected to compression. By not discriminating

2In traditional substitution coders the dictionary can contain entries ofdifferent lengths

between original genes andcompressed genes, the cGA canmake full use of the genetic combinations that where iden-tified as possible building blocks. It then becomes possiblefor the cGA to address problems where repetition is present.Crossover and mutation operators serve as natural mecha-nism to obtain this form oftranslocationof genetic infor-mation. Translocation occurs when a part of a chromosomeis detached and reattached at a position different from theits original position in the chromosome.

3.4 Compressing the population

Once the dictionary has been built the substitution coder(Subsection 3.1) can be used to compress the individuals inthe population. The cGA applies compression in a stochas-tic fashion to a part of the population. A fractionκ (∈ [0, 1])of the population will be compressed. This fractionκ of in-dividuals are selected using tournament selection. Since thecompression setup is repeated anew for each generation thismeans that the cGA does not systematically compress thesame individuals. This makes it possible to explore the ben-efit of the different modules (dictionary entries) in differentcontexts (ie. allele combinations).

3.5 Lossless and stochastic

The cGA applies a deterministic transformation, compres-sion, to some of the individuals in the population. Sincethe compression is lossless the (genetic) information presentin the population of the cGA or a GA is exactly the same.Yet, as described above, a stochastic component has beenadded to the overall population compression process. Thecreation of the dictionary and the selection of the individ-uals for compression are non-deterministic processes. Thisstochastic component is meant to counter balance the ef-fect of the substitutions on the search process. The pro-tection against crossover provided by the compression hasa negative impact on the population diversity. Several fac-tors explain this phenomenon. First the search efficiencyof the crossover operator is reduced by the compression ofthe genotype. Second, the compression of the individuals isdetrimental for the sampling of schemata. A last reason isthat the dictionary entries are not guaranteed to correspondto building blocks. This situation is especially exacerbatedduring the first generation as the population still needs todiscover promising gene combinations. This was illustratedby experiments where the entire population underwent com-pression. These experiments have invariably lead to prema-ture convergence and consequently suboptimal results. Theparameterκ is a way to limit the decrease in the popula-tion diversity. The other non-deterministic steps were intro-duced for the same reason as they maintain a minimum ofdiversity at the substring level.

3.6 Further changes

Although the cGA is meant to be a minimal extension to thestandard GA a few extra modification were required. Thebiggest difference is an unavoidable transition to a variable

Proceedings of the 6th WSEAS Int. Conf. on EVOLUTIONARY COMPUTING, Lisbon, Portugal, June 16-18, 2005 (pp70-75)

length linear representation. This change is due to the com-pression scheme.

3.6.1 Variable length

Despite the use of a lossless compression algorithm and thefact that the cGA, like a standard GA, starts with a popula-tion of individuals of identical size, individuals of differentsizes quickly emerge. This phenomenon, due to the use ofcompression, occurs through several mechanisms:

1. not all the individual are compressed

2. different individuals compress to different sizes

3. the genetic operators can create individuals of verydifferent sizes

Since the cGA does not discriminate between place-holder symbols andoriginal symbols, recombination andmutation can add or remove symbols representing a sub-string and vice versa. This can result in an important changein length. Depending on the situation individuals can ex-ceed the initial individual size or be shorter. Consider thefollowing example. Suppose the dictionaryD = {”xy” →α, ”zz” → β} over the alphabet{x, y, z}. Mutating thesecond gene of the individual with genotype”xx” from xto α creates”xα”. The decompressed version of this indi-vidual’s genotype is now”xxy”. Similar scenarios existsfor the crossover.

3.6.2 Recombination and mutation

As illustrated above the genetic operators can create indi-viduals of very different sizes. As a result a maximum indi-vidual length has to be enforced to avoid bloat. Individualscreated by crossover that exceed the maximum length aretruncated. Yet in the case of both mutation and crossoverone problem remains. As illustrated in the previous para-graph it is possible for an individual to exceed the maxi-mum once it has been decompressed. In this case the cGAtruncates the genotype after decompression. The mutationoperator was modified to be able to cope with a variablealphabet sizes.

4 Experiment

The advantage of the modularization of the cl-GP, providedby the cGA, has been illustrated in different GP problems:symbolic regression, classification and a real world data ap-plication. This section presents the experiments used tostudy the impact of the parameters of the cl-GP. We used theEven-n-parity problem for our different experiments. It hasbeen chosen since it is a standard GP problem. Furthermorethe difficulty of this problem can be adjusted by modifyingthe number of input bitsn. The cl-GP introduces three newparameters: the size of the pool used to build the dictionary,κ the fraction of the population being compressed andl, thelength of the dictionary entries.

The number of individuals used to build the dictionaryshould influence the quality of the dictionary entries. It isexpected at a large pool performs better than a small one.

κ controls the population compression. Compressing abig fraction of the population is expected to have a negativeinfluence on the search. The protection against crossover,the result of the cl-GP compression, reduces the number ofpossible individuals create by the crossover thus the popu-lation diversity.

The last parameter is the length of the dictionary entries.The cl-GP assumes there is a tight linkage in the representa-tion. Using large substrings as dictionary entries is expectedto reduce the performance. This is due to the increasingdifficulty of identifying substring which corresponds to aschema of lengthl as the value ofl increases.

4.1 Even-n-parity

The Even-n-Parity problem requires the correct clas-sification of bit strings of lengthn having an evennumber of 1’s. This classification is formulated as aboolean function returning the valuetrue for an evennumber of 1’s and false otherwise. The terminalsand function set areT = {b0, b1, · · · , bn} and F ={NOOP, AND, OR,NAND, NOR}. The NOOP in-struction is not executed during the evolution. This instruc-tion allows to represent programmas of variable length us-ing a fixed length representation [11]. We believe the inclu-sion of this instruction is necessary in order to compare thel-GP, driven by a GA (fixed length), and the cl-GP drivenby the cGA (variable length). The raw fitness (fraw) isthe ratio of correct classifications over the entire data set:fraw = 1 − #errors

2n . The standard fitness is equal to theraw fitness for this problem. The evaluator used for thisproblem is described in the next section.

4.1.1 Boolean evaluator

The Even-n-parity individuals are postfix encoded and eval-uated by a stack based virtual machine as described in [8].Its instruction set provides the four boolean operators3 andfive terminals. The operations take their 2 operands fromthe stack and push their result onto it. The terminal instruc-tions correspond to a push of the individual input bits on thestack. As for the numerical evaluator the result of the pro-gram is the value of the top of the stack after evaluation. Theindividual [b0, NOR, NAND,b0, b4, OR, AND,b2, OR,b2,NOOP, NOR, NAND ] corresponds to the boolean expres-sion (b0 OR b4) NAND (b2 NOR b2).

5 Results

All the results presented here are the average of 100 inde-pendent runs, using randomly seeded initial populations of500 individuals. The genotype length is 32. Other settingswere: crossover rate 80%, mutation rate 5% and the top 5%of the population was kept at every generation. Every ex-periment lasted 50 generations. If not mentioned otherwisethe following values were used for the runs using the cl-GP:

3Following the observations of [8] the lazy version of the operators wasimplemented.

Proceedings of the 6th WSEAS Int. Conf. on EVOLUTIONARY COMPUTING, Lisbon, Portugal, June 16-18, 2005 (pp70-75)

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

0 5 10 15 20 25 30 35 40 45 50

Fitn

ess

Generation

sGA (500)sGA (100)cGA (500)cGA (100)

Figure 3: An example run for the Even-5-parity problemfor population size of 100 and 500. The linear GP systemdriven by the cGA (cl-GP) outperforms the same systemusing the GA.

κ = 0.30 (tournament size 4) and the length of the dictio-nary entries equals two. A pool of 10 individuals (fitnessproportional selected) is used to build the dictionary. Be-fore presenting the results for the different experiments, thenext subsection illustrates the benefit of using the cl-GP fortwo Even-n-parity problem instances. It should be notedthat large problem instances (n >= 10) could not be solvedwith the given genetic primitives.

5.1 cl-GP (cGA) vs l-GP (GA)

This subsection compares the results for the Even-5-parityproblem instance using runs with 2 different populationsizes. The use of substitution had a positive effect on theperformance of the GP system. Figure 3 compares the fit-ness of the best of population for both algorithms.

In order to further compare the performance of the twoalgorithms the cumulative probability of success for Even-5-parity has been computed using runs of 200 generations(population 500). The linear GP driven by the GA achievesa 28% probability of success after 200 generations. Thesame setup but using the cGA, the cl-GP, obtains a prob-ability of success of 74%. For the Even-6-parity problemthese numbers become 8% and 28% respectively.

5.2 Population compression,κ

Table 1 contains the raw fitness for the best individuals ofthe population for different problem instances. The secondcolumn, 0% of population selected for compression, servesas the reference since in this case the cl-GP is equivalent toa normal l-GP. As was expected high values do not lead toimproved performance. This is most noticeable for smallproblem instances (n = 5 for example). Surprisingly, thistendency seems to reverse as the problem size increases.

Table 2 illustrates the effect ofκ on the average individ-ual length. Subjecting a larger fraction of the population tocompression logically results in a lower average individualsize.

n κ = 0.0 0.3 0.6 0.95 0.891 0.964 0.950 0.9497 0.680 0.783 0.796 0.7699 0.5 0.528 0.530 0.536

Table 1: The number of individuals that are selected forcompression influence the performance of the system. Highvalues (60% and 90%) reduce the diversity and reduce theoverall performance.

n κ = 0.0 0.3 0.6 0.95 32 20.76 17.85 17.257 32 21.99 18.63 17.359 32 22.71 19.03 17.63

Table 2: The average individual size after 50 generations.The reduction in size decreases slightly as the problem in-stances become harder to solve.

5.3 Pool size,N

Table 3 presents the raw fitness of the best of populationwhen using different pool sizes. The use of fitness propor-tional selection (with overselection) makes the cl-GP rela-tively insensitive to the pool size. This can be explainedby the use of overselection when creating the pool whichmakes it possible for the some individuals to be present sev-eral time in the pool. As a result pools of different sizes canin fact show little differences in diversity.

5.4 Substring length,l

Table 4 contains the fitness of the best individual when us-ing different substring lengths. Using longer dictionary en-tries decreases the performance. This decreasing trend canbe observed for all the problem instances. As the substringsget longer it becomes harder for the cl-GP to protect goodschemata. This clearly illustrates that by protecting sub-optimal building blocks the performance of the cl-GP is pe-nalized.

6 Discussion

The cl-GP uses the cGA presented in this paper. The cGAprovides a lowest level modularization mechanism based oncompression. Although the linear GP system performs bet-ter when using the cGA than with the GA, the differencesget smaller as the problem instances become harder. This

n N = 10 20 50 1005 0.964 0.964 0.964 0.9657 0.783 0.795 0.785 0.7749 0.528 0.537 0.531 0.524

Table 3: The raw fitness of the best of population as a func-tion of the pool size used to build the dictionary of the sub-stitution coder. The difference between various pool size isless pronounced compared to influence ofκ on the perfor-mance.

Proceedings of the 6th WSEAS Int. Conf. on EVOLUTIONARY COMPUTING, Lisbon, Portugal, June 16-18, 2005 (pp70-75)

n l = 2 3 4 55 0.964 0.948 0.931 0.9097 0.783 0.753 0.750 0.7179 0.528 0.524 0.527 0.522

Table 4: Increasing the length of the dictionary entriesmakes it less likely for the assumption of tight linkage be-tween gene values to hold. For the problem sizes the rawfitness decrease as the substring length increases.

can be explained by the fact that the cGA, being an ex-tension to the standard GA, capitalizes on the GA’s abilityto find good combinations. When the GA fails to identifybuilding blocks it becomes impossible for the cGA to reuseto combinations to its advantage. In that respect the influ-ence of the substring lengthl and the population compres-sion ratioκ suggests that it is better to make moderate use ofgenotype compression. Longer substrings give higher com-pression ratios (l to 1) but put more strain on the tight link-age assumption of the cGA. Similarly, a highκ indirectlyrestricts the population diversity and diminishes the perfor-mance.

7 Conclusions

In this paper we introduce the cl-GP system which uses aGA variant which uses compression as a modularizationmechanism. The modularization of scheme of the cGA as-suming a tight linkage between elements of building blocks.This allows it to replace substrings in the genotype with ashorter reference. This scheme was designed to be used ina linear GP system (cl-GP) where the assumption is morelikely to hold. We performed a empirical study of the in-fluence of the different the cl-GP parameters using sev-eral Even-n-parity problem instances. Results illustrate thestrong influence of the substring length and population com-pression ratioκ on the performance.

8 Future work

The influence of the parameters of the cl-GP needs to bestudied on a wider range of GP problems. Although sub-strings of length 2 seem to be best at this point, it may beinteresting to allow this size to change during the course ofevolution. Alternatives to the used of fitness proportionalselection for creating the pool of individuals should be ex-plored.

Bibliography

[1] M. Ahluwalia and L. Bull. Coevolving functions in ge-netic programming.Journal of Systems Architecture,47(7):573–585, July 2001.

[2] P. J. Angeline and J. B. Pollack. The evolutionary in-duction of subroutines. InProceedings of the Four-teenth Annual Conference of the Cognitive Science So-ciety, Bloomington, Indiana, USA, 1992. LawrenceErlbaum.

[3] P. J. Angeline and J. B. Pollack. Coevolving high-levelrepresentations. In C. G. Langton, editor,ArtificialLife III , volume XVII of SFI Studies in the Sciencesof Complexity, pages 55–71, Santa Fe, New Mexico,15-19 June 1992 1994. Addison-Wesley.

[4] M. Brameier and W. Banzhaf. Effective linear geneticprogramming. Technical report, Department of Com-puter Science, University of Dortmund, 44221 Dort-mund, Germany, 2001.

[5] E. D. de Jong and D. Thierens. Exploiting modular-ity, hierarchy, and repetition in variable-length prob-lems. In K. Deb, R. Poli, W. Banzhaf, H.-G. Beyer,E. Burke, P. Darwen, D. Dasgupta, D. Floreano, J. Fos-ter, M. Harman, O. Holland, P. L. Lanzi, L. Spector,A. Tettamanzi, D. Thierens, and A. Tyrrell, editors,Genetic and Evolutionary Computation – GECCO-2004, Part I, volume 3102 ofLecture Notes in Com-puter Science, pages 1030–1041, Seattle, WA, USA,26-30 June 2004. Springer-Verlag.

[6] D. Howard. Modularization by multi-run frequencydriven subtree encapsulation. In R. L. Riolo andB. Worzel, editors,Genetic Programming Theory andPractise, chapter 10, pages 155–172. Kluwer, 2003.

[7] J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Ri-olo, editors.Genetic Programming 1996: Proceedingsof the First Annual Conference, Stanford University,CA, USA, 28–31 July 1996. MIT Press.

[8] T. Perkis. Stack-based genetic programming. InPro-ceedings of the 1994 IEEE World Congress on Com-putational Intelligence, volume 1, pages 148–153, Or-lando, Florida, USA, 27-29 June 1994. IEEE Press.

[9] S. C. Roberts, D. Howard, and J. R. Koza. Evolv-ing modules in genetic programming by subtree en-capsulation. In J. F. Miller, M. Tomassini, P. L. Lanzi,C. Ryan, A. G. B. Tettamanzi, and W. B. Langdon,editors, Genetic Programming, Proceedings of Eu-roGP’2001, volume 2038 ofLNCS, pages 160–175,Lake Como, Italy, 18-20 Apr. 2001. Springer-Verlag.

[10] J. P. Rosca and D. H. Ballard. Hierarchical self-organization in genetic programming. InProceedingsof the Eleventh International Conference on MachineLearning. Morgan Kaufmann, 1994.

[11] K. Stoffel and L. Spector. High-performance, paral-lel, stack-based genetic programming. In J. R. Koza,D. E. Goldberg, D. B. Fogel, and R. L. Riolo, editors,Genetic Programming 1996: Proceedings of the FirstAnnual Conference, pages 224–229, Stanford Univer-sity, CA, USA, 28–31 July 1996. MIT Press.

Proceedings of the 6th WSEAS Int. Conf. on EVOLUTIONARY COMPUTING, Lisbon, Portugal, June 16-18, 2005 (pp70-75)