Co-existence of Evolutionary Mixed-Bias Scheduling withQuiescence and IEEE 802.11 DCF for Wireless Mesh Networks

Jason B. ErnstSchool of Computer ScienceUniversity of Guelph, Guelph

Ontario Canada N1G 2W1jernst@uoguelph.ca

Joseph Alexander BrownSchool of Computer ScienceUniversity of Guelph, Guelph

Ontario Canada N1G 2W1jbrown16@uoguelph.ca

Abstract—The evolutionary programming mixed biasscheduling has been shown to provide significantly reducedend-to-end delay while maintaining comparable packet deliveryratio when evaluated using simulation experiments comparedto the current IEEE 802.11 DCF standard. In this paper,proposed is an enhanced MB-EP algorithm which includes anew quiescent state called MB-EP-Q. This new approach isevaluated using network simulation with respect to throughputand delay. The new approach was found to have similarperformance to the existing MB-EP approach. Furthermoreto demonstrate that the MB-EP-Q approach can co-exist withexisting IEEE 802.11 DCF mechanisms another experiment isperformed where a portion of the routers are running theMB-EP-Q algorithm while the rest run IEEE 802.11 DCF.The results show an improvement in performance even whena small proportion of the nodes are running the MB-EP-Qalgorithm. This result is important since it shows that the MB-EP approaches can be applied to existing deployments graduallyand with lower cost than other competing approaches whichmay require the entire network infrastructure to be modified.

Index Terms—IEEE 802.11 DCF, co-existence, parametercomputation, feedback, scheduling, evolutionary algorithms,evolutionary programming, mixed bias, wireless mesh network

I. Introduction

Wireless Mesh Networks (WMNs), and in particular IEEE802.11 DCF has been shown to suffer from performanceproblems as the number of hops from the gateways inthe networks increase. This has been solved previouslyby biasing against the number of hops using proportionalfairness, max-min fairness, and mixed-bias [?], with mixed-bias performing the best [?] [?]. Recent enhancements to theoriginal mixed-bias approach include a tabu search method[?] and an evolutionary programming method [?]. However,this approach only considered two actions (up or down) onthe parameters in the mixed-bias approach. In this paper, wepropose to introduce a third action which allows a parametervalue to remain unchanged, known as a quiescent state.Further, we wish to study how the network performs withonly a subset of the mesh routers running the proposedalgorithm. Since the proposed MB-EP-Q algorithm has theability to evolve, the assumption is that the nodes running theevolving algorithm are able to work and adapt to overcomethe lack of co-operation by the other nodes. This would allowfor a smooth transfer between systems, allowing a network

to be slowly transferred into the new technology, rather thanrequiring the entire system to be replaced.

The remainder of the paper is organized as follows:Section II presents a background on evolutionary algorithmsand how it can be applied in an online manner for use inwireless networking algorithms. In Section III, the proposedapproach is given in detail. Next, Section IV details thethe simulation environment and experiments. In Section V,evaluates the performance of the MB-EP-Q algorithm againstthe previous MB-EP algorithm. Additionally, performanceresults for co-existence of the MB-EP-Q algorithm withIEEE 802.11 DCF are presented. Finally, Section VI givesconclusions and future directions for further research.

II. Background and RelatedWork

One of the goals of mixed-bias scheduling is to provide abalance between fairness and performance in wireless meshnetworks. Many existing approaches also have similar goals.There are many benefits for providing fair service to userswithin a network. In the case where all users are payingthe same amount of money to access the network, it shouldbe expected that all will receive similar levels of service.However, this is a difficult problem in particular in wirelessmulti-hop networks like wireless mesh networks. End-userswhich are farther away from the gateways are less likely tohave good performance as those which are located physicallycloser to the gateway since they are more likely to be stuckin queues at each hop along the way. These flows are subjectto more contention, interference, and other wireless problemsalong the path. This provides motivation for biasing againstundesirable network characteristics such as distance fromthe gateway, link quality, congestion and so forth. In [?]the authors proposed a proportional scheme to address thistype of problem. It is noted that these types of solutionsoften suffer from throughput oscillations and also proposeda mechanism to help prevent this. In this approach, since thethe routers are continually evolving the network parameters,oscillations should be avoided and extra mechanisms arenot required. There are also some problems with trafficoverhead co-coordinating the proportional fairness schemeswhich was noted by [?]. This is also avoided in our proposedscheme because each router independently evolves its ownparameters without communication from the other routers

2012 26th International Conference on Advanced Information Networking and Applications Workshops

978-0-7695-4652-0/12 $26.00 © 2012 IEEE

DOI 10.1109/WAINA.2012.215

678

other than a periodic exchange of packet delivery ratio anddelay information from the gateway. Singh et al [?] analyt-ically showed that mixed-bias fairness achieves per clientfairness and obtains higher performance than proportionalfairness when biasing against distance from gateway. Anothercommon approach to scheduling and resource allocation ismax-min fairness. In this case the limitation is that somenodes or flows may be subject to starvation. Since mixed-bias scheduling is based off of proportional fairness, it doesnot suffer from the starvation problem, but has improvedperformance than proportional fairness. One recent max-minfairness approach [?] is dynamic and allows the network tomove between aggressive and conservative resource alloca-tions. However, it is centralized and suffers from single pointof failure problems while our approach allows each router tooperate autonomously in a decentralized manner.

Evolutionary Algorithms(EAs) use approaches inspired bybiology to give solutions to problems of modelling, optimiza-tion, and prediction problems. EAs are used for prediction inboth offline, learning from a previously classified set and thenbeing applied to predictions, or online, making predictionsand changing the model. Online evolution allows for aresponsive modelling of current states within the system.Evolutionary Programming(EP) [?][?] provides an onlinemethod for the evolution of Finite State Machines(FSM)as string predictors. EPs mutate a phenotype of numerouscurrent population members, unlike the Genetic Algorithms(GAs) approach which searches at the genetic level, breedingamong population members[?].

Previous use of FSM for prediction problems have in-cluded their use in bioinformatics, such as the creation ofpolymerase chain reaction primers for corn[?] and mice[?].These FSM commonly include predictors which allow forquiescent state or undecided state; an “I don’t know” predic-tion. Agents, represented by FSM, for the prisoner’s dilemmahave been developed using EP[?], as well as GA[?], andother EA[?]. These created agents show complex behaviourssuch as handshaking and taking into account other agentsvulnerabilities. In channel encoding EP has been used inorder to create trellis-coded modulation schemes for datasequences over white Gaussian noise[?].

The EP approach has been used for the problem of settingparameters of a mixed bias scheduling and found it to bebetter than IEEE 802.11 DCF in terms of packet deliveryratio while still being on par in terms of delay[?]. In thisversion the output of finite state machines produced waseither to increase or decrease the parameter. One goal ofthis paper is to examine the same system with an additionaloutput which retains the current setting, a quiescent output.

III. Proposed Approach

In this section details are provided on the evolutionarymixed-bias algorithm with quiescence (MB-EP-Q). First, thenetwork model is defined and details are given as to theimplementation within the layered network model. Secondly,information is provided regarding the specific details of the

evolutionary approach, genetic operators used and how thefitness is evaluated.

A. Network Model

For this work, the network model considered is a wire-less mesh network. This network consists of mesh routers,which form the backbone and infrastructure of the network,gateways, which forward traffic to the Internet or other sub-networks through a gateway device, and mesh clients, whichare the end-users’ devices. Traffic is considered in flowsfrom a random source within the network towards the singlegateway device. It is assumed that the routers are able todetermine their distance in hops between the router and thegateway using the routing protocols present on the router. Theproposed algorithm runs at the Medium Access (MAC) layerand manipulates the MAC layer queues to give preferenceto certain flows based on the distance from the gateway. Itis also possible to consider other characteristics other thandistance from gateway such as node utilization, link qualityand others, but in this particular approach only gatewaydistance is considered.

B. Mixed-Bias Scheduling Model

The idea behind this mixed-bias scheduling approach isto give preference to nodes which are closer to the gatewaysince they handle their own traffic in addition to the trafficfrom the peripheral nodes in the network. At the same time,flows with a source closer to the gateway are given slightlyless priority than nodes which are farther out. This way delayis spread out across all of the flows more evenly. Also, as aflow from farther out traverses hops and gets closer to thegateway, it is more detrimental to performance within thenetwork if packets are lost before the gateway. If the packetis lost due to queueing because of a closer flow utilizingmuch of the link, all of the previous hop bandwidth used bythe flow was wasted.

R =α

cβ1+

1 − αcβ2

(1)

Mixed-bias scheduling is determined using equation 1 [?][?] [?]. R determines the probability a node will immediatelyforward a particular packet at a router where c is the distancefrom from the router to the gateway. If the packet is notforwarded, it is pushed back in the queue and packets fromother flows are allowed preference to transmit in its place.This scheduling scheme is similar to proportional fairness,where flows are given priority which is inversely proportionalto the distance from the node to the gateway. In mixed-bias scheduling, however, the proportionality can be morefinely tuned with a strong and weak biasing. This tuningcan be accomplished using the alpha and beta parametersin the equation. It has been shown previously that it ispossible to dynamically tune these parameters using variousapproaches such as online tabu search [?], and a previousevolutionary technique [?]. This approach aims to improveupon the previous evolutionary technique by addressingone of the shortcomings in the previous approach. In the

679

previous approach, the evolutionary approach allowed onlytwo actions - parameter increase or parameter decrease. Theintroduction of a third state which allows the parameter toremain unchanged is therefore a logical extension of thecurrent line of research.

C. Finite State Machines

The type of finite state machine (FSM) used, is known as aMealy Machine[?]. It is defined as a hextuple, 〈Q, I,Z, δ, ω, q〉with the following:

• Q is a finite set of states.• I is a finite set of input symbols which are taken from

calculations made by the system.• Z is a finite set of output symbols which is the manip-

ulation to be made to the system.• δ is the state transition function, such that δ : I×Q → Q.

It maps the current state and the input to the next state.• ω is the output function, such that ω : I × Q → Z. It

maps each state transition to an output symbol in Z.• q is an initial state such that q ∈ Q.

FSM are uniquely defined therefore by giving a statediagram with the initial state clearly marked or by a transitiontable showing the outputs of each of the state transition andoutput functions with the initial state. Figure 1 gives anexample of both of these representations for the same FSM.The FSM in the study are allowed to vary in size, however,the value of Nmax is used as a limit on the maximum numberof states a machine will be allowed to possess. In comparisonwith the previous work in [?], we have introduced some newtransitions and actions based on the quiescent state whichwas added.

D. Evolutionary Programming

Evolutionary Programming was developed in the 1960sby Lawrence J. Fogel as a method of evolution of FSMfor string prediction problems[?]. This type of EvolutionaryAlgorithm was originally used in an online fashion. Thatis the evolution occurs while prediction is being made.This allows for responsive predictions while still providinga capacity for learning. The EP accomplishes an onlineevolutionary search via the following process: a populationof FSM is first randomly initialized, where one is randomlyselected to be the current machine making a prediction. Whena new prediction is called for each machine saves its responseand the current prediction machine returns its response asthe action to be made by the system. After a window ofpredictions has been made, each of the machines is evaluatedand given a fitness based upon how well their responsematched actual result. The population is sorted by fitness, themost fit machines are copied with small changes, known asmutation, over the least fit members of the population. Themachine with the highest fitness now becomes the currentpredictor for the next window, where the cycle of prediction,fitness evaluation, and mutation continues.

start

1

2

+/+

3

-/- +/?

-/+ -/+

+/?

Initial: 1State If + If −

1 +→ 3 − → 22 ? → 2 +→ 33 ? → 3 +→ 2

Fig. 1. Example Mealy Finite State Machine State Diagram and TransitionTable with quiescent outputs. State diagram read as input/output. Transitiontable read as output symbol and transition to state.

E. Genetic Operators

The mutations to finite state machines which can beselected in this system are: change an action, change atransition, change the current initial state, adding a state, orremoving a state. Mutation operators are selected uniformlyat random. In the event that a FSM has Nmax states theaddition action was excluded, if it has one state then thedeletion is excluded. More details are about the operatorsare available in [?].

Each of these operators has a varying degree of disruptionto the underlying logic of the FSM. Changes to transitionsor output actions make little changes in the behaviour, andsometimes no change at all such as changing the outputtransition on a state with no incoming transitions. These nullmutations, on which has no effect on the fitness of a machine,are not discouraged. Their detection would be computation-ally expensive, the mutation might be an intermediary stepin a chain of mutations leading to greater fitness, and duringevolution it is often helpful to have a level of inheritancewhen fitness is relatively higher for a machine as it promotesexploitation. Conversely, their exist operations which willcause drastic disruption to current logic in the FSM, additionand deletion of states. Having both types of operators allowsthe EP to exploit current areas of the search space and explorenew areas of the search space.

680

F. Evaluation of Fitness

Fitness was determined by measurement taken from themesh routers. At each mesh router, the packet deliveryratio is maintained over time as well as delay. Fitness isdetermined by combined utility balanced between both ofthese measures. Over the prediction period, if the utilityincreases, good feedback is passed to the prediction engine,otherwise negative feedback is provided. For this experiment,the packet delivery ratio and the delay were given equalweight in the fitness, however this may be altered dependingon the application of the network.

In EPs the largest cost in runtime comes not from thegenetic operations made upon the FSM but the evaluation offitness. Simulation is commonly used in offline approachesto EAs, however it is not a suitable method of evaluation forthis problem for the following reasons: 1) each node wouldrequire a large coprocessor to run simulations of networkstate, 2) each node would require knowledge of the networktopology and the state of other nodes on the network, and3) the speed of simulations is slower than reality makingany results acquired from simulations long out of date whena prediction is required immediately. This reasoning meansthat in order to use an evolutionary approach some heuristicassumptions are required in the fitness evaluation. The mainheuristic is that if the current prediction machine’s choiceincreased the fitness, then the opposite choice would lead toa directly corresponding drop in fitness, and vice versa. Theparameters could be in a state where performance decreasesindependently due to a confounding factor and there is noway to discover that without costly simulation.

For quiescent states this same model of fitness is used.That is if the current prediction machine’s choice was cor-rect/incorrect, then the other outputs are incorrect and correctrespectively. This of course is an assumption which can leadto mistakes. Take for example if the current machine keepsthe parameter the same, however performance will improveif it is increased. Then machines which select an increase inthe parameter will suffer the same drawbacks as those whichselected to reduce the parameter. Again, this is unavoidableas we can only know the results of the current predictionmachine without resorting to simulation.

IV. Simulation and Experimental Environment

In the simulation there are two different experiments whichuse the network parameters as defined in Table I. Theseparameters match those in the earlier work [?] for comparisonpurposes.

A. Experiment One: Quiescent States

In the first experiment, evaluating the performance of theMB-EP-Q approach against MB-EP, the number of meshrouters range from between ten to thirty routers, each timeincreasing by five routers. In each case there are threenetwork flows originating in different places around thenetwork heading towards the single gateway node. The trafficwas originates near to the gateway, far from the gateway and

TABLE ISimulation Parameters

Parameter ValueSimulation runs 30Interarrival rate 0.01Packet size 1024 bytesMRs 10 - 30Distance between MRs 100mSource MRs 3Simulation time 100sPackets per prediction round 150Population size 10Mutation events per round 10Maximum mutations to a child 3Prediction period 20Nmax 10

somewhere in between to study the effect of the distancefrom the gateway.

B. Experiment Two: Co-existence

The second experiment evaluates how well the MB-EP-Q algorithm can co-exist with existing nodes running theIEEE 802.11 DCF mechanism. In this case, the maximumnetwork size of 30 mesh routers is used. Then the numberof routers running the MB-EP-Q algorithm varies betweenzero and all thirty of the routers. In the case where there arezero routers running the algorithm, this is comparing againstthe IEEE 802.11 DCF algorithm. In the case where all of thenodes are running MB-EP-Q, it is the same result as the firstexperiment with 30 routers. In this experiment, the trafficmodel is the same as the first experiment with three trafficflows from different areas of the network flowing towardsthe single gateway router.

V. Performance Evaluation

A. Quiescent States

In this subsection, provided evaluating of the performanceof the evolutionary mixed-bias approach with quiescence(MB-EP-Q) against the normal evolutionary mixed-bias ap-proach (MB-EP). Previously, it has been demonstrated thatthe MB-EP approach performs better than standard IEEE802.11 DCF and normal mixed-bias [?]. In this case, theinterest is in comparing the performance between the newapproach, MB-EP-Q and the previous approach MB-EP.

The difference between the states with is within the mar-gins of error for both PDR (Figure 2) and end-to-end delay(Figure 3). Therefore, we are indifferent between them for theuse. The MB-EP-Q is, therefore, is better than IEEE 802.11DCF in terms of delay and comparable in terms of packagedelivery ratio when the size of the network increases. As towhy there is not much change in the results we must lookback at the evaluation of fitness, the amount of informationthat the state machine can receive from the fitness, and thesize of the search space.

The evaluation of fitness in both cases is the same, howeverthe amount of information about a choice which can beextracted is not. By adding the additional state we have

681

0.8

0.85

0.9

0.95

1

10 15 20 25 30

Pac

ket D

eliv

ery

Rat

io

Mesh Routers

MB-EP-QMB-EP

Fig. 2. Quiescent States EP - Average packet delivery ratio with varyingnetwork sizes with 95% confidence intervals

removed some of the ability of the fitness function to providedirection in the search. As can be readily seen, each of thestate machines which can be produced by MB-EP exists inMB-EP-Q, they are a proper subset. The additional stateexpands the search space which can either give new bettersolutions or which can increase the time to find the samesolution as a reduced space. This trade off leads to nosignificant gains in our searches. These results, however,bolster the idea that the evolutionary method is resistant toparameter settings.

0

0.01

0.02

0.03

0.04

0.05

10 15 20 25 30

Ave

rage

End

-to-

end

Del

ay

Mesh Routers

MB-EP-QMB-EP

Fig. 3. Quiescent States EP - Average end-to-end delay with varyingnetwork sizes with 95% confidence intervals

B. Co-existence with IEEE 802.11 DCF Routers

This subsection discusses the performance of the MB-EP-Q algorithm running on a subset of the nodes in the network.In both cases, these results can be compared against theprevious two figures with the number of MRs set to 30. Itshould be noted that similar performance is obtained with asfew as on third of the routers running the EP algorithms.

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

0 5 10 15 20 25 30

Pac

ket D

eliv

ery

Rat

io

Mesh Routers Running MB-EP-Q

Fig. 4. Quiescent States EP - Average packet delivery ratio, varying numberof mesh routers running the algorithm with 95% confidence intervals - 30Mesh Routers total

Figure 4 demonstrates how the network performs whenonly a portion of the routers are running the MB-EP-Q algo-rithm. In this case, the performance is evaluated with respectto packet delivery ratio. The number of routers running thealgorithm varies from zero to thirty, increasing by five eachtime. In all cases, the total number of mesh routers in thenetwork is thirty. The purpose of this experiment is to showthat the proposed scheme can be used in conjunction withexisting unmodified IEEE 802.11 networking equipment. Ataround 10 nodes there is a point where the number of routersrunning the algorithm passes a threshold and the performanceimproves. This is an interesting result because it shows thatnetwork operators can gradually deploy a percentage of theirnetwork with this approach and yield improvements withoutswitching the entire set of equipment at once. Anotherinteresting result is that a mixture of IEEE 802.11 and theproposed MB-EP-Q algorithm performs best in the scenario.

Similarly, Figure 5 shows the results of varying the numberof routers running MB-EP-Q on the average end-to-enddelay. In this case, in terms of delay, all cases using anynumber of routers running the MB-EP-Q algorithm performbetter. But when you compare it with the packet deliveryratio figure above, it can be explained further. In the casewith only 5 routers running the proposed algorithm, thelower delay is due to more packets being lost. However,as the number of routers running the MB-EP-Q algorithmincreases, the improvement in delay can be attributed to thecorrect packets being given preference resulting in higherPDR and lower end-to-end delay at the same time. Again, aninteresting result is that the algorithm performs slightly betterwhen less than all of the nodes run the proposed algorithm.

This co-existence being beneficial compared to a strictMB-EP-Q, leads to a number of working hypotheses. Thefirst is that the IEEE 802.11 DCF nodes are closer tothe network gateway than the MB-EP-Q. It could be theproperties of the two algorithms; 802.11 DCF is known towork well close to the gateway, MB networking is known to

682

0

0.05

0.1

0.15

0.2

0 5 10 15 20 25 30

Ave

rage

End

-to-

end

Del

ay

Mesh Routers Running EP-MB-Q

Fig. 5. Quiescent States EP - Average end-to-end delay, varying numberof mesh routers running the algorithm with 95% confidence intervals - 30Mesh Routers total

give more priorities to nodes farther away from the gateway.Second, this result could be due to the evolutionary algorithmitself. The 802.11 DCF nodes may provide a regularity tothe inputs which the MB-EP-Q nodes can then rely upon.The MB-EP-Q nodes are co-evolutionary, a fitness of anyone node is affected by the other nodes within the system.Each node is only self interested in its fitness, this has thepotential for the creation of arms races where nodes willevolve to maximize their fitness, to the detriment of anothernode, the other node seeking to maximize its fitness willthen change to the detriment of the other’s fitness. Both thenend up lower to what would have been accomplished if theyhad some sense of the overall picture, much like a prisoner’sdilemma. Having 802.11 DCF nodes might solve some ofthese arms races by providing known set actions. Third, itcould be that 802.11 DCF nodes act as a clamp on positivefeedback loops that may exist in the evolving network.

VI. Conclusions and FutureWork

Since MB-EP and MB-EP-Q have similar performance,given the current experiment, MP-EP may be a better choicefor implementation with the particular scenario presented.MB-EP is slightly less complex due to fewer states, predic-tion round and mutation, and lower populations. Howeveras routers become more capable this is less of an issueconsidering the MB-EP-Q algorithm may be more suitablefor rapidly changing scenarios where the best course ofaction is to do nothing. Since the performance of bothended up being similar, we feel that further study into moredynamic scenarios is warranted in the future where the MB-EP-Q algorithm may perform better as expected.

In the second experiment it was shown that it is notrequired to have all of the mesh routers in the networkrunning the MB-EP-Q algorithm to obtain the benefits ofthe improved performance of the mixed-bias approach. Oncea threshold number of routers are running the algorithm,the performance approaches the same performance as if the

entire network was running the algorithm which is a usefulresult for real implementation of the mixed-bias approach inreal networks where cost may prohibit adoption if an entirenetwork is to be modified together.

It is also possible to expand this study in terms ofthe EP parameters to further improve the results. Lastly,implementation into real routing hardware would providefurther insights into how this approach performs under moredynamic network conditions outside of the controlled simu-lation environment.

Acknowledgements

The second author wishes to thank the Natural Sciencesand Engineering Research Council of Canada (NSERC) fortheir support of this work.

References

[1] S. Singh, U. Madhow, and E. M. Belding, “Beyond proportionalfairness: A resource biasing framework for shaping throughput profilesin multihop wireless networks”, in IEEE 27th Conference on ComputerCommunications (INFOCOM), Phoenix, 2008, pp. 2396–2404.

[2] J. B. Ernst and M. K. Denko, “Cross-layer mixed bias schedulingfor wireless mesh networks”, in IEEE International Conference onCommunications (ICC 2010), Capetown, 2010, pp. 1–5.

[3] J. B. Ernst and T. Nkwe, “Adaptive mixed bias resource allocation forwireless mesh networks”, in International Conference on Broadband,Wireless Computing, Communication and Applications (BWCCA),Fukuoka, Japan, 2010, pp. 622–626.

[4] Jason B. Ernst and Joseph Alexander Brown, “An online evolutionaryprogramming method for parameters of wireless networks”, in 2011International Conference on Broadband and Wireless Computing,Communication and Applications (BWCCA), 2011, pp. 515–520.

[5] Hui Zhou, Pingyi Fan, and Jie Li, “Global proportional fair schedulingfor networks with multiple base stations”, IEEE Transactions onVehicular Technology, vol. 60, no. 4, pp. 1867–879, 2011.

[6] Yu Sheng and M.H. MacGregor, “Dynamic resource allocation basedon weighted max-min fairness”, in Ninth Annual CommunicationNetworks and Services Research Conference, 2011, pp. 247–256.

[7] Lawarence J. Fogel, Alvin J. Owens, and Michael J. Walsh, ArtificalInteligence though Simulated Evolution, John Wiley & Sons, NewYork, 1966.

[8] D. B. Fogel and K. Chellapilla, “Revisiting evolutionary program-ming”, in Proceddings of the Society of Photo-Optical InstrumentationEngineers (SPIE), 3390, 2, Mar. 1998, pp. 2–11.

[9] John H. Holland, Adaptation in Natural and Artifical Systems,University of Michigan Press, Ann Arbour, 1975.

[10] D. A. Ashlock, A. Wittrock, and T. Wen, “Training finite stateclassifiers to improve PCR primer design”, in Proceedings of the2002 Congress on Evolutionary Computation, 2002, pp. 13–18.

[11] Salik R. Yadav and Steven M. Corns, “Improved pcr design for mousedna by training finite state machines”, in Proccedings of the 2010IEEE Symposium on Computational Intelligence in Bioinformatics andComputational Biology, 2010, pp. 249–253.

[12] David B. Fogel, “Evolving behaviors in the iterated prisoner’sdilemma”, Evol. Comput., vol. 1, no. 1, pp. 77–97, 1993.

[13] W. Ashlock and D. Ashlock, “Changes in prisoner’s dilemma strategiesover evolutionary time with different population sizes”, in Proceedingsof the 2006 Congress On Evolutionary Computation, 2006, pp. 1001–1008.

[14] Joseph Alexander Brown, “Multiple agent genetic networks for iteratedprisoner’s dilemma”, in SSCI 2011 FOCI - 2011 IEEE Symposium onFoundations of Computational Intelligence, Paris, France, 2011, pp.7–14.

[15] Martin Birgmeier, “Evolutionary programming for the optimizationof trellis-coded modulation schemes”, in Evolutionary Programming,1996, pp. 323–332.

[16] G. H. Mealy, “A method of synthesizing sequential circuts”, BellSystems Technical Journal, vol. 34, pp. 1054–1079, 1955.

683