ARTIFICIAL INTELLIGENCE THROUGH EVOLUTIONARY PROGRAMMING ... · evolutionary programming prediction...
Transcript of ARTIFICIAL INTELLIGENCE THROUGH EVOLUTIONARY PROGRAMMING ... · evolutionary programming prediction...
Research Note 86-86
ARTIFICIAL INTELLIGENCE THROUGH EVOLUTIONARY PROGRAMMING:PREDICTION AND IDENTIFICATION
In Lawrence J. Fogel and David FogelTitan Systems, Inc.
Contracting Officer's RepresentativeGeorge H. Lawrence
DTICELECTE
BASIC RESEARCHMilton S. Katz, Director
U.S. Army
Research Institute for the Behavioral and Social Sciences
AUGUST 1986
Appro.vd for publiC relese; distrobution unlimited.
U. S. ARMY RESEARCH INSTITUTE
FOR THE BEHAVIORAL AND SOCIAL SCIENCES
A Field Operating Agency under the Jurisdiction of the
Deputy Chief of Staff for Personnel
WM. DARRYL HENDERSON
EDGAR M. JOHNSON COL, IN g
Tcchnical Dircctor Commanding
Research accomplished under contract for
the Department of the Army .4
Titan Systems, Inc.
Accesion For
Technical review by DTIC TAB
Steve Kronheim Urannou:ced 0lJ:a.tfication
By ....Di. .i..............-....
Availability Codes
AVsSpecdial or
(DTIC) to comply with regulatory requirements. It has been given no primary distribution other than to DTICaind will be available only through DTIC or other reference services such as the National Technic~al Information .
Service (NTIS). The views. opinions, eand/or findings contained in this report are those of the authorls| endshould not be consiruej as an officia; Ioepatmeni of the Army position, policy, or decision, unless so designatedby other official documentation.
UNCLASSIFIEDSECURITY CLASSIFICATION OF THIS PAGE (Vlther Data Entered)
REPORT DOCUMENTATION PAGE READ INSTRUCTIONSBEFORE COMPLETING FORM
-0 1. REPORT NUMBER 2. GOVT ACCESSION NO. 3. RECIPIENT'S CATALOG NUMBER
ARI Research Note 86-86
4. TITLE (md Subtitle) 5. TYPE OF REPORT & PERIOD COVEREDARTIFICIAL INTELLIGENCE THROUGH EVOLUTIONARY InterimPROGRAMMING: PREDICTION AND IDENTIFICATION February 1986 - April 1986
6. PERFORMING ORG. REPORT NUMBER
7. AUTHOR(s) S. CONTRACT OR GRANT NUMBER(e)
Lawrence J. Fogel and David Fogel PO-9-X56-1102C-l
9. PERFORMING ORGANIZATION NAME AND ADDRESS t0. PROGRAM ELEMENT, PROJECT. TASKAREA & WORK UNIT NUMBERSTitan Systems, Inc.
P.O. Box 12139 2Q161102B74FLa Jolla, California 92037
11. CONTROLLING OFFICE NAME AND ADDRESS 12. REPORT OATEU.S. Army Research Institute for the Behavioral August 1986and Social Sciences. 5001 Eisenhower Avenue 13. NUMBER OF PAGES
Alexandria, Virginia 22333-5600 19114. MONITORING AGENCY NAME & ADDRESS(It different from Controlling Office) IS. SECURITY CLASS. (of this report)
UNCLASSIFIED
lie. DECLASSI FICATION/DOWNGRADINGSCHEDULE
.1%., 16. DISTRIBUTION STATEMENT (of thia Report)
Approved for public release; distribution unlimited.
I-*,
17. DISTRIBUTION STATEMENT (of the abstrt en tered In Block 20, if different from Report)
IS. SUPPLEMENTARY NOTES
Contracting Officer's Representative, George H. Lawrence.
9I. KEY WORDS (Continue on revers. aide If n.ces.sry mid Identify by block number)
evolutionary programming prediction and controlsingle state machines artificial intelligencenoisy environments forecasting
24. ASTRAC" rCmto as roesa ffng'eei.fy mod Identify by block numbow)'This report examines the manner by which evolutionary programming treats anarbitrary prediction problem. Additional experiments were conducted to clarifyuncertainties given increased machine size and the cost/benefit of retainingI"offspring" programs, the impact of noise on the predictive capability of theevolutionary process, and the efficacy of crossover as a mechanism for improvingsimulated evolution. It was also found that some difficult combinatorialproblems such as the classic Traveling Salesman Problem can be addressed throughless complex logics.
DD A 6 1473 EDITION OF I NOV OIS OBSOLETE UNCLASSIFIEDSECU9"TY CLASSIFICATION OF ThIS PAGE (Whm Defa Entered)
'l " / - '- .. ' .. , ; -.' '.-V -'. . -.,'. ,.' . *.," " ' .. / . ..- """" ": -
- .rnvuw n 7W -. -W
I
III
Evolution, it is often said, is the most patientof engineers, redesigning organismspiece-by-piece in the tiniest of increments,using the grand and gradual sweep of geologic
I time as a drafting table.*
*Omni Magazine, March 1986, page 20
Ix
TABLE OF CONTENTS
Page
FOREWORD. .. ..... ........ ........ ... iv
INTRODUCTION .. .. ......... ........ .... 1
DISCUSSION .. .. ........ ........ ....... 2
CONCLUSION. .. ..... ........ ........ .. 136
APPENDIX: Addressing the Traveling Salesman ProblemThrough Adaptation .. ...... ........ 137
jr%
FOREWORD
Military decisions hinge on an ability to forecast the developing
situation. A sufficient understanding of the enemy's response behavior
sets the stage for causing the desired response. Clearly, prediction
and control are essential aspects of the decision process. And yet,
classical techniques treat only a limited portion of such real world
situations. Least mean squared error reduction is rarely the appropriate
criterion. The "plant" or transducer is neither linear nor passive.
The usual approach is to extend the classical techniques through
Iquasi-linear approximations and higher ordered differential functions.
But, such progress leads to ever more complex formulations that are
increasingly unsuitable to primitive computation, and furthermore
requires an in-depth understanding of the physical process which is
not always available.
What is needed is an alternate formulation.., a new beginning
based on a different view of the logical process required for prediction,
control, and other aspects of intelligence. Evolutionary programing
constitutes such an approach... and simple high speed parallel processing
will be notably suitable for such fast time simulation of the evolutionary
process. The task at hand is to devise such programming in support of
military prediction control and decision processes while, at the samer time, to extend our intellectual capability.
I
I, v
1p7
II
i INTRODUCTION
The first Quarterly Progress Report described experiments in theprediction of diverse time series against various payoff functions through
the use of Evolutionary Programming. It is now worthwhile to resolve some
uncertainty concerning the structure of this program and to dimensionalize
the use of this predictive capability. Given an arbitrary prediction
Iproblem, how can one determine the manner in which it should be treatedthrough Evolutionary Programming?I
Some additional experiments were performed to clarify uncertainties
and to prepare for conducting a significant number of larger scale
experiments to be performed on the NASA Ames Cray computer.
I Various authors have suggested the use of crossover as a mechanism
for improving simulated evolution. Some experiments were performed to
explore the worth of this concept. Another series of experiments were
conducted to demonstrate Evolutionary Programming within the context of
I identifying an arbitrary unknown transducer. Some difficult combinatorial
problems such as the classic Traveling Salesman Problem can be addressed
I through the evolution of less complex logics (for example, single state
machines). A demonstration in this regard is contained in Appendix.
I
I
11I%
*.. pa %.~ 'a p ~'.%..t. ~. -*:v-aK~.: K. :...- n
... .' # - - - . ..'.' v -'-" ""'_ -' " '"- -~ ."- ,'- a.'- -' ". -"- " . .
DISCUSSION
Additional Experiments on Prediction
Increased Machine Size and the Saving of Equal-Worth Offspring
In view of the previous experiments, it remains unclear as to
whether or not it is worthwhile to save offspring that are of the same
worth as their parent. It is also of interest to enquire as to the worth
of allowing the evolution of larger finite state machines. The previously
used evolutionary program was altered to save equally worthwhile offspring
and to permit finite state machines of up to fifty states, this in the
hope that an increase in size will improve the predictive capability. In
addition, offspring of equal worth to the parent were also saved.
The first experiment required prediction of the same binary cyclic
environment used in the original experiments (101110011101). As expected,
in the first experiment, a perfect predictor was found, but this logic
was not discovered any faster. Figure I shows the predictive fit, while
Figure 2 indicates the cumulative percent correct prediction. Note that
perfect predictions were always made after the 170th prediction. Figure 3
indicates that adding a state is a beneficial form of mutation. The rate
of increase in the size of the machines follows the probability of adding
a state. Note that as the evolutionary process proceeds, the percentage
of such beneficial mutations (as compared with all possible mutations)Igets smaller, and therefore the evolutionary process "slows down." This
suggests that after a given complexity is reached, it might be appropriate
to include a penalty for that complexity or an increase in the probability
of deleting a state.IThe Prediction of Noisy Environments
Ten experiments were conducted to examine the impact of noise on the
predictive capability of the evolutionary process. The same binary cyclic
environment was corrupted by having each symbol change, this with a
2
QC) 0CN
Co 0
E CD
0 C
U)
0 U)
00
=30
0~ C)x C-
oE W
-0 ) Go
C.)
r 0 04-
CN
-~ or
41
Figure 1
3
"°6co~
E 0
0 0nb4- Co
0
10 U-
4-..
E~u
00
0 C)i V6
0 0
L....'-I. Goo
0
.00
Figure 2
CC,0
0
-j
40
00
-4-
0 0 U
0 C)
o Q)Do
cnq
>1
LO 't w4 oN0 N
Figure 3
&I kk5
probability of one-sixth. Figure 4 shows the best of these experiments
with the predictive fit score being very close to the expected value
for a machine that perfectly fits the underlying cycle. Note that the
excursion of the trace above the dashed line indicates that the evolving
finite state machines were fitting the experienced noise. As shown in
Figure 5, the evolution achieves about 65 percent correct predictions.
The size of the machines grew rapidly to an upper limit to twenty-five
states because of the noise, reference Figure 6.
I Figure 7 shows the worst of the ten experiments. Here the predictive
fit score was about ten percent below what could be expected. After 196
II predictions, there was no evidence that the evolutionary process could
predict better than fifty percent, see Figure 8. Figure 9 again reveals
a rapid increase in the size of the evolving finite state machines.
Figures 10 and 11 indicate the mean and two sigma limits for the ten
il experiments. The mean shows a steady but slow increase (probably to 83 1/3
percent, the highest possible percent correct for this noisy environment).
The two sigma "confidence limits" converge as expected. The mean worth
(average predictive fit) was very close to the maximum expected value
and had relatively narrow confidence limits, see Figures 12 and 13. Note
that if the environment is noisy and an all-or-none payoff function is
imposed, the evolutionary process constructs machines that fit both signal
and noise with equal weight. This is particularly apparent if the
environment is binary.
A brief experiment required the prediction of a noisy four-symbol
environment. Here the noise was quasi-Gaussian... a 67 percent chance
of altering each symbol ± 1 and a 33 percent chance of altering each
symbol ± 2. See Figure 14. A linear payoff function was used to provide
some "incentive" for predicting close to the correct true symbol. This
encourages discovering the signal as opposed to the noise. Figures 15
and 16 indicate the predictive fit and prediction capability generated
through use of this less severe payoff function. Since the four-symbol
environment was known to be simple (a two-state machine would perfectly
predict the cycle), a 0.01 penalty per state was imposed.
6L
00I*
C Ir 0
I C C::)
E c. C) 0 -
C71~
> r,
In tm C).
(9 1 0L E)~0 ) ) 0
00 qOw
4-*
M- 00 0,-W U)
I Figure 4
7
0
0)0
too0 I0 2 C
6~ 0-
0)
q~c 0 *0*
00.0 -o
0
0
a))
Fiur2
0
.4-' 4-
Ecoo K0
00
I C)
too
Figre
CL ) o f
0 C
0
a U) 0L IN
0 (
10 .
I N
I
00
0
•I N N
0 0
! °4- '1J-
I (00 -
CDC0
0 a
W -0
000
0-
C N
Fiur 8
co
o °Lr) 0,--
i0
"-00
_o' Z
coD
Go
0 z
0
0 C) 0 O to4--
Figure 9 "•°
12
I~ 'p
II °
C0N
c 0
1 o~
I o
(00 L
C!)0-
I -4- _ooU
. J
IZ0
IC 0 Q)z
r0- 0
I LOZ
I.-q
C,,
Figure 10
13
00
Cl) N
0 c V0
(S. 0IO-
"s--00
CfCl
O 0 0-~
+Figure 114.
o
0
I!D00
I -CN0
l °D
0
I I.' o .0
I U)0~
I 7 ; .'4--'' " ' "00
I Figure
I C1
. ..I" "" " " , ' " -,'o0., " .'-,- , . " * 0 0. . . ' ' " - - " ••" • "
-I , . - . _ ,. ,,, .. " . :¢ . .., , ,, . , . -. . . . . . . ... , . . . . . .I igr 12-
_00
T- -4-
~~Q)
C)
~~0o L - Qs
0S
oG
00
C'4'
FigureL1
C) E 0 GJLI16
.~*. uUV-W -~to
I0
CC
11
-~ ~ FLY~-X M.? 'W7 .. v~ 77
LC
U) )'
L) to
-D 04o~ cu -0E 0 I
L~~ 0)4
'4--
~> E
U))
UUo m
0~ LO)
to
Figure 15 .
18 .'
a LI)
E5
1 a az0
-4-4J
:E .0EO~ Z
0 0 0
L. w c -,L) Eo Lo 0!E -
V)V
() 0
C)0 0 1 0S- ) C7 c) P D.O '
C., '
Fiur 1-
(\J ~10
b
Saving Equal Offspring with a Penalty for Complexity
Nine experiments were performed on the above-referenced binary cyclic
environment wherein offspring equal in predictive fit to the parent were
saved, and a 0.01 penalty per state was imposed. In one experiment, an
eight-state perfect predictor machine was found on the 63rd prediction,
see Figure 17. 12,603 offspring were evaluated in the 196 predictions.
Figure 18 shows that the machines greatly increase in complexity as
required to "solve this problem." Figures 19 and 20 show that the mean and
I two sigma limits were slightly better than those previously obtained,
reference the first Quarterly Progress Report.
Predictin the Fibonacci Series, Modulo-t0
Having determined that saving offspring of equal worth to the parent
can be of benefit, an experiment was performed to reveal the predictive
capability of the evolutionary program with respect to an environment
generated by the Fibonacci Series, Modulo-10. Figures 21 and 22 indicate
significant improvement over the previously reported results. The
evolutionary process was now able to "learn" more of the 60-symbol long
cycle. After 3,020 predictions, a 42 percent cumulative predictive score
was achieved. In principle, this environment would eventually be perfectly
predicted by a ten-state machine.
Predicting the "I.Q. Test" Environment
Seven experiments were performed on the typical "I.Q. test"
environment 101001000... again saving offspring of equal worth; all other
conditions being the same as those indicated in the first Quarterly Progress
Report. Figures 23 through 29 show these experiments in terms of percent
correct and demonstrate the adaptation with respect to an ever-changing
environment. In the first experiment, the "discovery" that it is better
to predict the zeros was not yet demonstrated because it had successfully
predicted the early one's. The second experiment shows a distinct
I20
-'.~ -r vF~'. r
I °~ 4-j
o) 0
E 0
-0
C) 0 6 0
L--
CD L) ._
0 -"I0 0
0,. ,
0 4-'
(JOD
Figue 17
I 21
• , -, -, - . . 4 -.. . . .', . .. . _ _.. . . .. . . . . .. .. '. :
.o__ _____ N
.- cn
o C-ci 0
000 0- 1 C)
-00p 0
• .- E02 ;0 C : IQ)
a, 0)
-~( D
I CL
II
V' .-.
I Figure 18
1 22
.. .......... ......
o Q)
C:/
4-0-I L0E Q0I XC)4-%4 ..-'0
%0I C: D-c )1 001
I C)
C CN
CD G I- (D t 1- to C
Figure 1I i'-c~) I--, 1 23
CC4-j4
c U) C) 00c (0
0 (D U)x 0 UoW 0 C
_c E *0~ 0) C) 0rt
E L 40~> C)
C: to _
c CL 0
0-i0
0) C 'C'
C 24
crr)
0
E 1
0))I C
0 4!
LI)CCI~
0 0
'.4- 0 0 Q -
>--
A- L-,c
00I
0 0z J
o00)cLL.
Figure 21
01
0D4O z-4
0r E0)
0I
_ _ _ _ ___ V -
oG
-E oi
Figure 22
26
0 0
N 0 )
rZ0 .4 4
co
000
> 0 LO~D *.2
L. ..0C
0 a,
0L_ CDcn M %- w n .0
Fiur 23J I..27
- f' V u WV W V 4-'V'VVV--' 7 - :- -lt K I -T - T TV - WTT pi TT -- .
U)0CL)
0 _ __ _
N) C
'0 0)o 0>
0 0.
'4- U)0000 U') C
00ILL0
I CD
EIr- 0
E 0
(L) cc
CE
ci)00
I-O
ci)
Figure 24
28
-; - W- -Ta -W.L -W
ClC0 Co LO ~
W0
0)jL0 Cl)
CC
0 0 . LO0- 0 ) 0(1
Q) 00) BE
CD 0 C) -4-J
0 0L)
o 0- cQ) C4)
ca E 04- 00
0o0
-4-' 0
0)0
0 co r% c 0 '- I
Figure 2
029
o 0 -
CI) =-4-
00
4- 0
9 1.0
0
CD 0 4) 0
-) 0
E 0
L -
=
L i V
0
ccn 0I~ c 0 );CU).
:3 0
Fi~gure -
03
- ~~~~ 7. -- P -
IL C')___
0)0
0y)
0 v0
-CLL.
CC
0 0 Q)- Lo~~~ C 'i--.0a-
C)C
00 CO
.6 --1- 4 - 1C
0) U'c 0
E. 00 -
, 0
0 0~ 04-J E - O I) q ' ~ -
C- EI (/) :I~L Fiue2
X-31
I~ ~ i&§ ~ J.-'..- 0
0 CC
ci))CL 0
0 C 0 0
4- 0
o 0O i-0 I
E4-0' 0 ,Q) 00
- .-
0 0~ E-
0)0
- 0
LL
oo r o 0 w 0
Figure 2o 02
AveI
Ul) 6. %Ib
0i
0 CL
Cf) -0U)*
0 -5O
0
r) -ch
4-q
4- E o
0 0 UVE
~ 0
Q) &-05 L
:3 aC 0CL) CCU)-)t)
Q-)
Figure 29
33 P
improvement after 220 predictions. Clearly, the evolving finite state
machines now primarily predict the zeros. Figures 30 through 36 show
these seven experiments in terms of the size of the machines. In every
case, there was a rapid build-up to the imposed limit. Conceivably, a
higher limit would have allowed for a greater predictive fit score;
however, in the limit, the results would be the same. As the machines
increase in size, each state would correspond to a given symbol in the
environment. Eventually, there would be a steady state that predicts
only zeros. Figures 37 and 38 show the mean and two sigma limits for
these seven experiments.
Altered Mutation Variation
Ten experiments were performed to examine the worth of altering the
probabilities of mutation in order to improve the effectiveness of the
evolutionary process. Here the same binary cyclic environment was used
with a thirty percent chance of adding a state and ten percent chance of
deleting a state. Figures 39 through 44 indicate the cumulative percent
correct. In these ten experiments, the evolutionary process generated
five perfect predictors. As stated in the previous Progress Report, no
perfect predictors were found in thirty such experiments. Figure 45 -'.
indicates the manner in which the states bulid up in these experiments.
Figures 46 and 47 indicate the mean cumulative percent correct scores and
two sigma limits, respectively. The average number of evaluated offspring
was 3,238.8 with only a slight variance.
Predicting Cycles within Cycles
A "double obverse" environment was constructed consisting of ten
repetitions of the above binary cycle followed by ten repetitions
of 223445, this being followed by another ten cycles of the original
binary sequence. Offspring of equal worth were saved. Five experiments
were conducted demonstrating early learning. The prediction score then
.34 'p£
. . a---a
Ul)
Q) 0 4l
0~ 4.
00 CUCD - i*0. IQ) Co- Ca) 0-
0a 0 0
oc 0) ) C eCO~L c-0 C
0 4-- (1
c
0~ E
E1 - 0 M
Cfl
0~1 4.,
0
U')
E
ciO 0 O t 0Qr) to *
Figure 30
35
51)
0 0
o CLC0
01 00Q0 (1
0-
0 o CL;COcio 0
cn EoC
.I-J 00Q)
C 0
L 0O
0 E:3:
to) to 00
IFigure 31
36
UC)ciC)
00
0 0
N 0- ) CCJ) C) U/
C)C
c Ca)
CoL
4- o )-
0 C C) C .
0 -1
0 C
CLO
ot
-37
- 0 Z
o CfciO)
C
0 o-0-
LO0 r)4 0
0,QQ3 0'
0 0)0 4D
-) c cE- C.,
~ %
4-4-
0 0
r 0
zl 0
0~~~ to to o o
Figure 33
38
.N.
0CLC)C0
N to)
D ) 0
00
U))
0 a)4
U)) 00DL.O
V)o
0'0
c 0 %0 0J U). CN.. 0 -0..
-E o E%r- 0 =3
0"
4-j > E flp4-' -4- V
0' ID 0-4-
0) E
:3:0 LO 0 to' 0 LO)0) to C*4 T
Figure 34
39
'17 -9 7- Id- VVO T -- w -z W... I
es
ci.)(D 0o C
Cr ---o-- U-)
C .0
T- >
0 L
0 0O4--
0 0Ca lCD Q) a)) .2
o~ 0.
0)) EL 0 1-- C
C 0 ch
4-
4 -1 4 - E r.
0 0 =
0 L0
ci.) E
ci I) LO a t
V,
Figure 35
40
II
U')
41 NC CI--
00
I> 0 00
00(
C L at Cla• -5I
.a 4a
0 C)
I 0 0, -U . , )
S .
(Ic)0 0) o O 0 LO 0o
Figure 36
41
-7 -W pIW MV1W -I - %IIMTlWfPtP
I) U'0
N
0 c0
E
I -6 00
00-
X IC)
0 0
Q) 0 0U 0
Q c) 0
IU~)
1 ) 00 r-, to O 4t to 4 0-
I Figure 37I 42
Cl)ci)
Q)E
N LO0)0
NNC.) 0
0
1 0 U)
Q) L
CC\
Cfa)
'443
I0
00
o0
0IEE-_o.2 Wd- C:-
1Q) r") O4 " --Cl) Q~-J (D~ 4 0L
*C, 0 +. 0 0 .
C:) _no-- o G
*- 0 a) C:L*M oQ) 0
C 0o) c:-)oc
C.))
0 ~~~0 0~~'
00I0
II
0-4-' 0o 0I (N
o 0c -:
-40 0 K
E 0 U)E c
Joo
0- L-4
I 0)
00 0
to
CY, 0 d
Figue -
0 05
00
-4-' 0
o 0
:3:0 v1 I*- 40 b
L o'
E~ 0 -z
a)U Q)~vb
0 (9 0 a)Q ) J-f- (1) u a
0)- (1 -.. > 00
E4- 0)c )Q
-4. - - () 0
C C)
o 0NL.U
0 C)nN C
Figure 41
46
000V-
C14
c0
1,, 0
I l 5-
0 0 0
I Qc) V
I " 0
I 0""
I "- '.
Figure 42.'
47 -
.C.)
4I.
0.4-j
0C 0N
o Lrw
->- 0
o0(
-44-j
D C)C4-
0o 0a)W
oC 0 0 Q~*
0 0
CC
-448
oC
4-a4
,4
0~ tz'~*- c
0o 00
C:)Go .. =
E~
oo Z
oq CRLo2
Figure 44
49
0
0 .1
o0
o0(
00
LO ) c -0.4 0
Q) (L) .- n
L-- -4jV ~ :
CD0 - 5 0 -,00 (nOZ)o
c. E0.
0 V2C 3 C__
.jI (1 0 ) O
0 I.) 0If) 0 If) o
0))V
Fiur 45
05
0
> °0
0 C
0
EII
CC0 0-
CN-SL_
I0 z :CI 0
0.
L-QL- . .,0 E
00
0 C
Figure 46 4I .. a51
O
40
0 0
CC'
0
23
•0 Ei o
.4J4a
o V
Co
0 00
l Figure 47
52
NUN IA4
I
Ireflected difficulty in learning the second cyclic component of theenvironment, see Figures 48 through 52. Saving equally worthwhile off-
Ispring leads to a rapid increase in machine complexity, see Figures 53through 57. It is believed that the added complexity that results from
saving offspring of equal worth acts as a buffer to the coding that
predicts the original environment. As expected, therefore, when the
original environment was re-introduced, the evolutionary process quickly
redemonstrated its previous ability. Again, note that an increase in size
can markedly slow down the evolutionary process when it encounteres a newI-environment. Also note that the initial learning of the first sequence
was slower than previously demonstrated. This was due to the fact that the
evolutionary process "believed" it was dealing with a six-symbol envir-
onment instead of merely a two-symbol environment (the initial machine
I was experssed in a six-symbol language).
Similar experiments were also conducted wherein the cycles were allwithin the same four-symbol language. Here the sequence 0132 was repeated
fifteen times followed by 331022 repeated fifteen times with a return to
the original sequence repeated fifteen times. In the first experiment,
only offspring that were superior to their parents in predictive fit were
saved. Here the evolutionary process discovered a perfect predictor
of the first cyclic sequence at the 20th prediction, see Figure 58. 348
offspring were evolved to yield a four-state machine shown in Figure 59.
Note that only a single state machine is necessary to predict this firstsequence. After the environment changed at the 51 t prediction, the
evolutionary process showed a sharp drop in success but regained to a
- prediction capability of five correct out of the six symbols. At the 14 1st
prediction, the original environment returned, and a perfect predictor
was still evident. 200 predictions evaluated 6,585 offspring. Throughout
I the experiment, the machines remained of small size (three to four states)
because here only offspring superior to the parent were saved.
IIn the second experiment, the offspring were saved if they scored
equal to or better than the parent. Here again, the evolution indicated
53
Im
E~ 0
0 0- - - - - - - - - - C
> 0
lo
0 jC%JU> t C
10 0 .0
00 C:)
0 1% c EU') 0
0 cl0
C4J
N4. -4 a
~~0
J 0
Figure 48
54
r 0
- _ 0
0CI II"4T N
CC
0 N
o C
0 :-
C)
CD 0
C) S-00
N (1
co 0
:3 C)
I Figure 49
I5
I U)
1 C:
000CDIN
01 00 igoo
I U) K00 W.
0(
>) ..-
.0
0''
IQ0 a (01 '
Figure 50
56 .
1ini
_ _ 0
c C4
C)~
4-'
> loo of-4 Cl)
C-4-
*00
C C)0 - CL
ICC
00t
ClC0
Figure 51
57 5
IU
0 0D
IL qJ N
0> C))
0~' NL.
LC C)
cn0 C:)
x 0 _
C,4,
0n o 0 I)
I Figre 5
x5>I0-% 0Gq~ '% : % . I '~.~ . .~ * *. . * ~ **0.
0.
U) 0
_ 0
0D IIq0 I N
Occeq 0-:
(01)00
0 >8010
-00-
C) i.~-.C 0
C)C
L. J
0 o 00t oC,
Figure 52
59'4
C/i) 0
0o4- CojCIN
CfllC)
0 CC)
04
,-.. IC)C)
C~4CO
C _)
/3;
0 LO inLC 0 0OCN 0 "
Figure 54
60
U) 0
.0
cL0 C'4
Oo0- ~0
0 0N00(1) 19
0I V) C:)v- 1 0
o
_0
-A
U) 0 .
>20
0 0 o0
o
Figure 55
61
4--J
CL, Cl
0 10
OO6 0
0
L) 0 C
Q) x 2:
U) #- - C
0 T- o+)
E 1N
** CO 0 Lto jo CN
II
Co 0
Io 10~OGo
C)M
'0
j) C4- 04
0 C
~-0
0f 0) 10 L_0
00 z0 1
000
T-' 10%
o - 0 .. 0 .00LC0 -
u~~j Go
>iur 57o -W 63
U)1
L-
OLo
0-
0o
~LO (D
o 5'
0A
00
LL-I
leFiue5
E6
C4,
.%
Figure 5
65A
1 superior learning ability perfectly predicting the cyclic environment on
only the eighth prediction (having seen only four cycles) see Figure 60.
However, the evolutionary process had great difficulty in learning the
second environment (due to the added complexity gained while learning
II the first environment). When the initial environment returned, a perfect
predictor was still in evidence. The size of the finite state machines
quickly grew to the upper limit of twenty-five states. This experiment
tends to confirm the notion that large machines slow down the evolutionary
process. It is useful to compare the above results to the classical
zeroth order prediction of the same environment, reference Figure 61. Note
its extremely poor performance.
Crossover Experiments
J.H. Holland, in his book, Adaptation in Natural and Artificial
j Systems, 1975, proposed the use of genetic.operators. He believed that
simulated evolution could be improved by drawing an anology to sexual
reproduction. Specifically, he suggested that two machines be "crossed-
over" through a substitution of states. A large number of experiments have
been reported by Holland and others on the use of this technique...
without their being any reference to finite state machines as being the
embodiment of the evolving behavior.
Several experiments were therefore conducted to investigate the
worth of this proposition. The first fifteen experiments involved the
same binary cyclic environment and used a two-state crossover, that is,
I: two arbitrary states of the best machine was substituted for two states
in another machine of similar size. The probability of this crossover
was chosen to be fifty percent with the remaining probabilities uniformly
distributed over the five modes of mutation. Figures 62 through 67
show that the best experiment discovered a perfect predictor at the 63rd
i prediction. 2,976 offspring were evaluted, and the size of the evolving
machines rose steadily. The worst experiment did not discover a perfect
predictor in 196 predictions. 3,018 offspring were evaluated. The
I 66
I
'- - -U ME~M ff -.- ~ VRW~ m z~mw TWx w- 'RI -KV Wj - 7r -i V.Zu,. -S KV %v ;,,w-.-
Zx C0c\4
Q)CL c
'4-0 C)00
o 0
(1 0
V) 0J 04--
C- 440 0
T> N 6co 0
Qo 10 ojjU U) K
0C 1
fn 4 0
CC)U)C)
Figure 60
67
4-j C
Ix
EI 0
0-
0 10 CL0 0.o U
1x -o
00
0 L~.
I E.
D U3
(I) gur 61n~j& 18
CNN
00
o CNI 06C&n0 to>
= -W 0 DC
CI L- 0
00
'3)C)L4-
0 -C
04- 0 D)
al)
C,,
0
E) C)
E C
000 0 0 0 0 0 0 0C'0 OY) 00 rN* C0 U) 14- 10 cq
Figure 6269
-4- .__ ___0_
0
00
00 0x
'4-E= 0
0
0N 0
F i u r 63 LA -
700ca,
Ic
0Aa,* t% %~. ~ ~ -~ " ~w . .~ .E
a
0) 0,
0 0
>1 04
C..) CLie%
00
0 CDo Q.0
00
c 0 0 00000000in- w 0-
Fiue6
71E
C) 0CU
0 0
0- oo 0CU -4-
0 0)0 E -W
Q) 0J 0L1 0 I 0 VQ) >I 0 .
o ) 0 9-
c 0
- D E cn.c n = 0
0
0) L)
> E(U) CD Zq-i
00
0 0
.. .. .. .. 00 0 0 0 0 0 0 0 0 0 0
Figure 65
72
c*Iq
0
Q) 0
E VU 0o U L. C') CD
x ~~i .0 .oL o 0- 4) ) V
x x r
U - r 0 0
E mQ~z- 0 CU L
00
Q) 0
0- 0 L-
0)Q
>) E
o0
C) 00 1*a)LO q, n C
Fiur 6
E 073
U 00%1IU
0~0
0)
o10 0~0 :0
0 0
Fiur 67M 74
increase in machine size was similar to that previously demonstrated.
While the mean worth (predictive fit) of the process consistently increased
(Figures 68 and 69) the mean percent correct of future predictions was
almost identical to that without crossover; see Figures 70 and 71. On
the average, 3,001.2 offspring were evaluated in 196 predictions.
Crossover was also examined within the context of the I.Q. test
environment. Twenty experiments were conducted, each performing 85
predictions with an all-or-none payoff function. Figures 72 and 73 indiate
the mean cumulative percent correct and the two sigma limits. After the 40th
prediction, the process properly predicts all the zeros, although with a
significant variance. However, Figures 74 and 75 show that the predictive
fit variation around the mean is very narrow. All machines tend to fit
the previous history in the same manner. There was little difference
in the results with and without crossover. ,
The environment was then predicted using the asymmetric payoff function.
Ipredicted0 iiI ____
actual 0 3 0
1 0 10
Figures 76 through 79 show the result of eighteen experiments. The mean
worth of the abilities to fit previous history was slightly worse than that
without any crossover. The poorest results asymptotically fell toward
the expected value of the environment. The experiment with the best
results showed some ability to predict the early one's.
I75
L
C,,)
00
4-j4
00 x
UJ~ 0 Co (0
o 9-4-a4-, V2
0
L.0
100.40 U4
Q)
E(fl . .- D(
CDo0: mn 0 ,, 0 O ", t
Figue 0(~) 070
U.). %,
*%Y -wVIWVqM WI
C-) -j.
4- C
C~ I-
0 0
CL1 CN
V) D
0q ......... 1
:Fiur 69
Q) 1..77
.1>1
otx 0
0 D0)L
> 00 UD
E C.)
Cl) C- C)0 0 0
n C) m0 -0DC
Fiu) 7_ C78
0
00
00
0~
-4-, 0
00 E3
0.. 0U
U) E)0 00 m6o 0 S
4I -4'
L CLo) C C
Figure 71
79
0
o D0
o C0(I)
_ (0o--0
LL.
0 0 Qg
L- - 0 )0 4- .
1) 0
L- 0C) L)J
-~80
Elot
'./
0-
"on
oL 0
0
En .E_o 8 ...
0 0
C-C
CoL
Figure 73 S ,
01%L.81
0
r- c-,
C!):
0
(L) x U)
0 JU) Lfl
U) 0)U
0~0
C)00
CL)
ci TO
aFiur 74-c80
I
IGo
0
I- o _ ______00
LO,
oO
00
CLo
I 110
I. 0
I,
64.
I
Figure 75
83
I "2
0-0 (
n c - U
.4-" CL4-S-
4- a -
0~ 0 :Ecf) I ) 0 '4
(n
4I Cm
E 0
o) al~C ~'4 )~~
I Figure 76
84
.I0p
o
-00.
0)0r-
-%
0 c
o 0
Ile.
Figure L
1.~.85
o.
00C) 4
0.
U)
Q))Qo C
o p
0 (D
0
00
oo w*C.
3r.-
4-4 c
0. . .. .0.. .... .
C) c 00 -, t Ln t n.
Fiur 7
0(386
(f)(I)
o CEC44-
C) (1 '4-0
(a)D
0 VI)
0.- ) o
0(
oLn0
0
00
0
0 0
(0 N C0CL)0 0 M No W- co ut n N 0-
Figure 79
87
As pointed out by Dr. Wirt Atmar, Holland's crossover technique
does not resemble the manner in which recombination occurs in nature.
Sexuality in natural organisms results in the recombination of alleles.
Alleles are defined as one of two or more alternative genes that control
the same characteristic and occupy the same place on similar chromosomes.
For Holland's technique to be effective, it must be assumed that a finite
state machine is analogous to a chromosome and that each state output
and state transition is analogous to a gene. But, this is not correct.
Genes contain the instructions for generating specific physical functions.
They are much like subroutines. Here a finite state machine could be
thought of as a gene. However, the structure of a finite state machine
could never be thought of as a gene.
As pointed out in Elements of Biology by Charles K. Levy, Addison
Wesley Publisher, Reading, Massachusetts, 1982, changes by crossover are
not true mutations "since neither the amount nor the function of genetic
material is altered." Under Holland's crossover, while the amount of coding
is not altered, the function of the coding is greatly altered. The result
of this is a near-random search throughout the entire space of possible
solutions. As the size of the machines grow larger, the number of
I states used in corssover increases. As shown in Figures 80 through 85,
when predicting the cyclic environment 012345676532210, without loss of
generality, the evolutionary processes which do not use crossover perform
significantly better than those using two-state and ten-state crossover.
Holland's technique effectively destroys the link between parent and
offspring that is necessary for an evolving process to succeed. In the
words of Dr. Atmar, "While his original thought was in the right direction,
the technique he promotes is at too severe a scale. True sexuality does
not destroy (nor create) information; it only shuffles contending subroutines,
eventually trying almost every probable combination of those in existence,
retaining only those combinations found to be most beneficial.
Even at this far milder level of informational reorganization, evaluating
I the "costs" of sexual recombination in producing inappropriately behaving
1 88 5
.5
o 0
0-
V') 1 O
OD. r- L...0
Fiue8891
VV
C,)
I 0
0 U -
CLCx~1
G) cc
CD E
(.0 0NE
(0 a%C) V001
2 cn- 00t- D LO 1 n
090
* l-
U0
Go
(00
ro
(00
01
9~ (n
I
0 CL
Q)I .4,re-8
'I I'-~ * ~~91
05 0
r-I-
0OW In
c% 0
(.0
1 0
0~ (3)
(D 1,0C-
5 0) 0 W - 0M
FigureL8
v92
to C:
toc EnC) q6*
I- LI) 0IJ-
0 0
% ol4) 0
IE E
U) )
cn U) )1
0- 0
I C-L
CY))
0) OD N -O L)r 4 0755 *ID
Figure 84
93 4
0
LO)(D
1O c' 00 10
o~% ofoI -
4) co40 .
0 0
0 z
0)0I0U 0 --
C 0
070 0 N a O 10 0I 0 V0
I Figure 8594a.
organisms has been a topic of substantial debate which has continued on
in biology for some time now. The debate is currently being resolved
in the following manner: the costs of sexual recombination are no longer
considered severe because (1) the contending alleles (subroutines) have
generally been found to be quite similar, and (2) there are not nearly
as manny alleles contending for each site as was originally thought or
predicted.
Sexuality is not a mutational phenomenon. It is, rather, a method
of rapidly sorting through the effects of mutated subroutines. The
advantages of sexuality in an evolutionary optimization program are not
to be underestimated. However, a very specific structure must be in
place before these advantages are obtainable."
Identification Experiments
As previously indicated, prediction is key to the process of
identification. Suppose an unknown entity is responding to known stimuli
in a clearly observable manner. The task is to develop an explicit repre-
sentation for that transduction, this on the basis of the stimulus/response
experienced to date. The predictor observes the sequence of stimulus/
response pairs, the most recent stimulus, then predicts the next response...
this process being repeated as each new stimulus is observed. The
predictor must develop a most suitable logic in terms of the specific
payoff function and span of prediction. If this span is the single time
unit, the discovered predictive regularity should correspond with the
underlying logic of the transducer. Evolutionary programming provides a
basis for such prediction and identification.
To demonstrate this hypothesis, a series of pilot experiments were
conducted. Success in this regard might justify more formidable experi-
ments on the Cray computer. As a first step, the transducer or plant is
95
.4".'I
deterministic and fully controllable, that is, each of the possible
response symbols is generated by each of the states. Plants of increasing
complexity in terms of alphabet size and number of states were used. Once
a searching prediction score has been obtained, the predictive logic is
compared to the actual transduction as a test of the identification. In
these experiments, an all-or-none payoff function was used in that all
symbols are equally important, and no credit was to be given for nearness.
To ensure adequate exploration, random input sequences were used to drive
the plant.
The first experiment required the identification of a two-symbol,
two-state finite state machine, see Figure 86. The evolutionary process
properly identified this finite state machine at the second prediction,
this after evaluating only thirty offspring.
The next experiment required identification of a two-symbol, five-state
machine, see Figure 87. Here, in 254 predictions, the evolutionary process
correctly predicted 80 percent of the responses and had a predictive fit
score of 90 percent, see Figures 88 and 89. In all, 32,376 offspring were
evaluated. Figure 90 indicates the final evolved machine consisting of
six states. It is tempting to compare this machine with the one shown in
Figure 87, but this is dangerous. The addition of even a single state
1changes the meaning of all other states. There can be no simplistic state-
to-state comparison. Note that a fully controllable finite state machine
driven by random input in a binary alphabet has a fifty percent chance of
responding with either of the alphabet symbols. The predictive score
should be fifty percent. Clearly, the evolutionary process does
significantly better.
In the next experiment, a two-symbol, ten-state machine was the
unknown transducer, see Figure 91. Here, the evolutionary process was able
to achieve a high predictive fit score, see Figure 92, and predicted
extremely well, see Figure 93. The final machine was of fifteen states,I
I 96-V'
.. qf ly 't € ". . ... ' . " . . .. , . .... -,-.€ € ,- . -. .
JR
o co
~~Figure 86 "
I 97
I
IIiI
I I.r
an)
IIIi
I-o
Figure 87
98
*,j
Cq.
0
0 41
10
Go3
1 04-
1 .99
100
0
0
co 04) cN
0 0Cf
LU
I 0D
1.2 - OL) * 0
Fiur 89010ol
TI-Tv gIlrprp-.M rwt
ICIcI w
I0I0
C)C
Fiur 90
10
.
0#'0
o
0 CD
0
N
{ ..-.
f,-'-.
I5- V.,
o 5**.L,
-S-
- k ;..-.
.-,...,," ,4' • . .. 4' * " ',, '.- ,,.'- -',i '.. ..
"*. °-" ,, " ., . .'. ... -,.,-.'w,-. " " " "''' ''" " J' . ' "" " " "1 .'"" .
-0'
cI4.J 0
U C)
"' 0 N ,
o 4-. 4 0
Q)0 L)
1.-
4... . L
00.4-- 0.. 9-
0o~
0 0
r- -- 0
Figure 92
103
Uf)
0
Cr)
0'44 0 0.r
U) %o o C0
0L _o LL. v- : ::
2 < 0 o
0 0 0 0 0 0 0
wx
Figure 93 ,104
" 00 0
while evaluating 8,040 offspring in the 290 predictions. Note that
increased complexity of behavior may not be simply determined by the number
of states. Here the plant has a relatively simple structure which was
evidently discovered by the evolutionary programming. Presumably, further
evolution would finally discover the computer logic of this machine.
Next, a four-symbol alphabet was used driving a five-state machine
that represented the unknown transducer, see Figure 94. Four experiments
were conducted; the percentage correct prediction ranges from 36 to 50, but
in every case, this is better than the expected 25 percent prediction byzeroth order statistics. Figures 95 to 101 show the predictive fit score
and percent correct for each of these experiments.
An eight state machine was then used as the plant, see Figure 102.
Four experiments were conducted. Even though the size of the machine had
been increased, the evolutionary process was still able to predict 30 to 40
percent correct, see Figures 103 to 106. It is also of interest to
examine the predictive fit score for these experiments, see Figures 107 to 110.
The 60 to 70 percent worth indicates that if the evolutionary process had
been given more time between predictions, superior response prediction
could hve been achieved. In each of these experiments, there was a rapid
increase in the size of the machines, see Figures 111 to 114, even though
equally valued offspring were not being saved. An average of 11,422.5
offspring were evaluated in making the 290 predictions. The final predic-
tive machine of the first experiment consisted of ten states, see Figure 115.J
Finally, two experiments were conducted using an eight-symbol,
five-state machine as the plant, see Figure 116. Figures 117 and 118
indicate the precent correct predictions to be 20 to 25 percent, this being
twice the level that would be expected on the basis of a zeroth order
prediction (12.5%). Figures 119 and 120 indicate the predictive fit score
for these two experiments. The results range from 45 to 50 percent worth.
Evidently, five generations per prediction was insufficient. Greaterexploration before predictions would have increased the predictive fit
105
I
ICIL
I%
NO
C%
Figure 94
I N10-
0 ~0
Ct) 0NL. d
UO c0 4
(A) N0 gel
IIN 0,-L 0
000CC
0 c In0~ E4
00
0 0 50
0
t00
0 Q0
04 0
Cf) ULr) 0
0- Z-
1 0 U,
0N
(fc0 asC * t n W
~10
-4C ) 01
00
CLC
U- N 0
-~ 0
W C:
.00
Qo r- D n v 9
10
:4,
: %
L-V
V)Q)
00
UL) C N4
01 00
I.f)
00
-CCq 0ogo
0 CLJ
0 .p.
0 0
'4-1
Coco' I 000
(7) 00 P..a n V
Figure 98
110 p.
. 1,
00
oI o -
00
) .
og
r,'.
8 ,
ro
7-30)
LLLa
. ....N-EN
S-O) 90 t*. (D ,) 1 .-
Figure gg99;
';.
i ) _
i0
I 0
C CDU)C0
I CL0
ul)_ S)-
U) N
00.Q
0
L..
, , , " , , R"- , - , , " . . - . . . •, ... , W .
Ig e1
1.112
4. 0
(0 0E~
C-C
0N
0 Q O 00.
:11
C14
cliv
V4 S-
044
Cl LA-
CIj %--a
114k
PS
!
i CT)
.000 U, 0
,) . IN
¢N
4--
I Ia0 (n
q, c
I C-
"o b
00'
0>0 )I: LU)
pE
I3 0 .0)
*LI
0
o 00
00
0 LO n N
Figure 103
115
(j 0
0 e0
0 03
C) 0
I N
Fiue 0
11
Q) _____
-4--i 0
00
00 U
4- NL!- -
q)):w4z
L
EE
0 >
0 03
a.
Figure 105
117
, s . 6. J ' LL- Ic W Wn~~ V JWLW ~ W.qjW ~
Q)-
0
i2 0
I N > _
C/)C. 00
I -- 0
0..gI!
0 DL Q
0 5,f
C Ll-
I)
118'
07 . 1:-U) 0
4-'
o b_ -
-0-
00
Q11N
0I D in qt W, 04
Figur 107119 P
00) 0
C,
0 C0
N 0
WOCC)
00
C)f
0 000
QI I c.. ..
I ) IX) Cl o t) 1* ?) N - 0
Figure 108120
C)
4-) 0
SC0 CN
0 C)
C) oL --
In -C -) 0
fL- 0 ON
01 LO c
0 1"4
Cl) -0.
I EC ~z
0 000
I22
Q)0-4--J'
0 00 1*IU) -0
00 00C
0oc
N 0
I - i
I-Co
-4--'4-
(1) 0
0 -0
I Figure 110I 122
Q!)
-0-- 0
0l 0 n
o 0o
-5
0)
0LLO-00I :3
0 >%
4-L
o 0o
@12
U)
0 Jo:3
CL)rVW C 1
-0A W 0
4- E,
o00
4-j-
0n
4U-nEE
0~U)
in L.
TJ N
Figure 112 '
124 '
00
O0 3
oo
U) L
00 ,0 -1"0>2:
L- n
00 n - z%
-4--'a z0 "0
00
0 vE*0 0 )-
Figure 113
125
;..~*, p.' ~ . -55*~* I0
V ~ ~ o• l . ,. : .,.¢ . , . ., '. .. ., . 'W .'r . . . a.. % . . t ,.j'. ... . .*.
, ,- ,,.., . ."
ci) 0
00
(f) 2 04c
nl0 oL C>: 0
CI- UI O-C
-00
I U) EC L. :3
0 WU--
0000
FiueI1I 126
w - '. -- I. - . . -v -
I-II4 I.
-CI
00
C144
IL-
12
- I
1C 4
%D -7N
C-4I
IIIr4
I Figure 116
[I128
4-Q) c
) C)
0! ccCLLUL
0~~ 5QC* a
C- E.
-
FC) 'U(L) Jo
I0
0 ~ ~ 0 r0 0 u0 0 r)c
Figure 117
ii129
U) 4LC) ___ _0
0
Co to
01-L4. A.s'0L-& 1 "A&
0 0 0 0 0 -0
C/Do 0 w -)
1130
C) _ _
Co*
Cfl)
Lu--0
LO 4-
U))
( -) C 0
3+J
a Figure 119131
1O 0 _ __
LL.7
IZ~ CO 500LC) L
I- LA--
E 0'
(IDCT
UJ 0J
I Cl) o
-o ~ ~ 0 Go r- Coi 0~ q, n N 0
I Figure 120
1 132
worth and the percent correct. Once again, there was a steady increase in
the size of the evolving machine, see Figures 121 and 122. The final
machines range from 16 to 18 states.
These experiments indicate successful identification through
evolutionary programming. The logic of the unknown transducer was
represented by finite state machines of greater size than required. This
is especially true with a larger alphabet. However, no penalty for
complexity was invoked. It should be possible to evolve machines of
similar size to the unknown plant, but there might be some cost for this
decreased specificity.
On the basis of these results, it seems worthwhile to explore the
identification of time varying and noisy plants through pilot experiments,then design of experiments for predictive identification using the
Cray computer.
III "
I,
133
Ii
133 e 'S.
U) C,
021
C!)D
L0 0 '-(n
-0>0E 10U
E
0440
4-3
0 -0In 0
C-a _
0 lo CL
~00.
0: JN 0$
II
I 0
1 -o0 10 0 O
I Figure 122
135
CONCLUSION
Evolutionary Programming has now been demonstrated as a versatile
means for the prediction of nonstationary time series and the identification
of an unknown plant. It is fully recognized that much remains to be done
to dimensionalize the program as a function of the presented problem in
both prediction and identification. Arrangements are underway to use the
NASA Ames Cray computer for such work. In the mean time, further pilot
experiments are planned to guide these larger-scale demonstrations.
It is now considered appropriate to explore the application of
prediction and identification to real world problems in a preliminary
manner. Medical, financial and other data sources are being reviewed in
this regard. It is of particular interest to focus attention on situa-
tions wherein the least mean squared payoff function is notably inappropriate.
This is surely the case for epidemiologic and economic time series where
equally correct predictions are not of equal worth, and the cost of a"cry wolf" error is clearly different from the cost of a missed catastrophe.
Plans are being made to move from identification to control through a
series of pilot experiments. Success in this regard might well influence
the design of weapon systems.
II:
I 136-9
I.
II%
I.
I
e[ AppendixI!
AddressingThe Traveling Salesman Problem
Through Adaptation
13
[II
iI1= 137
I ,
138
ADDRESSING
THE TRAVELING SALESMAN PROBLEM
THROUGH ADAPTATION
By David Fogel
INTRODUCTION
The traveling salesman problem has received a great deal of attention
In recent years. The task Is to arrange a tour or n cities such that each
city is visited only once and the length of the tour (or some other cost
function) Is minimized. Here Is a simple yet recalcitrant combinatorial
optimization problem. For an exact solution the only known algorithms
require the number or steps to grow at least exponentially with the
number or elements in the problem. Brute force finding of the shortest
path by which a traveling salesman can complete a tour or n cities
requires compiling a list of (n- )1/2 alternative tours...a number that
grows faster than any finite power of n. The task quickly becomes
unmanageable.
Two recent papers1 addressed the traveling salesman problem
1- "Proedlngs of an International Conference on Genetic Algorithms end Their ,Applications," John J. Grefenstette, Editor, Carnegie-Mellon University, July 1985.
I A e
I
- ,;' , ,- < . , . , , ,, , , ... . < ., . ,...... , , . ,.. -,.,. ., .. , ...- .... ... '
139 "
through use of the genetic algorithm as proposed by J.H. Holland In 1975.2 ,
This algorithm Is an offshoot of the evolutionary programming concept
offered by L.J. Fogel In 1962,3 then demonstrated In his doctoral
dissertation4 and described In the book Artificial Intelligence through
Simulated Evolution.5 Here, Intelligent behavior was viewed as requiring
prediction of an environment then the use of such predictions for the sake
of Its control (at least to the extent possible). The logical process of
iterative mutation and selection of behavior Is simulated In fast time to
eventually evolve a logic most suitable for resolving the given problem.
The behavior of each artificial organism Is portrayed by a finite state
machine.., a mathematical function that does not constrain the represented
transduction. It need not be linear, passive, or without hysteresis. The
original 'machine" (an arbitrary logic or a 'hint") Is measured In Its ability
to predict each next event In Its "experience' with respect to whatever
payoff function has been prescribed. An offspring is now created through
random mutation of this "parent" machine. It Is scored in a similar manner
to the parent In predictive ability. If the parent Is better than Its
offspring, the parent Is used to generate another offspring. If, however,
the offspring Is better than the parent, the offspring becomes the new
parent. This assures non-regressive evolution.
2- Afttntion in Natural und Arttficial Systems, J.H. Holland, University of MichiganPress.Ann Arbor, 1975.
p.'.
3- "Autoomos AutomrAa," L.J. Fogel, Indusfrlal Rewrch, February 1962.
4- 00n t Organlztion of Intellect," L.J. Fogel, Ph.D. Dissertation, U.C.LA. 1964.
5- Wtlfi&Sa IntellwnM thro Smulaed Evolution,, LA Fogel, J. Owens, M.J. Walsh, JohnWiley& Sons, Now York, 1966.
2
140
An actual prediction Is made when the predictive fit score demon-
strates that a sufficient level of credibility has been achieved. The
surviving machine generates a prediction, Indicates the logic of this
prediction and becomes the progenitor for the next sequence of progeny,
this In preparation for the next prediction. In this way, randomness is
selectively incorporated Into the surviving logic. The sequence of
predictor machines constitutes phyletic learning... idcv
generalization.., generation of new hypotheses concerning the relevant
regularities found within the experienced environment, this In the light of
the given payoff function. Prediction can be used for the purpose of
Identifying an unknown transducer. Feedback of the evolved model then
forms a basis for control.
Rather than describe each organism only in terms of Its behavior,
Holland chose to evolve the code which generates such organisms.
According to D.H. Ackley,6 Holland's genetic algorithms search a parameter
space where "any point in the parameter space can be represented as an n
bit vector. The technique manipulates a set of such vectors to record
Information" about the parameter space. "There are two primary operations
applied to the population by a genetic algorithm. Bacrduclion changes the
contents of the population by adding copies of genotypes with
above-average figures of merit. In addition to this...It Is necessary to
generate new, untested genotypes and add them to the population, else the
population will simply converge on the best one it started with. Crsoe
Is the primary mean of generating plausible new genotypes for addition to
the population."
6- 6A DornectionstAlgorithm for Gerstic Seerch," D. Ackley, in Procemo/ngs of wnInternmionol Confaranoc an ntic Aprfthms and Ther Appl/cutions ,
J.J. Orefenstett, Editw, Crnegle-Mellon University, Juy 1985.
3
" - ,- . . ..'.. .'...--. ... - . .
141 'n
As defined by Holland, crossover takes two structures,
Al=al lal2...aln and A2 = a2 1a2 2...a2n, and at a random point x between
I and n, exchanges the set of attributes to the right of this position
yielding offspring of the form: A* -a I I a 12...a I xa2(x* 1 )..a2 n Ackley
continues to state: "This 'offspring' is added to the population, displacing
some other genotype according to various criteria where it has the
opportunity to flourish or perish depending on its fitness." "Iutatil
provides a chance for any allele to be changed to another randomly chosen
value.* "if the mutation rate is too low, possibly critical alleles missing
from the initial population will have only a small change of getting...into
the population. However, if the probability of a mutation is not low enough,
Information... will be steadily lost to random noise.' $,
Holland likened the actual code being mutated to that of the genetic
code that defines a given organism. While Fogel, et al. only used small
degrees of "background" mutation, Holland examined the operations of gene
crossover" and "inversion" among other actual biologic genetic
recombinations. Although Holland's work went largely unnoticed for some -
time, today a great deal of attention is being given to genetic algorithms.
At the above referenced conference D. Goldberg and R. Lingle, Jr,7
offered several observations:
1) Simple genetic algorithms work well in problems which can be
7- "Alleles, Loci, and the Traveling Selesmm Problem," D.E. Ooldberg, R. Lingle, Jr., InProVnW ofan I,ntrnah'onl Conforsnc on Onwfi Algorithms and heir
Applicotons ,. J. Orefwtette, Editor, Carnagie-Mellon University, July 1985.
AI,
142
coded so the underlying building blocks (highly fit, short defining
length schemata) lead to Improved performance.
2) There are problems (more properly codings for problems) that are
GA-hard -- difficult for the normal reproduction + crossover +
mutation processes of the simple genetic algorithm.
3) Inversion is the conventional answer when genetic algorithmists
are asked how they Intend to find good string ordering, but Inversion
has never done much in empirical studies to date.
4) Despite numerous rumored attempts, the traveling salesman
problem has not succumbed to genetic algorithm-like solution."
The authors then suggested a new type of crossover operator,
partially-mapped crossover (PM)," that would lead to a more efficient
solution of the traveling salesman problem.
Specifically, consider two possible codings of a tour of eight cities,
A, and A2 , a return to the Initial city being Implicit:
A,:3 5 1 2 7 6 8 4
A2:I 8 5 4 3 6 2 7
PMX would proceed as follows: Two positions are determined randomly
along the AI coding. The actual cities located between these positions
along A1 are exchanged with the cities located between the same
C ,-(.9-..9*9I. n.. ,(VV % .~% %'V.1. V* .. , ,. .. ' .. '"
143 '.
positions along A2. For example, if the positions three and five are chosen,
the sub-coding along A1 Is 1-2-7, and the sub-coding along A2 is 5-4-3.
Each of these cities Is then exchanged, leading to the new tours, A* I and 4A*2: ';
A*I:7 1543682 82 i.
A*2:5 8 1 2 7 6 4 3.
They reported two experiments on ten cities where the PMX operator
enabled the search to efficiently find either the absolute or near optimal :.
solution. Goldberg and Lingle stated that this operator was more complex
than "simple crossover,' as proposed by Holland. As will be shown, In fact,
It Is not.
In another paper by J.J. Grefenstette, R. Gopal, B. Rosmaita and
D. Van Gucht,8 Hollands "simple crossover* was utilized. This required the
formation of a special coding structure. Clearly, using this operator on
two valid tours could result In an offspring' that was not a valid tour. As
A.K Dewdney 9 related, the authors' method for devising the appropriate
coding was Ingenious.
8- "Genetic Algorithms for the Traveling Salesman Probler," J.J. Grefenstette, R. opal, B.
Rosmalta, D. Ya Oucht, In Prxeings of n Inernatoal Conference on Ofnetic
Algorithms end Their AppIhwtions, J.J. Orefenstette, Editor, Carnegie-Mellon
University, July 1985.
9- 'Computer Recreations: Exploring the field of genetic algorithms In a primorilial computer
se full of flibs, AK. Dewdney, 5cientific American, November 1985.
6
144
"The representation for a five-city tour such as % c, e, o b turns out
to be 12321. To obtain such a numerical string reference Is made to
some standard order for the cities, say, a , c, d, e Given a tour such
as a, c, e, d, b, systematically remove cities from the standard list
in the order of the given tour-, remove a, then c, e and so on. As each
city Is removed from the special list, note Its position just before
removal, ais first, cIs second, els third, d Is second and, finally 0
is first. Hence the chromosome 12321 emerges. Interestingly, when
two such chromosomes are crossed over, the result Is always a tour."
Dewdney continued further to report that unfortunately the experiments
with this representation were "not very encouraging." The authors
conducted larger experiments than those of Goldberg and Lingle, Including
50, 100 and 200 cities. In the three reported experiments, after a large
number of trials (approximately 14000. 20000 and 25000, respectively),
the best tours were still far away from the expected optimal solutions.
At this point it Is natural to ask "why?'. After all, the traveling
salesman problem only requires discovery of a logical pattern. This seems
completely analogous to what occurs in nature. If the crossover of genes
works in natural evolution, why shouldn't It work here? The answer Is, in
fact, as noted In observation '2 by Goldberg and Lingle: The traveling
salesman problem is GA-hard -- difficult for the normal reproduction +
crsor+ mutation. This is due to the fact that Holland's crossover
operation d nl mimic the biologic crossover of genes.
As defined by C.K. Levy, 10 crossover Is the phenomenon where "old
10 - Elements Bof l R }, C.K. Levy, third edition, Addison-Wesley Publishing Company In., "'
Reading M ssehusettes, 1982.7
145
linkages between genes on homologous chromosomes are broken, and new
linkages are established. Genes that reside on the same chromosome and
move together are said to be 'finked.' A linkage group is any group of genes
physically linked on one chromosome." Levy goes on to state: "Cangesn
linkage groups are not truly mutations, however, since neither the amount
nor the function of genetic material is altered "'
Crossover allows for different combinations of alleles. Alleles, by
definition, control the same characteristic and occupy the same place on
similar chromosomes. Holland's crossover treats the entire tour as a
chromosome and each city In a tour as a gene. While Holland's crossover
does not change the amount of coding, It greatly alters the function of the
coding! A more appropriate biologic interpretation of a tour would be that
it is itself a gene. Crossover inside a gene is a nonsequetor. The tour Is
absolutely nmL analogous to a chromosome and each city In a tour is nt
analogous to a gene. These relations are in fact anomolous.
The result of Holland's crossover, therefore, is a near random search
throughout the entire space of possible tours. Perhaps Dewdney said it
best when he stated that by using crossover "there is so much juggling of
genes and cracking of chromosomes that...(a parent)...is hard put to
recognize its own grandchildren." This is, of course, the very essence of
the difficulty! Adaptive plans must retain already made advances In order
to ensure that an optimal solution will be found. As the number of cities
grows larger, Holland's crossover effectively destroys the link between
each parent and its offspring. The results can even be worse than a
*-Underline addd by this "uhor.
8
146
complete enumeration of all possible tours (reference the Addendum).
AN ALTERNATI VE APPROACH
As an alternative solution, consider the Adaptive Algorithm, so named
because it does not include any of the pseudo-genetic operators that
Holland has suggested. In this algorithm, which is equivalent to
evolutionary programming restricted to single state machines, only slight
mutations are made to an existing tour by removing just = city from a
given list and replacing It in a different randomly chosen position. This
mutation is only slightly more complicated than the simplest possible
mutation...swapping adjacent cities. It is clearly less complex than both
the PIX operator and Holland's crossover. Through multiple mutation, this
single alteration can be made equivalent to either of these crossover
operators.
According to Holland: "If successive populations are produced by
mutation alone (without (genetic) reproduction), the result is a random
sequence of structures drawn from (all possible structures)." I1 This is
only partly correct. The Adaptive Algorithm does result in a random
search, but only in a portion of the space relatively close to the parent
that generates the offspring. This Increases the effectiveness of the
algorithm dramatically as It allows for the retention of advances.
I I- aiptatin In Natural mdArtlfcial Systems,.J.H. HOWlln, University of Michigan Press,
Ann Arbor, 1975.
9.....................~ ~ - ,t
147
But more is still required. Not only must advances be retained but
"dead-ends" must be circumvented. Because there is a finite number of
of fspring that can be generated through mutation, there might well be
stagnation on a local optimum. To prevent this it is useful to randomly
alter the adaptive topography that is being searched. This can be
accomplished in various ways. One of these Is to occasionally allow for
the survival of offspring that are slightly worse than their parents. In
effect, the scoring function is made "noisy." What results is analogous to
the searching of a maze; when a dead-end is reached some backtracking Is
allowed and the overall search Is reinitiated.
Unfortunately, the topography is much like an upside-down bed of
nails, with some nails being longer (better) than others. From any given
nail, it is possible to travel to any of n(n- I) other nails In a single
mutation. Thus, unlike a maze, when the evolving phyletic line reaches a
non-optimal nail from which no single mutation results in a better tour, it
zeis Impossible to determine the "direction" in which to backtrack. A given
nail can be reached In a multitude of ways; therefore, bactracking is
allowed in any direction. To further aid In the circumvention of stagnation,. . the degree of "poorer quality" offsping that are occasionally accepted
should gradually be reduced as the adaptive process proceeds.
EXPERIMENTAL FINDINGS
0Experiments were performed to demonstrate the effectiveness of the
- "Adaptiv Toporphy" refers to the wing function on the rypersps of poiblei "pims."
4" .
°.'.. 1n
148
Adaptive Algorithm. Initially, 128 independent trials were performed on a
24 city traveling salesman problem where the cities were positioned on I
the periphery of a rectangle. Clearly, the minimum length tour is equal to
the perimeter of the rectangle, here, 250. Of the 128 trials, 90.625X found
the optimum solution in an average of 5297.48 Iterations, see Figure 1, the
maximum number of iterations being arbitrarily set at 14,000. Figure 2
Indicates the results of the remaining 9.375% of the trials. Here, at least
temporarily, the evolving tours were trapped on a local optimum. Despite
the seemingly non-complex arrangement of cities, the numerous local
optima make this traveling salesman problem difficult.
The cities were then distributed at random. First, 19 experiments
were conducted requiring a tour of 50 cities. The cities were redistributed
for each experiment. In each, no optimum tours were discovered in 20,000
iterations, but It was clear that the evolutionary process was "solving the
problem." Figures 3 through 21 indicate the results of each experiment
Figure 22 Indicates a typical example of the evolutionary process
discovering more and more suitable tours as offspring are evaluated. Note
that "backtracking" plays an Integral part of the search.
Experiments were then performed requiring a tour of 100 cities under
similar conditions. Again, while none of the eight experiments found a
perfect tour, the evolutionary process performed well. Figures 23 through
30 indicate the results of the eight experiments while Figure 31 indicates
the reduction in tour length as offspring are evaluated.
%a.
IIII
149 1
c 0
00
NN
oco
o00
0~ c;
.E 0.
o "* . .%
to_
o 0 DoCL-
o w-nv
Fiir
*'r--i-. .L-- -
150
0/) 0
L. 0
C-)
0o 0
0 x
re))
lb.-0
E E " o
0.0
0 -
xC
0I
00
C- o o o o o 0 0 0 0 0 0 :
xn 0
F-
Figure 2
-* - ~V ~*~F *t*~U~1 - - * ~ * 7 77 .-34-r 07 ~wJWe~ ~
151
U)C
0
00
l CCE E0ci) CL.x0
0 0 -
_0
c 1.0 '0 C)It) -4-'
E DD p~U)
o -o
Q) E
C- C) M) P- (DO It W1 ) cq '
Figure 3 .
W--4-L ar- 16F. -. - -
152
0cN-
II0)
>41%
0~0
C
E cE
C- m
Ec0
L...~~ 1 0 co I,-- c~o 0f ' '
Figure 4
153
0 - 0
(0
ICC)
0
0)0
E 00CL) -p
00L-..
o U~cc
o 0)C % DL *K *
CFigur 5
154
0
'0E co
0o
cic
L.C) 0
oL00
C-,
otC.)
0
C. ( 0 0 0 0 0 0 0 0
0 h o ~ to Lo qt cq
Figure 6
- -.- ~ ~ .p F W sP . .y ~ ~ W r~r~w - grrx -V W -w
155
ICC
L-,J
0-
ot
or
UZ)
0 C)
Fiur
156
IC
C'))
00
LOLLJ
CC
0l
m 0
0.EoO
ol ILF)n0
E Figure 8
- ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~~~~- W7917.- Ti~w~1U'~~~ V ~T%.i~~~* '~~ L - ~~
157
'I-,
4 J 0
0 0
Li
00
cc0
C)'0)0
E
C' 0 0 00 O cOCO~- 0 C) 00 -~ ED U*) 1q t
Figure 9
158 LI
Cl)
LC).0
-.
C) CO
00
0) 0
E C)
cc
0 0 0 0 0 c 0 0 00oC 1
Fiur 10
159
00
0
aC)
Q))
U) c
00
I:' C!0
E co 0
o 00 00 000 00 0000 ) CO r*- CO LO '1 to 04 ~
Figure I1I
- - - -. ~ ~ *FJ'~ VINP'~~ '~W~* ~ h 1- V ~l ~ -~7~~
160
Coo
- 0
LC) 1'
E0o to
CC
Co C)0 0
0tr 12
161
U
LC)0
00
E 0o
00
o -0
0
CCE -'-a
0*
ci)
Qo c
0 CE 0
o 0 M
Figure 13
162
ClC)
0
C)
o ) 0
Q0)
LLJ 0
0 -
L... 0-C"- C)
0
E C)
o) C.
00
0
O 0 00000000 00 0
Figure 14
163
U)
00
0 L.J
QL UL. 0
coC
W 0)) 00
Fiur 1
--- ------ A5 AMU-WT1Wqtwtflt ~ P~ , MflTnt . r- 5.= -2~~' r pU ~ I 72
164
Cl)0C'~i3
09i0
LC)
IQCL -- W
0) 00 "0x
CLdL.n o,
(D =:
o>>4'00
o" o) coC L
Fi(g) 0 e
165
0)0
LCJ)C:C
Qi)
00LD0
Cl- Ja0
0
o 0C ~ b- '.
E U
cn
~1) C0
CC
Cf) 0
0~ 00 0 0 0 00 00 0'- 0 0 co r- to i C) ' to N) C
Figure 17
* -3-.. - a~ -F U~! . -~~ ~ ~ '~r~N ~ Mk ~ MW".q~VL L
166
4-4'
Cl))ci)
Q) Q)0-0
LCD0~
a))L.O
00
0 p
0 0
0) 0 0 )000C D0C)C0) OCr)( n
Fiue1
167
C'c)
00
C0~~~~- (7 'p, C U) 't t N
Figure 1
168
E 0
E- 0
0'
0
O0O
0o
0-00
0 m0 0
Fiur 20C, CN1
E "
> ", 'E
Fiue2II _______
169
4-j4
ClC)
ESCL L .0-
U-)
E C)-Cl)
00
E
0 C C O 0 0 0 COO
Figure 21
*~ZW7 -w .. .- 7~ 7 - . . -
170
U)
C.) E.
LO >0
Eg 0 U)- 0 4 -
0,*0
0Qbi
0
C!))Lo -'-
C /6
CDJ
0 C 0 0 0
Figure 22
171
Cl))00
CC
0)
C/)00
£~ L C
EEQ) i
>1 .. . W4U .. '-
0~1 0
- 0 0 - O.q
Figure 23
.7 V~sJ w:-j.rr
172
C) 4-0
1"*0Q) U)x )
0-
LO
0 >2CC)
ci)n00
0 0000c0 00 00 0001~ 0 0) CO r- LC ) -.0, n cN
Figure 24
- - -- - - - - - -' a - .,tru ~wL~tr'.lr'. fl.~. WSt 09~t-. *. - t, -u ,W
173
Cl)CL
0
A0
to
ci)
Q) -(0 0 0 0 0 0
00
Fiue2
.~ i. -u~ ~ .~ L ~ i~ ~w - ~u -. it J w r i-L ~WT~t U . i. t.Ir ~k ~~ZVI~.V
174
ci)
- C)
o UoxI
a_ 0
E
0(
Q)Q
F.mI.- I1 A IA0 0 0 0
CFiur 26
175
C 'C
E QQ) C)
CL 0L0o0
0 Q) 0C) to
C!);
E
w- toN- l
Figure 27
1 176
I Cl)
0 c
QI 0Q
-4-' 0
I I 0
CD~~ 0
LL.
Ir- 0
Eci) 0n
C!):
'I0 000 0 00 0 00 00
II Figure 28
T-M .- -A - --l 47 -j W
177
U)
00 ,0
oJ
N C
LU
-E co
0- 0* (D
0 L 0
cc(1) 0
0 0
* C) 0 ~
00
I-0 '-t
U)p
Q).
Figure 29
178
IkU)
000
CL
I CO0 d a) 1
00
0 0)1to
CL 0Q
E: 000
00
00
Cf)4
(D C
1 000 0 00 000 00L.. '- 0 M r%, 0 0 1* n' N '
I Figure 30
179
0> -o
cE0
0 L
C*4 0
0 c0 4J
0C 00 _
004C00
0' 0
-0
U0 000 0 qt tn n NV0T
Figure 31E
180
Another experiment required a tour of 90 cities. Here, ten groups of
nine cities were randomly placed on the coordinate grid. The process was
allowed to evolve 32,000 offspring. While the optimum solution remained
undiscovered, it is of interest to note that the problem was evidently
addressed at two distinct levels. The evolutionary process initially solved1
the problem at a gross level, discovering the minimum tour btwen the
groU. of cities, see Figure 32. Insufficient time was allowed to sort out
the problem at a finer level of detail.
Finally, an extremely large traveling salesman problem was analyzed.
Here, 256 cities were randomly distributed. The previous results Indicated
that the Adaptive Algorithm would not discover the optimum solution;
however, in only 10,000 iterations It reduced the initial tour length by
roughly 50%. Figure 33 Indicates the surviving tour after evaluating
10,000 offspring while Figure 34 Indicates the success of the evolu-
I ltionary process in discovering better and better tours. The available
computation time limited the analysis, however the results were certainly
I encouraging.
Clearly, the Adaptive Algorithm is an effective method for addressing
Sthe traveling salesman problem. Several Important conclusions can be
drawn from the previous experiments:I" Sophisticated" mutation operations are not only unnecessary,
but are derimetal The experiments point up the necessity for
BiI,,,
%' . . A ..o.A S ~ ~ ~ *S
r~~vr ~r, -, f V fl~~ -. -. - - -' - -W -Tv LK u -f S W*Wf s -tw, .Ct..LL ~
181
Q)* -)
-4-
-o0
CD <D
<0Q)
Eo0Q) _
000 0
CC-
E_ LU00
E
LO) 0 tp 0O
Figure 32
-~ ~ ~ ''~W~' - V~ ~V ~W VVI.W YL~ V~.~ 2~ i W1Ju U l 'Wh ' r- kr JW U 1WV rVI.U W I . L-u.I LVL' .
182
U)0
000
0.(0)
LOO
(N o
EE
0)C C 0 -
0 0 -. w L t
Figure 3
183
ci)J
* 0 )
4_ __ _ __j__ __ _ __ _ C :
UU(00
0 0
0 o
o- 0-I
00E0
0>
0'
0CL4 000 w
F--
Figure 34
1 184IJ
maintaining a substantial link between parent and off spring. The
more sophisticated and complex mutation operations destroy this
link. The PMX operation may perform generally superior to
Holland's crossover operation because it tends to retain moreInformation during each generation.
. There is a beneficial effect of using a noisy payoff function.
The concept of a noisy payoff function is similar to that suggested
by S. Kirkpatrick, C.D. Gelatt and M.P. Vecchi, 12 for optimizing
simulated annealing, but it is not necessary to resort to such
specific analogies. In a constantly changing environment, the
rewards and penalties for different behaviors vary. The search for
I better and better solutions Is never ending. Evolution is acontinuous process with no truly optimum solution.
.. . Further, it appears unlikely that any specific noisy payoff
function exists that will allow discovery of the optimum solution
In every traveling salesman problem. Each such problem offers a
different adaptive topography; therefore the appropriate
distribution and amount of noise cannot be determined a priorl
Although no single Adaptive Algorithm can optimally solve e .:
traveling salesman problem, a unique Adaptive Algorithm can be developed
to address eaj= traveling salesman problem In a very efficient manner.
12 - "OpUmzUon by Similatd Anmaling." S. Kirkpatrick. CJ). 6eutt Jr., lP. Vecchi. Sdenee.
hMy 1953.
I ,.' . , , . .. ,, .., " '' , . ' ,,, . , , . ., • ,. ' ' ,'. . , . - - - -. , , ," .." . -..
185
References:
1. D. Ackley, 0A Connectionist, Agoithm for Geneic Search," In Procaigs of en7Interna&tional Con7ference w7 C&nvtc Algorithms &7d Their Applications , .J
Grefenstette, Editor, Carnegie-Mellon University, July 1985. -
2. A.K. Dewckwv, "Computer Recreatons. Exploring the field of genetic algorithmns In a .
primordial computer see full of flubs," Scientific American, November 198S.
3. L.J. Fogel, "Autonomous Automata," IndustrIei Aerch, February 1962.
4. L.J. Fogel, "On the Organization of Intellect." Ph.D. dissertation, U.C.LA 1964.
5. L.J. Fogel, A.J Owens, M.J. Wash, Artiffic/al /nleligelnce Through SlmulatWEvolultn John Wiley& Sonis, New York, 1966.
6. D.E. Goldberg, R. Lingle, Jr., *Alleles, Loci, and the Traveling Salesman Problem,0 In
Pr xeedfngs of an7International Ctenca on 08netfc Algor~ims and' Thir/A4pplications , J.J. Grefenstette, Editor, Carnegie-Mellon University, July 1985.IR
7. J.J Greustette, R. (3opal, B. Rosmaita, D. Van Oucht. "Genetic Algorithms for teTraveling Salesman Problem," In Prxings of an Internetional Corence on60notIc Alarihms and1 Their Applications , J.J. OeasteEditor, Carnegie-MellonUniversity, July 1985.
8. J.H. Holland. A~woalon? i97 Natural and Artificial Systems, University ofMichigan Press, Ann Arbor, 1975.
9. S. Kirkpatrick, C.D. Oelatt, Jr., M.P. Yecchi, 00ptimization tby Simulated Annealing,"
Sciuncs, May 1983.
10. C.K. Levy, Elemins of Siolcy, third edition, Addison-Wesley Publishing Company
Inc., Reading, Mp~ad ichust, 1982. .
F = ~~~~~~~~~~~~~~~~~~~~~~~ .. . .-~ ... ..j ..... T .., . ,,. ;7 : ., , - , . , .
186
Addendum!
A completely random search (with replacement) will take roughly
twice as long to find the optimum solution as an enumerative search
(without replacement). To show this, consider the following two theorems:
I Theorem 1: It there are B possible solutions and only one
optimum solution, the expected number of trials that must be
made before the optimum solution Is found, using an enumerative
Isearch, assuming one trial Is made at a time. Is equal to
(B+ 1)/2.IProof- In an enumerative search, sampling Is made without replacement
I The probability, therefore, of discovering the optimum solution on any
I given trial Is equal to the product of the probabilities of not discovering
the optimum solution on any prior trial multiplied by the reciprical of the
I number of untried solutions. The expected number of trials that would have
to be examined before finding the optimum solution would therefore be:
I -f W 1 B- 1 + 2'[(B- I )/B]'(B- I )- + 3'[(3- I )/13][(-2)/(B- 1 )](13-2) - 1
+ (B- 1 )[(13- 1 )/8[2/31[1/2] + B1[(B- I )/1]".[2/3]-[1/21 1
1 " I13- + 2'13- + 3"-3 + "'" 4 (B-I)'B"1 + 13-
B"1 '(I + 2+ 3 +"" + (B-1)+ B)
I 3-1[1303+1)/2]j (B+ I1)/2. Q.E.D.
I " , " . " , r . " . " . . ' " . " • . " . " 4 " . . " . " . " . . " - . " . " . ' . r ." . " .
Theorem 2: If there are B possible solutions and only one
optimum solution, the expected number of trials that must be
made before the optimum solution Is found. In a completely
random search, assuming one trial Is made at a time, Is equal to
B.
Proof" In a completely random search, sampling Is made with
Il replacement. The probability, therefore, of discovering the optimum
solution on any given trial Is equal to the product of the probabilities of
I not discovering the optimum solution on any previous trial multiplied by
1 the reciprical of the total number of possible solutions. The expected
number of trials that would have to be examined before finding the optimal
solution would therefore be:
I I xt(x) - .1 #-' 2.1(3-1)/3)-1 # 3.[(0-1)/I 2.f-1 .1 +
I1 '1 * 2"1(0-I)/B] 3)(3O-)/J)2. 013?
-. O.E.D.
!
I.R
" 'r ' ':+Y":¥" ' , "' . .'%''""' """ 'x +; ,":, ' ,' : ' '; ' 'g ' ' ' ; . '';+ ,%