Y. Tan, Y. Shi, and K.C. Tan (Eds.): ICSI 2010, Part I, LNCS 6145, pp. 541–548, 2010. © Springer-Verlag Berlin Heidelberg 2010

An Improved Thermodynamics Evolutionary Algorithm Based on the Minimal Free Energy

Fahong Yu*, Yuanxiang Li, and Weiqin Ying

State Key Lab of Software Engineering, Wuhan University, Wuhan 430072, China fhyu520@126.com

Abstract. In this paper, an improved thermodynamics evolutionary algorithm (ITEA) is proposed. The purpose of the new algorithm is to systematically har-monize the conflict between selective pressure and population diversity while searching for the optimal solutions. ITEA conforms to the principle of minimal free energy in simulating the competitive mechanism between energy and en-tropy in annealing process, in which population diversity is measured by simi-larity entropy and the minimum free energy is simulated with an efficient and effective competition by free energy component. Through solving some typical numerical optimization problems, satisfactory results were achieved, which showed that ITEA was a preferable algorithm to avoid the premature conver-gence effectively and reduce the cost in search to some extent.

Keywords: evolutionary algorithm, free energy component, similarity entropy.

1 Introduction

The evolutionary algorithm is wildly used in searching, optimization, and machine learning [1], in which connotative parallel character and efficiency of using global information are two prominent characteristics. However, it often suffers from a con-tradiction that premature convergence and low rate in searching, which is a tunable contradiction between exploration and exploitation in evolutionary algorithm.

There are a wide variety of improvements in evolutionary algorithms focused on harmonize the contradiction such as scaling the fitness, sharing the fitness, and driv-ing all individuals moving as the most possible [2] etc. Kirkpatric has proposed an-other general optimization algorithm called the simulated annealing (SA) [3] which controlled the process of searching systematically by the cooling temperature and the Metropolis rule. NDEA [5] is an algorithm based on information entropy defined in search space VSP divided to some grids while it can not be used to real-number opti-mization problem with high-dimension. Mori and his fellows have proposed the ther-modynamics genetic algorithm (TDGA) [6] combined SA and GA. They introduced a greedy thermodynamics selection rule based on the principle of minimal free energy in TDGA and attempted control to control population diversity systematically through tuning to temperature and entropy in annealing. Although TDGA was effective to * Corresponding author.

542 F. Yu, Y. Li, and W. Ying

solve some problems, the satisfied performance and extremely high cost in computa-tion were still unstable. To improve the stability and decrease the computational cost of TDGA, an improved thermodynamics evolutionary algorithm (ITEA) is proposed.

2 Description of ITEA

The second law of thermodynamics describes an isolated system that is always evolv-ing towards the direction of increasing entropy and its formula can be described in algebra as the express dS≥0, which means that the isolated system will evolve to-wards the stable status with the continually increasing of the entropy.

There is a useful connection between annealing in solids and convergence in GA. The population and the individuals in GA may be regarded as a thermodynamics system and particles, and the energy, entropy and temperature is served as the fitness, measurement of population diversity and controllable parameter respectively. This analogy provides an approach for ITEA to simulate the competitive mechanism be-tween energy and entropy in annealing to systematically harmonize the conflict be-tween selective pressure and population diversity in GA.

2.1 Similarity Entropy and Measurement of Population Diversity

It is a critical part how to measure population diversity when introducing the competi-tive mechanism into GA. Therefore, in order to measure population diversity more effectively, similarity entropy defined in search space is introduced.

Definition 1: Let Xo be the best individual in population P＝{X1, X2, …, Xn} ⊆ HD, Xi = ( xi1, xi2, …, xiD)T, and set = /i i iX X X when 0iX ≠ , The directional similarity

degree be defined as the express Si =1-XiTXo∈[0,1].

Definition 2: Let R=max{di|i=1, 2,…,n} be the maximal distance between the Xo and Xi

in population P＝{X1, X2, …, Xn} ⊆ HD and ( )/l D I D popsizeδ ⎡ ⎤= + +⎢ ⎥ , the variable I be

the longest span of boundary in decision-making space and δ be a parameter, set a parti-tion π={Lj|1≤j≤kΛj∈N } on field [0,1], kl =1 and L1ΛL2…ΛLk=Φ. The individual Xi belong to the partition Lj when (di/R+Si)/2∈[l(j-1)，lj). If the number that the individu-als in population P belonged to Lj is nj, the similarity entropy be defined as S(P).

1

( ) logk

j jk

j

n nS P

n n=

= −∑ . (1)

The higher degree the individuals in population P congregate, the easier convergence the optimization problem is. According to definition 1 and definition 2, the more near to zero Si is, the more similar the individual Xi and Xo in their directions and Si = 0 when Xi and Xo is the same completely in direction. In addition, di = 0 when Xi and Xo is in the same position. So the smaller the value of the express di/2R+Si/2 is, the indi-vidual Xi and Xo is more similar in direction and position and the more high the simi-lar degree of that two individuals. Since the variable nj(i=1, 2,…,k) is the number

An ITEA Based on the Minimal Free Energy 543

belonged to partition Lj for those individual in population P, the sum 1

k

ji

n=∑ is n. Simi-

larly, since the probability pj can be get by the formula nj/n., (j=1,2,…,k), the sum

1

k

jj

p=∑ is 1. Any change for the probability to be close to the same value will contribute

the similarity entropy to increase. If all value of the probability pj is i/k, (j=1,2,…,k), the similarity entropy S(P) is 1 that is its maximum. If only one value of the probabil-ity is 1 and the others is 0, the similarity entropy S(P) is 0 that is it’s the minimum.

By similarity entropy, the process in calculating is simplified greatly and the diversity of population can be measured effectively. The calculation of similarity entropy on similarity degree and distance has avoided the complicated calculation on every gene.

2.2 Minimization of Free Energy at Each Temperature

In order to apply the principle of minimal free energy in thermodynamics to ITEA, the method of free energy of population is presented as follow.

Definition 3: For Pt＝{X1,X2,…,Xn}∈ nH , the energy of population Pt be defined as

E(Pt).

1

( ) ( )n

t ii

E P f Xn

μ=

= ∑ . (2)

The value of variable μ will be set -1 if the optimized problem is minimum max (f(X)) or the value of variable μ will be set -1.

Definition 4: For Pt＝{X1,X2,…,Xn}∈ nH , π={Lj|1≤j≤kΛj∈N }, F(π ,T, Pt) be called

the free energy of population Pt at temperature T for partition π .

F(π,T,Pt)=E(Pt)-TS(π,Pt) (3)

The aim of algorithm is to minimize the free energy of population at each temperature during the process of evolution so as to drive the thermodynamic system towards equilibrium state.

In order to minimize the free energy rapidly at each temperature, a thermodynam-ics selection rule to descend the free energy of the next generation most steeply should be designed. Its mission is to select n individuals from n parent individuals and m offspring individuals as the next generation with the minimum free energy. How-ever, It is infeasible to exactly minimize the free energy for each generation because of the extremely high complexity O(nC n+m

n) . Hence, TDGA uses a greedy thermo-dynamics selection (GTS) rule with the complexity O(mn2). But its reliability can’t be guaranteed. In this section, a competition of free energy components (CFC) by assign-ing the free energy of the population to its individuals will be proposed.

Definition 5: Let P＝{X1,X2,…,Xn}∈HD, π={Lj|1≤j≤kΛj∈N }. For an individual Xi belong to the partition Ld, the free energy of the individual Xi in P at temperature T for π is defined as Fc(π, T, P, Xi).

544 F. Yu, Y. Li, and W. Ying

( , , , ) ( ) log ( / )c i i k dF T P X f X T n nπ μ= + (4)

It is clear that the time complexity is lowed to O((n+m)m). The procedure of competi-tion by free energy component CFC(π, T, Pt ) is as follows:

1. Produce an interim population Ptn+m of size n+m by appending the offspring popu-

lation Ot to the parent population Ptn, calculate a partition according to definition 1

and definition 2 and count particles in each partition. 2. Calculate the free energy Fc(π, T, Pt

n+m, Xi) according to the formula (4). 3. Select individuals with the minimum free energy to serve as a next population Pt+1

n from population Pt

n+m .

2.3 Procedure of ITEA

Procedure of ITEA is as follows:

1. Initial parameters. i.e., population size n, T0, MAXGENS, offspring population size m, randomly generate P0;

2. Calculate the number of field partition k; 3. t=0; 4. while Termination_test(Pt) < > true do

5. Generate offspring population Ot with m individuals by crossover and mutation; 6. Pt+1= CFC(π, T, Pt); 7. t++, Tt=T0 /(1+t/100); 8. endwhile 9. Output the final results.

3 Experiments and Results Analysis

The significance of a new optimization strategy is applied to solve practical problems depends effectively. Here chose two functions as test examples, which are often used for optimization of the test by many scholars at home and abroad.

Schaffer:

( )( )

2 2 21 2

22 21 2

sin 0.5max ( ) 0.5

1 0.001

x xf x

x x

+ −= −

⎡ ⎤+ +⎣ ⎦

-100 x 100≤ ≤

Schaffer [10] function is a strong oscillatory behavior function with multiple local maximization and a global maximization is maxf(x)=1 when X=(0, 0).

Needle in haystack (NiH):

2

2 2 21 2 2 2

1 2

3.0max ( ) ( )

0.05 ( )f x x x

x x

⎛ ⎞= + + ⎜ ⎟+ +⎝ ⎠

-5.12 x 5.12≤ ≤

An ITEA Based on the Minimal Free Energy 545

NiH is a typical kind of strong deceptive problem with the global maximization is maxf(x)=3600 in central point and multiple local maximization near the point X=(±5.12,±5.12).

In this experiment, the individual was expressed in vector X = ( x1, x2, …, xD) with real-number encode and four algorithms included simple genetic algorithm (SGA), niche genetic algorithms (NGA), greedy thermodynamic evolutionary algorithm (GTEA) and ITEA were applied. In order to compare rationally, a uniform circum-stance were adopted in every algorithm, which include the same machine, the same arithmetic crossover operator with probability Pc=0.8, mutation operator with prob-ability Pm=0.02, population size n=30, were utilized. The other parameters were set as follow: the niche radius on NGA r=4, the offspring population size m=4 and the length of Markov under the temperature Tk is Lk=2000 for GTEA and ITEA. The worst results, the mean results, the best results and first hitting time respectively for 30 times were gotten through four algorithms carried out on functions Schaffer and NiH. The compare of statistic results are given as table 1.

Table 1. Statistic results on Schaffer and NiH for four algorithms

Instance Algorithm Worst Mean Best First hitting time

SGA 0.962776005 0.987615296 1.0000000000 >4.464 NGA 0.999999821 0.999999961 1.0000000000 =5.626 GTEA 0.990284144 0.996599451 1.0000000000 >665.248 Schaffer

ITEA 1.000000000 1.000000000 1.0000000000 =1.491 SGA 2748.782226562 3344.634667969 3600.0000000000 >1.415 NGA 2748.782226562 3004.147558594 3600.0000000000 >17.298 GTEA 2748.782226562 3131.830224609 3600.0000000000 >696.211 NiH

ITEA 3600.000000000 3600.000000000 3600.0000000000 =0.807

There were some statistic results included the worst results, the mean results, the best results and first hitting time respectively for 30 times in table I. The first hitting time implies that the first time to achieve f(x)≥0.999 for Schaffer and f(x)≥3599.9 for NiH. According to the count s for every individual being taken part to evaluate re-spectively before the best solution were get at the first time on four algorithms carried on Schaffer and NiH every time, the performance can be evaluated on four ranks contained that excellent (s≤8.0×104), good (8.0×104<s≤ 3.2×105), weak (3.2×105<s≤1.28×106) and bad (s>1.28×106) and the distribution of algorithm per-formance on Schaffer and NiH were described as fig.1, in which the vertical axes represent the performance ratio and the level axes represent the ranks. The average convergence curves on Schaffer and NiH were described as fig.2.

In addition, the effect caused by offspring population size on ITEA was tested as table 2 and fig.3.

Viewing from results of the experiment, some conclusion can be drawn that ITEA could obtain very good accuracy, stability and faster rate to converge comparing other three algorithms for its strong search capabilities in every generation, reliability to main-tain the right search direction and ability to avoid premature convergence effectively. So it is regarded as a very important algorithm for some engineering optimization problems.

546 F. Yu, Y. Li, and W. Ying

Excellent Good Weak Bad0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Score

Rat

io

SGANGAGTEAITEA

(a)

Excellent Good Weak Bad0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Score

Ratio

SGANGAGTEAITEA

(b)

Fig. 1. Distribution of algorithm performance on Schaffer (a) and NiH (b)

0 2 4 6 8 10 12

x 105

0.986

0.988

0.99

0.992

0.994

0.996

0.998

1

Evaluations

Fitn

ess

SGANGAGTEAITEA

(a)

0 2 4 6 8 10 12

x 105

2900

3000

3100

3200

3300

3400

3500

3600

Evaluations

Fitn

ess

SGANGAGTEAITEA

(b)

Fig. 2. Convergence curves on Schaffer (a) and NiH (b)

Table 2. Statistic results on Schaffer and NiH for ITEA with various offspring population sizes

Instance size Worst Mean Best First hitting time

4 1.0000000000 1.0000000000 1.0000000000 =1.491 8 0.9999999404 0.9999999970 1.0000000000 =0.202 Schaffer 16 0.9626941681 0.9908478171 1.0000000000 >4.445 4 3600.0000000000 3600.0000000000 3600.0000000000 =0.807 8 2748.7822265625 3514.8782226563 3600.0000000000 >1.597 NiH 16 2748.7822265625 2958.2632568359 3600.0000000000 >4.208

The main reason why the outstanding performance could be archived for ITEA lies in its adaptive capability to maintain the dynamic balance between “selective pres-sure” and “species diversity” by adjusting the temperature. Generally, in competition between “selective pressure” and “species diversity”, the entropy occupies the pre-dominance for high temperature in the early stages of evolution which lows level of aggregation and energy plays main function for low temperature in the latter evolu-tion which allows a higher degree of aggregation contributing to converge to the op-timization solution set.

An ITEA Based on the Minimal Free Energy 547

0 2 4 6 8 10 12

x 105

0.984

0.986

0.988

0.99

0.992

0.994

0.996

0.998

1

Evaluations

Fitn

ess

M=4M=8M=16

(a)

0 2 4 6 8 10 12

x 105

2800

2900

3000

3100

3200

3300

3400

3500

3600

Evaluations

Fitn

ess M=4

M=8M=16

(b)

Fig. 3. Convergence curves on Schaffer (a) and NiH (b) for ITEA with various offspring popu-lation sizes

4 Conclusion

In order to systematically harmonize the conflict between selective pressure and population diversity, an improvement of thermodynamics evolutionary algorithm was proposed. In ITEA, the definition of similarity entropy and the application of free energy component thermodynamic selection strategy were given. ITEA simulates the competitive mechanism between energy and entropy in annealing to harmonize the conflict between selective pressure and population diversity effectively by tempera-ture and reduced the complexity of the algorithm to some extent. By solving some typical numerical optimization problems, the efficiency and good performance of the ITEA were achieved. Therefore, ITEA avoids premature convergence successfully and obtains faster rate, good accuracy and stability.

Acknowledgment. This work was supported by the National Natural Science Foun-dation of China under Grant No. 60773009.

References

1. Pan, Z.J., Kang, L.S., Chen, Y.P.: Evolutionary computation. Tsinghua Uinv. Press, Beijing (1998) (in Chinese)

2. Michalewicz, Z., Fogel, D.: How to Solve It: Modern Heuristics. Springer, Berlin (2000) 3. Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Sci-

ence 220, 671–680 (1983) 4. Vavak, F., Fogarty, T.C.: A comparative study of steady state and generational genetic algo-

rithms for use in nonstationary environments. In: Fogarty, T. C. (ed.) AISB workshop on Evolutionary Computing, pp. 297–304. Springer, Brighton (1996)

5. Hu, T., Li, Y.X., Ding, W.: A new dynamical evolutionary algorithm based on the principle of minimal free energy. In: Kang, L.S., Cai, Z.H., Yan, X.S. (eds.) Progress in Intelligence Computation and Applications, pp. 749–754. China University of Geosciences, Wuhan (2005)

6. Mori, N., Yoshida, J., Tamaki, H., Kita, H., Nishikawa, Y.: A thermodynamic selection rule for the genetic algorithm. In: Fogel, D.B. (ed.) Proc. of IEEE Conf. on Evolutionary Com-putation, pp. 188–192. IEEE Press, New Jersey (1995)

548 F. Yu, Y. Li, and W. Ying

7. Ying, W.Q., Li, Y.X.: Improving the Computational Efficiency of Thermodynamical Ge-netic Algorithms. Journal of Software, 1613–1622 (2008)

8. Su, X.H., Yang, B., Wang, Y.D.: A Genetic Algorithm Based on Evolutionarily Stable Strategy. Journal of Software 14(11), 1863–1868 (2003) (in Chinese)

9. Abs da Cruz, A.V., Vellasco, M.M.B.R., Pacheco, M.A.C.: Quantum-inspired evolutionary algorithm for numerical optimization. In: Yen, G.G., et al. (eds.) Proc. of IEEE Congress on Evolutionary Computation, pp. 2630–2637. IEEE Press, New Jersey (2006)