Optimisation (Repaired)

8/10/2019 Optimisation (Repaired)

1/22

4. Global Optimisation

Optimisation methods aim to find the values of a set of related variable(s) in the objective

function that will produce the minimum or maximum value as required. There are two types of

objective function, deterministic and stochastic. When the objective function is a calculated

value in the model (deterministic), we simply find the combination of parameter values that

optimise this calculated value. When the objective function is simulated random variable, we

need to decide on some statistical measure associated with that variable that variable that

should be optimised. .then the optimisation alorithm must run a simulation for each set of

decision variables values and record the statistic. There are many optimisation methods

available in the literature and implemented in the commercial software.

The history of lobal optimisation beins from !"#$%s simulation based optimisation

research due to the invention of the enetic alorithm by &ohn 'olland !. * enetic alorithm

is a class of population based adaptive stochastic optimi+ation procedures, characterisin the

randomness in the optimisation process. The randomness may be present as either noise in

measurements or ontecarlo randomness in search procedure or both. The basic idea behind

the enetic alorithm is to mimic a simple picture of the -arwinian natural selection in order to

find a ood alorithm and involves the operation such as mutation%, selection%, and

evaluation% of the fitness repeatedly.

* little later, in !"#/, *imo T0rn introduced his 1 2lusterin alorithm3 of lobal

optimisation 4. The method improves upon the earlier local search alorithms that needed

multiple start% from several points distributed over the whole optimisation reion. 2lusterin

alorithm avoids the drawbac5 of the ulti6start (many startin points are used) convered to

same minimum. The 2lusterin method avoids this repeated determination of local minima.

This is realised in three steps, which may be iteratively used, (!) sample points in the reion of

interest (4) transform the sample to obtain the points rouped around the local minima and (7)

use clusterin technique to roup these points. 8tartin a sinle local optimisation from each

cluster would determine the local minima and, thus also the lobal minimum.

9ittle later in !"/7 another lobal optimisation alorithm namely 18imulated annealin

method3 was proposed to mimic the annealin process in metallury by :ir5patric5 et al 7,;.


2/22

that the system at any time is approximately in thermodynamic equilibrium. *s coolin

proceeds, the system becomes more ordered = the liquid free+es or the metal recrystalli+es and

attainin the round state at T>$. The simulated annealin method optimisation ma5es very

assumptions reardin the function to be optimised and therefore , it is quite robust with respect

to irreular surfaces.

* little later, in !"/? @red Alover B introduced his 1Tabu search3 method. This method in

some sense is close to 2lusterin alorithm. Alover attributes it%s oriin to about !"##. The

basic concept of Tabu search is overall approach to avoid entrainment in cycles by forbiddin

or penalisin moves which ta5e the solution, in the next iteration, to points in the solution space

previously visited ('ence 1tabu3). The tabu method was partly motivated by the observation

that human behaviour appears to operate with random element that leads to inconsistent

behaviour iven similar circumstances.

The pace of search in lobal optimisation by stochastic process accelerated considerably in

the !""$%s. arco -orio in his Ch- thesis ? introduced his 1*nt colony3 method of lobal

optimisation.


3/22

The method of differential equation (-E), another lobal optimisation method, rew out of

:enneth Crice%s attempts to solve 2hebychev polynomial fittin problem in !""?/. The

crucial idea behind -E is a scheme for eneratin trial parameter vectors.


4/22

8tep !F 8et 5> $ ,select an initial point $x

8tep 4F Cic5 a candidate point +5at random from G(x5)

8tep 7F Toss a coin with probability of 'E*- equal to min(!, ( )( )( ) ( ) Hk k kf z f x T

e ),.


5/22

The C8O was developed by :ennedy and Eberhart #.


6/22


7/22

There are many other improved versions of C8O in the literature such as co6evolutionary

particle swarm !4, hybrid particle swarm optimi+ation !7 and Nuantum behaved C8O.

Nuantum6behaved particle swarm optimisation (NC8O), which can be uaranteed theoretically

to find optimal solution in search space, has few control parameters !;6!#.

(pi,!,pi,4, pi,7,Lpi,n) of which the

coordinates are defined as

( ) ( )

( )

! , 4 ,

,! 4

, , ,

, !, 4, ,

! ! ! , ! is a uniform random number

i j $ j

i j

i j i j $ j

c P c P ! j nc c

or

! r P r P r

+= =+

= + =

L

(;.7)

We can obtain the position of the particle usin followin equation.

!

4 4

ij

i j ij

X ! ln

r

=

where r4 is a uniform random number and !ij is defined as the coordinates of the local

attractor. To evaluate the 9i,j(t), a lobal point called mean best position of the population is

introduced onto C8O

( ) ( ) ( )( ) ( ) ( ) ( )! 4 ,! ,4 ,! ! !

! ! !( ) , , , , , ,

' ' '

n i i i n

i i i

m t m t m t m t P t P t P t ' ' '= = =

= =

L L (;.;)

Where is the population si+e and Ci is the pbest position of the particle i. The values of

9i,j (t) is determined by

( ) ( ) ( ), ,4 .i j j i j t m t X t= (;.B)

and the position of the particle Pi,j is calculated as


8/22

( ) ( ) ( ) ( ), , ,!

! . .ln , 7 a uniform random number 7

i j i j j i jX t ! t m t X t rr

+ = =

(;.?)

The parameter Q is called contraction = expansion coefficient, which can be tuned to control

the converence speed of the alorithm. The value of Q is linearly chanin from ! to $.B when

the alorithm is runnin.

The NC8O alorithm is iven below.

!.


9/22

=

Xmbest!X ijjjij

!ln. , where )!,$(R * and parameter is called contraction6

expansion coefficient.

/. Depeat steps 46# until a stoppin criterion is satisfied or a pre6specified number of iterations

are completed.

4.3.3. Gaussian &utation in #uantum $eha%ed PSO

ost C8O alorithms use uniform probability distribution to enerate random numbers. Gew

approaches usin Aaussian, 2auchy and exponential probability distributions to enerate

random numbers to updatin the velocity equation of C8O have been proposed by various

researchers !#, 4/67$. Aeneratin random numbers usin Aaussian distribution sequences

with +ero mean and unit variance for the stochastic coefficients of C8O may provide a ood

compromise between the current points and a small probability of havin hiher amplitudes

which may allow particles to move away from the current point and escape from local minima

!#.

@irstly random numbers are enerated usin the absolute value of Aaussian probability

distribution with +ero mean and unit variance (G($,!). These new NC8O approaches combined

with mutation operator are described as follows !#

*pproach !F in NKC8O the term

( ) ( )

=+

txmbest!tx iii!

ln.T.!

is replaced by

( ) ( )

=+#

txmbest!tx iii!

ln.T.! , where A >G($,!)

*pproach 4F in AKC8O the term( )

( )! , 4 ,

,! 4

i j $ j

i j

c P c P !

c c

+= +

is replaced by ( ) ( ), ,

,i j $ j

i j#P $P !

# $+= +where >G($,!)


10/22

4.3.4. #uantum $ased PSO using &utation based on Chaotic Se'uences

2haos is a 5ind of characteristic of nonlinear systems, which is a bounded unstable dynamic

behaviour. The utili+in chaotic systems in optimisation methods have attracted increased

interests from various fields !B, 7!, 74. *s essential feature of chaotic systems is that sliht

chanes in the parameter or startin values for the data leads to vastly different future

behaviours. These behaviours can be analysed based on 9yapunov exponent (a quantitative

measure of chaos) attractor theory. The application of chaotic sequences instead of random

sequences in NKC8O is a powerful stratey to diversify the population and improve the

NKC8O%s performance by preventin the converence to local minima !#. *n interestin

dynamic systems evidencin chaotic behaviour is the Uaslavs5ii map whose equation is iven

by

[ ]

( )( ) ( )

( ) mod ( !) ( ),!

( ) cos 4 ! !r

% t % t v az t

z t % t e z t

= + +

= +

where mod is the modulus after division. The Uaslavs5ii map has a strane attractor with larest

9yapunov exponent for v > ;$$, r > 7, and a > !4.??"B. in this case, the values of

[ ]( ) !.$B!4,!.$B!4z t . NKC8O approach usin chaotic sequences based on Uaslavs5ii map

can be more capable of escapin from local optima than random number sequences !#.

+ =

+ =

+

=

=


11/22

Where !m(t) is the mutation rate, ris a value enerated accordin to a uniform probability

distribution in rane $,!,zj(t)is the value enerated for each desin variable j (j >!, 4, ..., n), n

is the n6dimensional optimi+ation problem. V is scale factor equal to !.$B!4, uj and ljare upper

and lower values ofjthdesin variable and t is the current eneration.

4.4 Genetic Algorithm Optimisation !GA"

The concept of the A* was developed by 'olland and his colleaues in the !"#$%s

!/.Techniques of A* is inspired by the evolutionist theory explainin the oriin of species.


12/22

characteristics of chromosomes. utation is enerally applied at ene level of the chromosome.

Gew chromosome produced by mutation will not be sinificantly different from the oriinal

chromosomeJ nevertheless mutation plays a critical role in A*. utation reintroduces enetic

diversity bac5 into the population and prevents the solution to trap into the local optimum.

Deproduction involves selection of chromosomes for the next eneration.


13/22

ijj

jii

xaaxx

xaaxx

)!(

)!(

R

R

+=

+=

Where ais random number between 6$.B and !.B.

&utation

* widely used mutation operator in real coded Aenetic alorithm is Gon Sniform utation

47. This mutation scheme of the alorithm is as follows. @rom a chromosome

( )iniiii xxxxx ,,,, 74! = the mutated chromosome ( )

!!

7

!

4

!

!

! ,,,, +++++ = iniiii xxxxx is created as

follows.

( )( )

+

=+otherwisexxix

rifxxixx

l

j

i

j

i

j

i

j

j

i

ji

j,

B.$,!

Where i is the current eneration number and r is a uniformly distributed random number

between $, !.l

j

j xx , are upper and lower bounds of thethj component of the mutated

chromosome respectively. The function ),( %i iven below ta5es values in the interval $, y.

b

'axIter

i

%%i

=

!

!),(

Where is a uniformly distributed random number in the interval $, !, 'axIter is the

maximum number of iterations and bis a parameter, determinin the strenth of the mutation

operator. B.

)ocal *echni'ue

This technique helps to concentrate the points in the reion 8 around the lobal minimum

4!. The procedures of the local technique are as follows.

(!)8elect a random number.

(4)2onstruct a train point

x usin the formula.


14/22

( ) jjbest

jjj %xx +=

! nj ,,4,! =

Where j is a random number in 6$.B, !.B andbest

jx is thethj component of the best

chromosomebestx

(7)Deplace the worst pointworstx in 8 with

x , if )()(

worstxfxf

4.4.2. (eal Coded GA ith )aplace Cross+o%er Operator

* new crossover operator which uses 9aplace distribution is proposed by . The density

function of 9aplace distribution is iven by

!( ) exp ,

4

x af x x

b b

= < <

and distribution function of 9aplace distribution is iven by

!exp

4( )

!! exp

4

x ax a

b, x

x ax a

b

= >

where a is called a location parameter and bX$

is termed as scale parameter. Ssin 9aplace crossover operator two offsprin%s,

( ) ( )! ! ! ! 4 4 4 4

! 4 ! 4, , , and , , ,n n% % % % % % % %= =L L are enerated from a pair of parents

( ) ( )! ! ! ! 4 4 4 4! 4 ! 4, , , and , , ,n nx x x x x x x x= =L L in followin way.

( )!

lo ,4

!lo( ),

4

a b

a b

= + >


15/22

The offsprin%s are iven by the equations

! ! ! 4

4 4 ! 4

i i i i

i i i i

% x x x

% x x x

= +

= +

for smaller values of b, offsprin%s are li5ely to be produce near the parents and for larer

values of b offsprin%s are expected to be produced far from the parents. This way the 9aplace

crossover operator exhibits self adaptive behaviour. Other crossover operator in the literature

is 'euristic crossover operator

&a,inen- Periau and *oi%anen &utation !&P*P"

4./. Constrained Optimisation problem

any search and optimisation problems in science and enineerin involve a number of

constraints which the optimal solution must satisfy. * constraint optimisation problem is

usually written as a nonlinear prorammin (G9C) problem.

n

x

-xxf ),(min

8ubject to

( )n

iii

j

i

xxxx

nibxa

kjxh

mix$

,,,

!,

,,4,!,$)(

,,4,!,$)(

4!

=

===

(Y)

Where f(x) is an objective function, i(x) and hj(x) are inequality and equality constaints

respectively, and aiand biare the search space upper bound and lower bound respectively for

xi. The formulation of the constraints is not restrictive, since an inequality constraint of the


16/22

form i(x) Z $ can also be represented as =i(x) [ $, and the equality constraint hj (x) > $ is

equal to two inequality constraints i(x) Z $ and i(x) [ $. The most common approach to

solvin the constraint optimisation problems is the use of a penalty function. The purpose of

usin the penalty function is to transform the continuous non linear prorammin (2G9C)

problem to the uncontraint G9C (SG9C) problem by buildin a sinle objective function and

penali+in the constraints. Then we can minimi+e the new sinle objective function usin the

unconstraint optimisation alorithm. This is main reason behind the concept of popular usae

of the penalty function approach. The drawbac5 of this approach is the difficulty to select

suitable penalty values.


17/22

Aradient repair method proposed by 2hootinan and 2hen 4B utilises radient information

obtained from the constraint set to systematically repair infeasible solutions by directin

infeasible solutions toward the infeasible reion. The steps of radient repair method is

summarised as below.

@irst determine the deree of constraint violation M, by equation (!)

{ }

=

=

)(

)(,$

!

!

xh

x$'in&

h

$&

k

m

Where M consists of vectors of inequality constraints and equality constraints h for the

problem.

2ompute &X , where &X are the derivatives of the constraints with respect to the

solution vector x is expressed by

&&xx&& xx == !

2ompute the oore6Censore inverse or pseudoinverse+

&X which is the approximate

inverse of &X to be used.

oore6Censore inverse is defined as ( ) TxxT

xx &&&& = + !

Spdate the solution vector by &&x&&xxxx xt

x

ttt ++=+= ++ !!

9oop throuh steps ! to ;

Constraint 0itness Priority+based ran,ing method

This method proposed by -on et al 4? solves the difficulty of selectin an appropriate

penalty value that appears in the penalty method. The method introduces constraint fitness

function 2f(x) for constraints, and it is computed from both inequality and equality constraints

as follows.

@or inequality constraint j(x) [ $


18/22

( )

( )

( )

( ) $

$

!

,!

)(

max

>

=

x$If

x$If

x$

x$x/j

jjj

Where max(x) >max\j(x), j>!, 4, 7,Lm] and 2j(x) is the fitness level of point x for constraint

condition (j)

@or equality constraint hj(x) [ $

( )( )

( )

( ) $

$

!

,!

)(

max

=

= xhIf

xhIf

xh

xhx/j

jjj

Where ( ) }kjxhh j ,,4,!,maxmax ==

The constraint fitness function evaluated at point x is equal to the weihted sum of

( ) jkm

wwx/wx/fj

km

jjj

km

jj +==

+

=

+

=

,!

$,!,)(!!

Where wj is a randomly enerated weiht for constraint j. the sum sinifies the fitness level of

point x as related to the feasible domain N. if 2f(x)>!, it is an indication that 0x , and on the

other hand , if $ ` 2f(x) `!, the smaller 2f(x) indicates that the solution x in infeasible domain

N or further away.

neasible degree selection

This method is currently applied in particle swarm optimisation method.


19/22

inversely proportion to the distance between solution xi and the feasible reion. To increase the

selection pressure with the evolutionary process, the threshold of infeasible deree in defined

as the product of a linearly decreasin ene and the averae infeasible deree value for

population (


20/22

!. &. 'olland, 1*daptation in Gatural and *rtificial 8ystems, Sniv. of ichian Cress,

*nn *rbor, !"#B

4. *. *. T0rn, 1* 8earch 2lusterin *pproach to Alobal Optimi+ation3, in -ixon, 92W

and 8+e0, A.C. (Eds) Towards Alobal Optimi+ation = 4, Gorth 'olland, *msterdam,

!"#/

7. 8. :itr5patric5, 2. -. Aelatt &r, and . C. Mecchi, 1Optimi+ation by 8imulated

*nnealin3, 8cience, 44$, ;B"/, ?#! = ?/$, !"/7

;. M. 2erny, 1Thermo dynamical *pproach to the Travelin 8alesman CroblemF *n

Efficient 8imulation *lorithm3, &. Opt. Theory *ppl., ;B, !, pp. ;! = B!, !"/B

B. @. Alover, 1@uture paths for


21/22

!B. 9eandro dos 8antos 2oelho, * Nuantum Carticle 8warm Optimi+er with 2haotic

mutation Operator%, 2haos, 8olitons and @ractals, 7#, 4$$/, !;$"6!;!/

!?. aolon Pi et al , Nuantum6 behaved Carticle 8warm Optimi+ation with Elitist ean

Kest Cosition%, 2omplex 8ystems and *pplications6 odellin, 2ontrol and

8imulations, !;(84), 4$$#, !?;76!?;#

!#. 9eandro dos 8antos 2oelho, Aaussian Nuantum6behaved Carticle 8warm

Optimi+ation *pproaches for 2onstrained Enineerin -esin Croblems%, Expert

8ystems with *pplications, 7#, 4$!$, !?#?6!?/7

!/. 'olland, &. '., *daptation in Gatural and *rtificial 8ystems%, *nn *rborF Sniversity

of ichian Cress, !"#B

!". Den UiWu, 8an e and 2hen &un @en, 'ybrid 8implex


22/22

4". :rohlin, D. *, Aaussian 8warmF * Goel Carticle Optimi+ation *lorithm%,

Croceedins of the

Optimisation (Repaired)

Documents

Transcript of Optimisation (Repaired)